pg.ddx.io pgsql-general@postgresql.org mailing list archive
help / color / mirror / Atom feedpg_upgrade and schema complexity...
6+ messages / 3 participants
[nested] [flat]
* pg_upgrade and schema complexity...
@ 2023-06-03 00:44 Ron <ronljohnsonjr@gmail.com>
0 siblings, 2 replies; 6+ messages in thread
From: Ron @ 2023-06-03 00:44 UTC (permalink / raw)
To: pgsql-general
Ran into this when upgrading from 13.11 to 15.3...
The pg_restore phase failed with "ERROR: out of shared memory", and
recommended that I increase max_locks_per_transaction. Doing so let the
process run to completion.
It took 12.5 minutes to upgrade a 13GB instance. Soon after, I upgraded a
78GB cluster, and it only took 3.1 minutes.
(Both are VMs (same number of CPUs and RAM) connected to the same SAN.)
A "pg_dump --schema-only" of the two systems shows that the small-but-slow
schema is 5.9M lines.
Is this to be expected of such a huge schema?
--
Born in Arizona, moved to Babylonia.
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: pg_upgrade and schema complexity...
@ 2023-06-03 00:58 Adrian Klaver <adrian.klaver@aklaver.com>
parent: Ron <ronljohnsonjr@gmail.com>
1 sibling, 1 reply; 6+ messages in thread
From: Adrian Klaver @ 2023-06-03 00:58 UTC (permalink / raw)
To: Ron <ronljohnsonjr@gmail.com>; pgsql-general
On 6/2/23 17:44, Ron wrote:
> Ran into this when upgrading from 13.11 to 15.3...
>
> The pg_restore phase failed with "ERROR: out of shared memory", and
> recommended that I increase max_locks_per_transaction. Doing so let the
> process run to completion.
>
> It took 12.5 minutes to upgrade a 13GB instance. Soon after, I upgraded
> a 78GB cluster, and it only took 3.1 minutes.
Where/how did you measure those sizes?
>
> (Both are VMs (same number of CPUs and RAM) connected to the same SAN.)
>
> A "pg_dump --schema-only" of the two systems shows that the
> small-but-slow schema is 5.9M lines.
Anything special you are doing in this cluster to create all those lines?
What is the line count for the other instance?
>
> Is this to be expected of such a huge schema?
>
--
Adrian Klaver
adrian.klaver@aklaver.com
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: pg_upgrade and schema complexity...
@ 2023-06-03 01:06 Ron <ronljohnsonjr@gmail.com>
parent: Adrian Klaver <adrian.klaver@aklaver.com>
0 siblings, 1 reply; 6+ messages in thread
From: Ron @ 2023-06-03 01:06 UTC (permalink / raw)
To: pgsql-general
On 6/2/23 19:58, Adrian Klaver wrote:
> On 6/2/23 17:44, Ron wrote:
>> Ran into this when upgrading from 13.11 to 15.3...
>>
>> The pg_restore phase failed with "ERROR: out of shared memory", and
>> recommended that I increase max_locks_per_transaction. Doing so let the
>> process run to completion.
>>
>> It took 12.5 minutes to upgrade a 13GB instance. Soon after, I upgraded
>> a 78GB cluster, and it only took 3.1 minutes.
>
> Where/how did you measure those sizes?
Does it really matter?
START_SECS=$(date +"%s")
pg_upgrade ...
FINISH_SECS=$(date +"%s")
ET=`echo "scale=2;(${FINISH_SECS} - ${START_SECS})/60" | bc`
date +"%F %T pg_upgrade finished. Elapsed time: ${ET} minutes."
(Text copied between air-gapped computers, so there might be errors.)
>>
>> (Both are VMs (same number of CPUs and RAM) connected to the same SAN.)
>>
>> A "pg_dump --schema-only" of the two systems shows that the
>> small-but-slow schema is 5.9M lines.
>
> Anything special you are doing in this cluster to create all those lines?
I do nothing; the schema is provided by the vendor.
>
> What is the line count for the other instance?
227K rows.
>>
>> Is this to be expected of such a huge schema?
>>
--
Born in Arizona, moved to Babylonia.
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: pg_upgrade and schema complexity...
@ 2023-06-03 01:08 Christophe Pettus <xof@thebuild.com>
parent: Ron <ronljohnsonjr@gmail.com>
1 sibling, 0 replies; 6+ messages in thread
From: Christophe Pettus @ 2023-06-03 01:08 UTC (permalink / raw)
To: Ron <ronljohnsonjr@gmail.com>; +Cc: pgsql-general
> On Jun 2, 2023, at 17:44, Ron <ronljohnsonjr@gmail.com> wrote:
> Is this to be expected of such a huge schema?
pg_upgrade time is pretty much proportional to the number of database objects in the schema, so a much larger schema taking much longer is to be expected.
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: pg_upgrade and schema complexity...
@ 2023-06-03 02:22 Adrian Klaver <adrian.klaver@aklaver.com>
parent: Ron <ronljohnsonjr@gmail.com>
0 siblings, 1 reply; 6+ messages in thread
From: Adrian Klaver @ 2023-06-03 02:22 UTC (permalink / raw)
To: Ron <ronljohnsonjr@gmail.com>; pgsql-general
On 6/2/23 18:06, Ron wrote:
> On 6/2/23 19:58, Adrian Klaver wrote:
>> On 6/2/23 17:44, Ron wrote:
>>> Ran into this when upgrading from 13.11 to 15.3...
>>>
>>> The pg_restore phase failed with "ERROR: out of shared memory", and
>>> recommended that I increase max_locks_per_transaction. Doing so let
>>> the process run to completion.
>>>
>>> It took 12.5 minutes to upgrade a 13GB instance. Soon after, I
>>> upgraded a 78GB cluster, and it only took 3.1 minutes.
>>
>> Where/how did you measure those sizes?
>
> Does it really matter?
>
> START_SECS=$(date +"%s")
> pg_upgrade ...
> FINISH_SECS=$(date +"%s")
> ET=`echo "scale=2;(${FINISH_SECS} - ${START_SECS})/60" | bc`
> date +"%F %T pg_upgrade finished. Elapsed time: ${ET} minutes."
Unless I'm not mistaken the above is how the elapsed time was measured.
I was looking for the procedure for determining the size.
>
> (Text copied between air-gapped computers, so there might be errors.)
>
>>>
>>> (Both are VMs (same number of CPUs and RAM) connected to the same SAN.)
>>>
>>> A "pg_dump --schema-only" of the two systems shows that the
>>> small-but-slow schema is 5.9M lines.
>>
>> Anything special you are doing in this cluster to create all those lines?
>
> I do nothing; the schema is provided by the vendor.
Alright so it is not your design, but you do have an idea of what is in
the database correct?
>
>>
>> What is the line count for the other instance?
>
> 227K rows.
>
>>>
>>> Is this to be expected of such a huge schema?
>>>
>
--
Adrian Klaver
adrian.klaver@aklaver.com
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: pg_upgrade and schema complexity...
@ 2023-06-03 13:51 Ron <ronljohnsonjr@gmail.com>
parent: Adrian Klaver <adrian.klaver@aklaver.com>
0 siblings, 0 replies; 6+ messages in thread
From: Ron @ 2023-06-03 13:51 UTC (permalink / raw)
To: pgsql-general
On 6/2/23 21:22, Adrian Klaver wrote:
> On 6/2/23 18:06, Ron wrote:
>> On 6/2/23 19:58, Adrian Klaver wrote:
>>> On 6/2/23 17:44, Ron wrote:
>>>> Ran into this when upgrading from 13.11 to 15.3...
>>>>
>>>> The pg_restore phase failed with "ERROR: out of shared memory", and
>>>> recommended that I increase max_locks_per_transaction. Doing so let the
>>>> process run to completion.
>>>>
>>>> It took 12.5 minutes to upgrade a 13GB instance. Soon after, I
>>>> upgraded a 78GB cluster, and it only took 3.1 minutes.
>>>
>>> Where/how did you measure those sizes?
>>
>> Does it really matter?
>>
>> START_SECS=$(date +"%s")
>> pg_upgrade ...
>> FINISH_SECS=$(date +"%s")
>> ET=`echo "scale=2;(${FINISH_SECS} - ${START_SECS})/60" | bc`
>> date +"%F %T pg_upgrade finished. Elapsed time: ${ET} minutes."
>
> Unless I'm not mistaken the above is how the elapsed time was measured. I
> was looking for the procedure for determining the size.
du -cm -d1 $DATA13
du -cm -d1 $DATA15
>
>>
>> (Text copied between air-gapped computers, so there might be errors.)
>>
>>>>
>>>> (Both are VMs (same number of CPUs and RAM) connected to the same SAN.)
>>>>
>>>> A "pg_dump --schema-only" of the two systems shows that the
>>>> small-but-slow schema is 5.9M lines.
>>>
>>> Anything special you are doing in this cluster to create all those lines?
>>
>> I do nothing; the schema is provided by the vendor.
>
> Alright so it is not your design, but you do have an idea of what is in
> the database correct?
"In" like the /purpose/ of the database? Sure.
"In" like the /design/ of the database? No; that's all on the vendor.
>
>
>>
>>>
>>> What is the line count for the other instance?
>>
>> 227K rows.
>
--
Born in Arizona, moved to Babylonia.
^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2023-06-03 13:51 UTC | newest]
Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-06-03 00:44 pg_upgrade and schema complexity... Ron <ronljohnsonjr@gmail.com>
2023-06-03 00:58 ` Adrian Klaver <adrian.klaver@aklaver.com>
2023-06-03 01:06 ` Ron <ronljohnsonjr@gmail.com>
2023-06-03 02:22 ` Adrian Klaver <adrian.klaver@aklaver.com>
2023-06-03 13:51 ` Ron <ronljohnsonjr@gmail.com>
2023-06-03 01:08 ` Christophe Pettus <xof@thebuild.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox