public inbox for [email protected]
help / color / mirror / Atom feedRe: Performance issues during pg_restore -j with big partitioned table
2+ messages / 2 participants
[nested] [flat]
* Re: Performance issues during pg_restore -j with big partitioned table
@ 2025-04-02 17:39 Adrian Klaver <[email protected]>
0 siblings, 1 reply; 2+ messages in thread
From: Adrian Klaver @ 2025-04-02 17:39 UTC (permalink / raw)
To: Dimitrios Apostolou <[email protected]>; [email protected]
On 4/2/25 10:32 AM, Dimitrios Apostolou wrote:
> Hello list.
>
> My database includes one table with 1000 partitions, all of them rather
> sizeable. I run:
>
> pg_restore -j12 --no-tablespaces --disable-triggers --exit-on-error
> --no-owner --no-privileges -n public -d newdb custom_format_dump.pgdump
>
> Right now after 24h of restore, I notice weird behaviour, so I have
> several questions about it:
>
> + 11 postgres backend processes are sleeping as "TRUNCATE TABLE waiting".
> I see that they are waiting to issue a TRUNCATE for one of the
> partitions and then COPY data to it. Checking the log I see that
> several partitions have already been copied finished, but many more
> are left to start.
>
> Why is a TRUNCATE needed at the start of a partition's COPY phase? I
> didn't issue a --clean on the command line (I don't need it as my
> database is newly created), and I don't see a mention of related
> TRUNCATE in the pg_restore manual.
--clean will drop the object entirely not TRUNCATE.
I'm guessing that this is being done by you per:
https://www.postgresql.org/message-id/53760c70-4a87-a453-9e02-57abc9cb2e54%40gmx.net
"After each failed attempt, I need to issue a TRUNCATE table1,table2,...
before I try again. "
>
>
> Thanks in advance,
> Dimitris
>
>
--
Adrian Klaver
[email protected]
^ permalink raw reply [nested|flat] 2+ messages in thread
* Re: Performance issues during pg_restore -j with big partitioned table
@ 2025-04-02 17:46 Dimitrios Apostolou <[email protected]>
parent: Adrian Klaver <[email protected]>
0 siblings, 0 replies; 2+ messages in thread
From: Dimitrios Apostolou @ 2025-04-02 17:46 UTC (permalink / raw)
To: Adrian Klaver <[email protected]>; +Cc: [email protected]
On Wed, 2 Apr 2025, Adrian Klaver wrote:
>
>
> On 4/2/25 10:32 AM, Dimitrios Apostolou wrote:
>> Hello list.
>>
>> My database includes one table with 1000 partitions, all of them rather
>> sizeable. I run:
>>
>> pg_restore -j12 --no-tablespaces --disable-triggers --exit-on-error
>> --no-owner --no-privileges -n public -d newdb custom_format_dump.pgdump
>>
>> Right now after 24h of restore, I notice weird behaviour, so I have
>> several questions about it:
>>
>> + 11 postgres backend processes are sleeping as "TRUNCATE TABLE waiting".
>> I see that they are waiting to issue a TRUNCATE for one of the
>> partitions and then COPY data to it. Checking the log I see that
>> several partitions have already been copied finished, but many more
>> are left to start.
>>
>> Why is a TRUNCATE needed at the start of a partition's COPY phase? I
>> didn't issue a --clean on the command line (I don't need it as my
>> database is newly created), and I don't see a mention of related
>> TRUNCATE in the pg_restore manual.
>
> --clean will drop the object entirely not TRUNCATE.
>
> I'm guessing that this is being done by you per:
>
> https://www.postgresql.org/message-id/53760c70-4a87-a453-9e02-57abc9cb2e54%40gmx.net
>
> "After each failed attempt, I need to issue a TRUNCATE table1,table2,...
> before I try again. "
Thanks Adrian. I'm now testing restore without --data-only. All I'm doing
prior to the above pg_restore command is "createdb -T template0 newdb".
It's possible though that I'm missing something here, the whole thing is
way more complicated than I expected...
Dimitris
^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2025-04-02 17:46 UTC | newest]
Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2025-04-02 17:39 Re: Performance issues during pg_restore -j with big partitioned table Adrian Klaver <[email protected]>
2025-04-02 17:46 ` Dimitrios Apostolou <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox