public inbox for [email protected]  

help / color / mirror / Atom feed

From: Dimitrios Apostolou <[email protected]>
To: Adrian Klaver <[email protected]>
Cc: [email protected]
Subject: Re: Performance issues during pg_restore -j with big partitioned table
Date: Wed, 2 Apr 2025 19:46:43 +0200 (CEST)
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
	<[email protected]>

On Wed, 2 Apr 2025, Adrian Klaver wrote:

>
>
> On 4/2/25 10:32 AM, Dimitrios Apostolou wrote:
>>  Hello list.
>>
>>  My database includes one table with 1000 partitions, all of them rather
>>  sizeable. I run:
>>
>>    pg_restore -j12 --no-tablespaces --disable-triggers --exit-on-error
>>  --no-owner --no-privileges -n public -d newdb custom_format_dump.pgdump
>>
>>  Right now after 24h of restore, I notice weird behaviour, so I have
>>  several questions about it:
>>
>>  + 11 postgres backend processes are sleeping as "TRUNCATE TABLE waiting".
>>     I see that they are waiting to issue a TRUNCATE for one of the
>>     partitions and then COPY data to it.  Checking the log I see that
>>     several partitions have already been copied finished, but many more
>>     are left to start.
>>
>>     Why is a TRUNCATE needed at the start of a partition's COPY phase? I
>>     didn't issue a --clean on the command line (I don't need it as my
>>     database is newly created), and I don't see a mention of related
>>     TRUNCATE in the pg_restore manual.
>
> --clean will drop the object entirely not TRUNCATE.
>
> I'm guessing that this is being done by you per:
>
> https://www.postgresql.org/message-id/53760c70-4a87-a453-9e02-57abc9cb2e54%40gmx.net
>
> "After each failed attempt, I need to issue a TRUNCATE table1,table2,...
> before I try again. "

Thanks Adrian. I'm now testing restore without --data-only. All I'm doing
prior to the above pg_restore command is "createdb -T template0 newdb".
It's possible though that I'm missing something here, the whole thing is
way more complicated than I expected...


Dimitris

view thread (2+ messages)

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected]
  Subject: Re: Performance issues during pg_restore -j with big partitioned table
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox