Feedback-ID: i76984098:Fastmail
Message-ID: <7d3fc27a-1229-4739-8d15-f4005dfa2435@aklaver.com>
Date: Wed, 2 Apr 2025 10:39:36 -0700
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Performance issues during pg_restore -j with big partitioned
 table
To: Dimitrios Apostolou <jimis@gmx.net>, pgsql-general@lists.postgresql.org
References: <84379bfb-bbda-84e6-cacc-e863ba9d6c37@gmx.net>
Content-Language: en-US
From: Adrian Klaver <adrian.klaver@aklaver.com>
In-Reply-To: <84379bfb-bbda-84e6-cacc-e863ba9d6c37@gmx.net>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Archived-At: <https://www.postgresql.org/message-id/7d3fc27a-1229-4739-8d15-f4005dfa2435%40aklaver.com>
Precedence: bulk


On 4/2/25 10:32 AM, Dimitrios Apostolou wrote:
> Hello list.
> 
> My database includes one table with 1000 partitions, all of them rather
> sizeable. I run:
> 
>    pg_restore -j12 --no-tablespaces --disable-triggers --exit-on-error 
> --no-owner --no-privileges -n public -d newdb custom_format_dump.pgdump
> 
> Right now after 24h of restore, I notice weird behaviour, so I have
> several questions about it:
> 
> + 11 postgres backend processes are sleeping as "TRUNCATE TABLE waiting".
>    I see that they are waiting to issue a TRUNCATE for one of the
>    partitions and then COPY data to it.  Checking the log I see that
>    several partitions have already been copied finished, but many more
>    are left to start.
> 
>    Why is a TRUNCATE needed at the start of a partition's COPY phase? I
>    didn't issue a --clean on the command line (I don't need it as my
>    database is newly created), and I don't see a mention of related
>    TRUNCATE in the pg_restore manual.

--clean will drop the object entirely not TRUNCATE.

I'm guessing that this is being done by you per:

https://www.postgresql.org/message-id/53760c70-4a87-a453-9e02-57abc9cb2e54%40gmx.net

"After each failed attempt, I need to issue a TRUNCATE table1,table2,...
before I try again. "

> 

> 
> Thanks in advance,
> Dimitris
> 
> 

-- 
Adrian Klaver
adrian.klaver@aklaver.com