Feedback-ID: i76984098:Fastmail
Message-ID: <7be2dcc6-3ba4-4e3f-a154-8d13d816aa9b@aklaver.com>
Date: Wed, 2 Apr 2025 10:45:29 -0700
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Performance issues during pg_restore -j with big partitioned
 table
From: Adrian Klaver <adrian.klaver@aklaver.com>
To: Dimitrios Apostolou <jimis@gmx.net>, pgsql-general@lists.postgresql.org
References: <84379bfb-bbda-84e6-cacc-e863ba9d6c37@gmx.net>
 <7d3fc27a-1229-4739-8d15-f4005dfa2435@aklaver.com>
Content-Language: en-US
In-Reply-To: <7d3fc27a-1229-4739-8d15-f4005dfa2435@aklaver.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Archived-At: <https://www.postgresql.org/message-id/7be2dcc6-3ba4-4e3f-a154-8d13d816aa9b%40aklaver.com>
Precedence: bulk


On 4/2/25 10:39 AM, Adrian Klaver wrote:
> 

> --clean will drop the object entirely not TRUNCATE.
> 
> I'm guessing that this is being done by you per:
> 
> https://www.postgresql.org/message-id/53760c70-4a87-a453-9e02-57abc9cb2e54%40gmx.net
> 
> "After each failed attempt, I need to issue a TRUNCATE table1,table2,...
> before I try again. "

Oops, forgot to engage brain.

 From pg_backup_archiver.c:

* In parallel restore, if we created the table earlier in
* this run (so that we know it is empty) and we are not
* restoring a load-via-partition-root data item then we
* wrap the COPY in a transaction and precede it with a
* TRUNCATE.  If wal_level is set to minimal this prevents
* WAL-logging the COPY.  This obtains a speedup similar
* to that from using single_txn mode in non-parallel
* restores.
*
* We mustn't do this for load-via-partition-root cases
* because some data might get moved across partition
* boundaries, risking deadlock and/or loss of previously
* loaded data.  (We assume that all partitions of a
* partitioned table will be treated the same way.)

> 
>>
> 
>>
>> Thanks in advance,
>> Dimitris
>>
>>
> 

-- 
Adrian Klaver
adrian.klaver@aklaver.com