public inbox for [email protected]
help / color / mirror / Atom feedRe: Performance issues during pg_restore -j with big partitioned table
2+ messages / 2 participants
[nested] [flat]
* Re: Performance issues during pg_restore -j with big partitioned table
@ 2025-04-02 17:45 Adrian Klaver <[email protected]>
2025-04-02 17:51 ` Re: Performance issues during pg_restore -j with big partitioned table Dimitrios Apostolou <[email protected]>
0 siblings, 1 reply; 2+ messages in thread
From: Adrian Klaver @ 2025-04-02 17:45 UTC (permalink / raw)
To: Dimitrios Apostolou <[email protected]>; [email protected]
On 4/2/25 10:39 AM, Adrian Klaver wrote:
>
> --clean will drop the object entirely not TRUNCATE.
>
> I'm guessing that this is being done by you per:
>
> https://www.postgresql.org/message-id/53760c70-4a87-a453-9e02-57abc9cb2e54%40gmx.net
>
> "After each failed attempt, I need to issue a TRUNCATE table1,table2,...
> before I try again. "
Oops, forgot to engage brain.
From pg_backup_archiver.c:
* In parallel restore, if we created the table earlier in
* this run (so that we know it is empty) and we are not
* restoring a load-via-partition-root data item then we
* wrap the COPY in a transaction and precede it with a
* TRUNCATE. If wal_level is set to minimal this prevents
* WAL-logging the COPY. This obtains a speedup similar
* to that from using single_txn mode in non-parallel
* restores.
*
* We mustn't do this for load-via-partition-root cases
* because some data might get moved across partition
* boundaries, risking deadlock and/or loss of previously
* loaded data. (We assume that all partitions of a
* partitioned table will be treated the same way.)
>
>>
>
>>
>> Thanks in advance,
>> Dimitris
>>
>>
>
--
Adrian Klaver
[email protected]
^ permalink raw reply [nested|flat] 2+ messages in thread
* Re: Performance issues during pg_restore -j with big partitioned table
2025-04-02 17:45 Re: Performance issues during pg_restore -j with big partitioned table Adrian Klaver <[email protected]>
@ 2025-04-02 17:51 ` Dimitrios Apostolou <[email protected]>
0 siblings, 0 replies; 2+ messages in thread
From: Dimitrios Apostolou @ 2025-04-02 17:51 UTC (permalink / raw)
To: Adrian Klaver <[email protected]>; +Cc: [email protected]
On Wed, 2 Apr 2025, Adrian Klaver wrote:
>
>
> On 4/2/25 10:39 AM, Adrian Klaver wrote:
>>
>
>> --clean will drop the object entirely not TRUNCATE.
>>
>> I'm guessing that this is being done by you per:
>>
>> https://www.postgresql.org/message-id/53760c70-4a87-a453-9e02-57abc9cb2e54%40gmx.net
>>
>> "After each failed attempt, I need to issue a TRUNCATE table1,table2,...
>> before I try again. "
>
> Oops, forgot to engage brain.
>
> From pg_backup_archiver.c:
>
> * In parallel restore, if we created the table earlier in
> * this run (so that we know it is empty) and we are not
> * restoring a load-via-partition-root data item then we
> * wrap the COPY in a transaction and precede it with a
> * TRUNCATE. If wal_level is set to minimal this prevents
> * WAL-logging the COPY. This obtains a speedup similar
> * to that from using single_txn mode in non-parallel
> * restores.
This makes sense. It creates the table earlier, and then truncates it just
before copying data into it. I wonder if this really circumvents the WAL
since I don't have --single-transaction (incompatible to -j).
Dimitris
^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2025-04-02 17:51 UTC | newest]
Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2025-04-02 17:45 Re: Performance issues during pg_restore -j with big partitioned table Adrian Klaver <[email protected]>
2025-04-02 17:51 ` Dimitrios Apostolou <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox