From: Tom Lane <tgl@sss.pgh.pa.us>
To: David Gilman <davidgilman1@gmail.com>
Cc: Justin Pryzby <pryzby@telsasoft.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: Warn when parallel restoring a custom dump without data offsets
Date: Wed, 20 May 2020 10:48:30 -0400
Message-ID: <25161.1589986110@sss.pgh.pa.us> (raw)
In-Reply-To: <CALBH9DAmhfBw=VsKCcmig9jP2pBtuyLU5U5Tun_G=3ULs+q-dg@mail.gmail.com>
References: <CALBH9DDuJ+scZc4MEvw5uO-=vRyR2=QF9+Yh=3hPEnKHWfS81A@mail.gmail.com>
<20200519130740.GO18456@telsasoft.com>
<CALBH9DDpO8dyttAx34_pu4dPXphm+k8Ca0ee+-1e7A+vXx4emQ@mail.gmail.com>
<CAOaQA5xsa7KTsX3js5mt00Do+AiarQb5qd4ZYM3AR_NwXtFzeg@mail.gmail.com>
<CALBH9DAmhfBw=VsKCcmig9jP2pBtuyLU5U5Tun_G=3ULs+q-dg@mail.gmail.com>
David Gilman <davidgilman1@gmail.com> writes:
>> I think the PG11
>> commit you mentioned (548e5097) happens to make some databases fail in
>> parallel restore that previously worked (I didn't check).
> Correct, if you do the bisect around that yourself you'll see
> pg_restore start failing with the expected "possibly due to
> out-of-order restore request" on offset-less dumps.
Yeah. Now, the whole point of that patch was to decouple the restore
order from the dump order ... but with an offset-less dump file, we
can't do that, or at least the restore order is greatly constrained.
I wonder if it'd be sensible for pg_restore to use a different parallel
scheduling algorithm if it notices that the input lacks offsets.
(There could still be some benefit from parallelism, just not as much.)
No idea if this is going to be worth the trouble, but it probably
is worth looking into.
regards, tom lane
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tgl@sss.pgh.pa.us, davidgilman1@gmail.com, pryzby@telsasoft.com, pgsql-hackers@lists.postgresql.org
Subject: Re: Warn when parallel restoring a custom dump without data offsets
In-Reply-To: <25161.1589986110@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox