From: Tom Lane <tgl@sss.pgh.pa.us>
To: Joni Ruuskanen <joni.ruuskanen@likeit.fi>
Cc: pgsql-admin@lists.postgresql.org
Subject: Re: Restore fails if using multiple threads and database is dumped through pipe
Date: Mon, 20 Jan 2020 10:59:03 -0500
Message-ID: <12160.1579535943@sss.pgh.pa.us> (raw)
In-Reply-To: <d940b5c3b4bb00714662c0dd82d4a7841eae3586.camel@likeit.fi>
References: <d940b5c3b4bb00714662c0dd82d4a7841eae3586.camel@likeit.fi>
Joni Ruuskanen <joni.ruuskanen@likeit.fi> writes:
> After upgrading from Postgresql 11 to 12 I noticed pg_restore fails if -j / jobs parameter is specified and database is dumped through pipe.
> This doesn't happen if dump is saved through redirect (for example: > file.dmp) or file parameter (-f).
> Is this intended behaviour?
> Dump to file via stdout: pg_dump -Fc dumptest > dumptest-stdout.dmp
> Restore: pg_restore -j4 -d dumptest -O -Fc --role=tomcat dumptest-stdout.dmp
> No errors
> Dump to file via pipe: pg_dump -Fc dumptest | tee dumptest-pipe.dmp >/dev/null
> Restore: pg_restore -j4 -d dumptest -O -Fc --role=tomcat dumptest-pipe.dmp
> pg_restore: error: could not find block ID 5648 in archive -- possibly due to out-of-order restore request, which cannot be handled due to lack of data offsets in archive
Yes, if you don't write directly to the output file then pg_dump has
no way to seek in the file, so it can't go back and fill in the data
offsets in the table of contents after it finishes dumping the tables.
pg_restore can cope with the lack of information as long as it's doing
a simple serial restore; but in parallel mode, not so much.
regards, tom lane
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: tgl@sss.pgh.pa.us, joni.ruuskanen@likeit.fi, pgsql-admin@lists.postgresql.org
Subject: Re: Restore fails if using multiple threads and database is dumped through pipe
In-Reply-To: <12160.1579535943@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox