pg.ddx.io pgsql-admin@postgresql.org mailing list archivehelp / color / mirror / Atom feed
Restore fails if using multiple threads and database is dumped through pipe 2+ messages / 2 participants [nested] [flat]
* Restore fails if using multiple threads and database is dumped through pipe @ 2020-01-20 14:13 Joni Ruuskanen <joni.ruuskanen@likeit.fi> 2020-01-20 15:59 ` Re: Restore fails if using multiple threads and database is dumped through pipe Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 1 reply; 2+ messages in thread From: Joni Ruuskanen @ 2020-01-20 14:13 UTC (permalink / raw) To: pgsql-admin@lists.postgresql.org Hi, After upgrading from Postgresql 11 to 12 I noticed pg_restore fails if -j / jobs parameter is specified and database is dumped through pipe. This doesn't happen if dump is saved through redirect (for example: > file.dmp) or file parameter (-f). Is this intended behaviour? Some examples: Dump to file: pg_dump -Fc -f dumptest-file.dmp dumptest Restore: pg_restore -j4 -d dumptest -O -Fc --role=tomcat dumptest-file.dmp No errors Dump to file via stdout: pg_dump -Fc dumptest > dumptest-stdout.dmp Restore: pg_restore -j4 -d dumptest -O -Fc --role=tomcat dumptest-stdout.dmp No errors Dump to file via pipe: pg_dump -Fc dumptest | tee dumptest-pipe.dmp >/dev/null Restore: pg_restore -j4 -d dumptest -O -Fc --role=tomcat dumptest-pipe.dmp pg_restore: error: could not find block ID 5648 in archive -- possibly due to out-of-order restore request, which cannot be handled due to lack of data offsets in archive pg_restore: error: a worker process died unexpectedly Piped dump is successfully restored if using only 1 thread/job. Regards, Joni Ruuskanen ^ permalink raw reply [nested|flat] 2+ messages in thread
* Re: Restore fails if using multiple threads and database is dumped through pipe 2020-01-20 14:13 Restore fails if using multiple threads and database is dumped through pipe Joni Ruuskanen <joni.ruuskanen@likeit.fi> @ 2020-01-20 15:59 ` Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 2+ messages in thread From: Tom Lane @ 2020-01-20 15:59 UTC (permalink / raw) To: Joni Ruuskanen <joni.ruuskanen@likeit.fi>; +Cc: pgsql-admin@lists.postgresql.org Joni Ruuskanen <joni.ruuskanen@likeit.fi> writes: > After upgrading from Postgresql 11 to 12 I noticed pg_restore fails if -j / jobs parameter is specified and database is dumped through pipe. > This doesn't happen if dump is saved through redirect (for example: > file.dmp) or file parameter (-f). > Is this intended behaviour? > Dump to file via stdout: pg_dump -Fc dumptest > dumptest-stdout.dmp > Restore: pg_restore -j4 -d dumptest -O -Fc --role=tomcat dumptest-stdout.dmp > No errors > Dump to file via pipe: pg_dump -Fc dumptest | tee dumptest-pipe.dmp >/dev/null > Restore: pg_restore -j4 -d dumptest -O -Fc --role=tomcat dumptest-pipe.dmp > pg_restore: error: could not find block ID 5648 in archive -- possibly due to out-of-order restore request, which cannot be handled due to lack of data offsets in archive Yes, if you don't write directly to the output file then pg_dump has no way to seek in the file, so it can't go back and fill in the data offsets in the table of contents after it finishes dumping the tables. pg_restore can cope with the lack of information as long as it's doing a simple serial restore; but in parallel mode, not so much. regards, tom lane ^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2020-01-20 15:59 UTC | newest] Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2020-01-20 14:13 Restore fails if using multiple threads and database is dumped through pipe Joni Ruuskanen <joni.ruuskanen@likeit.fi> 2020-01-20 15:59 ` Tom Lane <tgl@sss.pgh.pa.us>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox