From: Justin Pryzby <pryzby@telsasoft.com>
To: David Gilman <dgilman@gilslotd.com>
Cc: David Gilman <davidgilman1@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-hackers@lists.postgresql.org
Cc: thomas.munro@gmail.com
Subject: Re: Warn when parallel restoring a custom dump without data offsets
Date: Sat, 13 Jun 2020 17:51:21 -0500
Message-ID: <20200613225121.GN14879@telsasoft.com> (raw)
In-Reply-To: <20200525185429.6pp5vdvfu3vxlwac@dar.local>
References: <CALBH9DDuJ+scZc4MEvw5uO-=vRyR2=QF9+Yh=3hPEnKHWfS81A@mail.gmail.com>
<20200519130740.GO18456@telsasoft.com>
<CALBH9DDpO8dyttAx34_pu4dPXphm+k8Ca0ee+-1e7A+vXx4emQ@mail.gmail.com>
<CAOaQA5xsa7KTsX3js5mt00Do+AiarQb5qd4ZYM3AR_NwXtFzeg@mail.gmail.com>
<CALBH9DAmhfBw=VsKCcmig9jP2pBtuyLU5U5Tun_G=3ULs+q-dg@mail.gmail.com>
<25161.1589986110@sss.pgh.pa.us>
<CALBH9DBsWbwAXseNJdk4DbH1aR=sdczyAirp1aSxjpBSi86FzA@mail.gmail.com>
<CALBH9DBYLxYxA7-=CZNa0K_Sg4wCv=RHFJDxOcSW8cZB5orezQ@mail.gmail.com>
<20200523224751.GP4472@telsasoft.com>
<20200525185429.6pp5vdvfu3vxlwac@dar.local>
On Mon, May 25, 2020 at 01:54:29PM -0500, David Gilman wrote:
> > Is it possible to dump to stdout (or pipe to cat or dd) to avoid a new option ?
>
> The underlying IPC::Run code seems to support piping in a cross-platform
> way. I am not a Perl master though and after spending an evening trying
> to get it to work I went with this approach. If you can put me in touch
> with anyone to help me out here I'd appreciate it.
I think you can do what's needed like so:
inline patch 1: src/bin/pg_dump/t/002_pg_dump.pl (+6/-3 lines) download inline-1.patch--- a/src/bin/pg_dump/t/002_pg_dump.pl+++ b/src/bin/pg_dump/t/002_pg_dump.pl@@ -152,10 +152,13 @@ my %pgdump_runs = (
},
defaults_custom_format_no_seek_parallel_restore => {
test_key => 'defaults',
- dump_cmd => [- 'pg_dump', '-Fc', '-Z6', '--no-sync', '--disable-seeking',+ dump_cmd => (+ [+ 'pg_dump', '-Fc', '-Z6', '--no-sync',
"--file=$tempdir/defaults_custom_format_no_seek_parallel_restore.dump", 'postgres',
- ],+ ],+ "|", [ "cat" ], # disable seeking+ ),Also, these are failing intermittently:
t/002_pg_dump.pl .............. 1649/6758
# Failed test 'defaults_custom_format_no_seek_parallel_restore: should dump GRANT SELECT (proname ...) ON TABLE pg_proc TO public'
# at t/002_pg_dump.pl line 3635.
# Review defaults_custom_format_no_seek_parallel_restore results in /var/lib/pgsql/postgresql.src/src/bin/pg_dump/tmp_check/tmp_test_NqRC
t/002_pg_dump.pl .............. 2060/6758
# Failed test 'defaults_dir_format_parallel: should dump GRANT SELECT (proname ...) ON TABLE pg_proc TO public'
# at t/002_pg_dump.pl line 3635.
# Review defaults_dir_format_parallel results in /var/lib/pgsql/postgresql.src/src/bin/pg_dump/tmp_check/tmp_test_NqRC
If you can address those, I think this will be "ready for committer".
--
Justin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: pryzby@telsasoft.com, dgilman@gilslotd.com, davidgilman1@gmail.com, tgl@sss.pgh.pa.us, pgsql-hackers@lists.postgresql.org, thomas.munro@gmail.com
Subject: Re: Warn when parallel restoring a custom dump without data offsets
In-Reply-To: <20200613225121.GN14879@telsasoft.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox