Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jkEzt-00074k-14 for pgsql-hackers@arkaria.postgresql.org; Sat, 13 Jun 2020 22:51:33 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1jkEzp-0001C4-Db for pgsql-hackers@arkaria.postgresql.org; Sat, 13 Jun 2020 22:51:29 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jkEzp-00018E-0c for pgsql-hackers@lists.postgresql.org; Sat, 13 Jun 2020 22:51:29 +0000 Received: from mail-il1-x142.google.com ([2607:f8b0:4864:20::142]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1jkEzk-0006qk-To for pgsql-hackers@lists.postgresql.org; Sat, 13 Jun 2020 22:51:27 +0000 Received: by mail-il1-x142.google.com with SMTP id b5so12067496iln.5 for ; Sat, 13 Jun 2020 15:51:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telsasoft-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=/FnrvcM4hJkpYaThrMaPJPUpIEb7e4whdDvRRUkwiMg=; b=iMwy633w5JGjqCvFmWw0LFw4wy4XndqQlDl1SICBZpjJ/2ZIS8pVFNEGrYsZ1bZF60 5yXr5B7gPu8hp29zjeYimHhsJp0LgfjZOKvnoOkQ/jQD1vLC2etZkbgqXziC9a3BM0uk z530K8F5w49V9Bc8k3XJ4AVjBoE1MfaTBGLygFiLZBSaY9rUZOvOUItKO9+Zi6rxMI3M QWy5+vR4VC9rst3sT6fbKkad0SMSX49v7/76gU9dkGLncEUl3onxfRQnv5DLXks3e9UZ 1O7xuu/2c3Dpd5yVSZIzwv9m3T4IgYOcc+XPnG0u/NN/tQ84su4NULcbLi74cL4tlZoC q/9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=/FnrvcM4hJkpYaThrMaPJPUpIEb7e4whdDvRRUkwiMg=; b=p/bvTlwKmQBv5TU7BLuH4Ch8vztTBZidkJ3MDtcPMVM6X1ULbL/HlMekxF6XPOaN1n YWFjJWWF67+LE7aWpXnRZkVIeYxtfF9WSyS/7K185ROg8fqJFrYPg/bfkjoyBLdc+gP3 NLQRa1uOWMbPzRyZa+gGk5aG1jG6Ge3l/+QDa1lYdcEp379Hpn3E1SpQbfJlhg3o1reQ EwliPelfWAI88rJpECgSQ9TQhWpT5GBeeA2PPG+4bKkLhRNB1W2oTiZMNbRA4gSf8/ox tkvl+7kDVGcNqL5uavGBjaNJkDro9F9T61CfmrQV8SYoaBXhFUu/sZUHCi+tmb13NbiD 7cCQ== X-Gm-Message-State: AOAM533Dtb/bI2W5yBBDZ736Pw6wIGCYd55tDrY8b7jtnRwZIqze/OTJ N0ppQ9V227cYiTo/lf1LfpRyPA== X-Google-Smtp-Source: ABdhPJzo6Caiz+M6otgKg6ghJ2DNYf4FLNY3jzcEO5rFrhxQGXMU4J4uNAJcT0ULhIzph02mwjWxXw== X-Received: by 2002:a05:6e02:8ea:: with SMTP id n10mr18857489ilt.58.1592088683556; Sat, 13 Jun 2020 15:51:23 -0700 (PDT) Received: from pryzbyj.telsasoft (charmander.telsasoft.com. [50.244.222.1]) by smtp.gmail.com with ESMTPSA id v18sm5558340ilk.6.2020.06.13.15.51.22 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 13 Jun 2020 15:51:22 -0700 (PDT) Received: by pryzbyj.telsasoft (Postfix, from userid 1000) id 6EA6880012D; Sat, 13 Jun 2020 17:51:21 -0500 (CDT) Date: Sat, 13 Jun 2020 17:51:21 -0500 From: Justin Pryzby To: David Gilman Cc: David Gilman , Tom Lane , pgsql-hackers@lists.postgresql.org, thomas.munro@gmail.com Subject: Re: Warn when parallel restoring a custom dump without data offsets Message-ID: <20200613225121.GN14879@telsasoft.com> References: <20200519130740.GO18456@telsasoft.com> <25161.1589986110@sss.pgh.pa.us> <20200523224751.GP4472@telsasoft.com> <20200525185429.6pp5vdvfu3vxlwac@dar.local> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200525185429.6pp5vdvfu3vxlwac@dar.local> User-Agent: Mutt/1.9.4 (2018-02-28) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On Mon, May 25, 2020 at 01:54:29PM -0500, David Gilman wrote: > > Is it possible to dump to stdout (or pipe to cat or dd) to avoid a new option ? > > The underlying IPC::Run code seems to support piping in a cross-platform > way. I am not a Perl master though and after spending an evening trying > to get it to work I went with this approach. If you can put me in touch > with anyone to help me out here I'd appreciate it. I think you can do what's needed like so: --- a/src/bin/pg_dump/t/002_pg_dump.pl +++ b/src/bin/pg_dump/t/002_pg_dump.pl @@ -152,10 +152,13 @@ my %pgdump_runs = ( }, defaults_custom_format_no_seek_parallel_restore => { test_key => 'defaults', - dump_cmd => [ - 'pg_dump', '-Fc', '-Z6', '--no-sync', '--disable-seeking', + dump_cmd => ( + [ + 'pg_dump', '-Fc', '-Z6', '--no-sync', "--file=$tempdir/defaults_custom_format_no_seek_parallel_restore.dump", 'postgres', - ], + ], + "|", [ "cat" ], # disable seeking + ), Also, these are failing intermittently: t/002_pg_dump.pl .............. 1649/6758 # Failed test 'defaults_custom_format_no_seek_parallel_restore: should dump GRANT SELECT (proname ...) ON TABLE pg_proc TO public' # at t/002_pg_dump.pl line 3635. # Review defaults_custom_format_no_seek_parallel_restore results in /var/lib/pgsql/postgresql.src/src/bin/pg_dump/tmp_check/tmp_test_NqRC t/002_pg_dump.pl .............. 2060/6758 # Failed test 'defaults_dir_format_parallel: should dump GRANT SELECT (proname ...) ON TABLE pg_proc TO public' # at t/002_pg_dump.pl line 3635. # Review defaults_dir_format_parallel results in /var/lib/pgsql/postgresql.src/src/bin/pg_dump/tmp_check/tmp_test_NqRC If you can address those, I think this will be "ready for committer". -- Justin