pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Michael Paquier <michael@paquier.xyz>
To: Peter Geoghegan <pg@bowt.ie>
Cc: pgsql-committers <pgsql-committers@lists.postgresql.org>
Cc: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Subject: Re: pgsql: Add more $Test::Builder::Level in the TAP tests
Date: Sat, 16 Oct 2021 15:22:12 +0900
Message-ID: <YWpvlCClxxDy8D1I@paquier.xyz> (raw)
In-Reply-To: <CAH2-Wz=wga3=NJAiQRg0cs-wiqrvkFn6nb4ZPWJwhVV9zWpHRw@mail.gmail.com>
References: <E1ma7Ll-0006i4-3r@gemulon.postgresql.org>
	<CAH2-Wzn8ftvcDPwomn+y04JJzbT=TG7TN=QsmSEATUOW-ZuvQQ@mail.gmail.com>
	<YWepL/jDBoidu/Ek@paquier.xyz>
	<CAH2-WznrTMoZnYG2PW5mnZGdD_1_mzvi955kbuTdO4mmFuBAUA@mail.gmail.com>
	<YWerXvr2L4uO3TBj@paquier.xyz>
	<CAH2-Wz=wga3=NJAiQRg0cs-wiqrvkFn6nb4ZPWJwhVV9zWpHRw@mail.gmail.com>

On Wed, Oct 13, 2021 at 09:04:11PM -0700, Peter Geoghegan wrote:
> On Wed, Oct 13, 2021 at 9:00 PM Michael Paquier <michael@paquier.xyz> wrote:
>> It does not change the fact that the test needs to be fixed.  TAP
>> tests should be designed to not be influenced by the external world.
> 
> Right.

Hearing nothing, I have just poked at this problem.  And I'd rather
make all the tests go through PostgresNode::psql from which it is
possible to grab stdout, stderr and the result code, as we could miss
local changes in this test if we need a centralized fix, for problems
like the one with SIGPIPE where we don't use -c in a psql command.
Peter, does it take care of your problem?
--
Michael

Attachments:

psql-tap-fix.patch (text/x-diff, 2.6K) download diff --git a/src/bin/psql/t/001_basic.pl b/src/bin/psql/t/001_basic.pl index 685744744a..eb9041d1f3 100644 --- a/src/bin/psql/t/001_basic.pl +++ b/src/bin/psql/t/001_basic.pl @@ -15,10 +15,46 @@ program_options_handling_ok('psql'); my ($stdout, $stderr); my $result; +# Execute a psql command and check its result patterns. +sub psql_like +{ + local $Test::Builder::Level = $Test::Builder::Level + 1; + + my $node = shift; + my $test_name = shift; + my $query = shift; + my $expected_stdout = shift; + my $expected_stderr = shift; + + die "cannot specify both expected stdout and stderr here" + if (defined($expected_stdout) && defined($expected_stderr)); + + # Use the context of a WAL sender, some of the tests rely on that. + my ($ret, $stdout, $stderr) = $node->psql( + 'postgres', $query, + on_error_die => 0, + replication => 'database'); + + if (defined($expected_stdout)) + { + is($ret, 0, "$test_name: expected result code"); + is($stderr, '', "$test_name: no stderr"); + like($stdout, $expected_stdout, "$test_name: stdout matches"); + } + if (defined($expected_stderr)) + { + isnt($ret, 0, "$test_name: expected result code"); + like($stderr, $expected_stderr, "$test_name: stderr matches"); + } + + return; +} + # test --help=foo, analogous to program_help_ok() foreach my $arg (qw(commands variables)) { - $result = IPC::Run::run [ 'psql', "--help=$arg" ], '>', \$stdout, '2>', \$stderr; + $result = IPC::Run::run [ 'psql', "--help=$arg" ], '>', \$stdout, '2>', + \$stderr; ok($result, "psql --help=$arg exit code 0"); isnt($stdout, '', "psql --help=$arg goes to stdout"); is($stderr, '', "psql --help=$arg nothing to stderr"); @@ -34,11 +70,13 @@ max_wal_senders = 4 }); $node->start; -$node->command_like([ 'psql', '-c', '\copyright' ], qr/Copyright/, '\copyright'); -$node->command_like([ 'psql', '-c', '\help' ], qr/ALTER/, '\help without arguments'); -$node->command_like([ 'psql', '-c', '\help SELECT' ], qr/SELECT/, '\help'); - +psql_like($node, '\copyright', '\copyright', qr/Copyright/, undef); +psql_like($node, '\help without arguments', '\help', qr/ALTER/, undef); +psql_like($node, '\help with argument', '\help SELECT', qr/SELECT/, undef); # Test clean handling of unsupported replication command responses -$node->command_fails_like([ 'psql', '-d', 'replication=database', '-c', 'START_REPLICATION 0/0' ], - qr/^unexpected PQresultStatus: 8$/, 'handling of unexpected PQresultStatus'); +psql_like( + $node, + 'handling of unexpected PQresultStatus', + 'START_REPLICATION 0/0', + undef, qr/unexpected PQresultStatus: 8$/);
signature.asc (application/pgp-signature, 832B) download

view thread (12+ messages)  latest in thread

Message-ID: <YWpvlCClxxDy8D1I@paquier.xyz>
Permalink:  ../YWpvlCClxxDy8D1I@paquier.xyz/
Also on:    postgresql.org/message-id/YWpvlCClxxDy8D1I@paquier.xyz

 ·  · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-committers@postgresql.org
  Cc: michael@paquier.xyz, pg@bowt.ie, peter.eisentraut@enterprisedb.com
  Subject: Re: pgsql: Add more $Test::Builder::Level in the TAP tests
  In-Reply-To: <YWpvlCClxxDy8D1I@paquier.xyz>
  References: <E1ma7Ll-0006i4-3r@gemulon.postgresql.org> <CAH2-Wzn8ftvcDPwomn+y04JJzbT=TG7TN=QsmSEATUOW-ZuvQQ@mail.gmail.com> <YWepL/jDBoidu/Ek@paquier.xyz> <CAH2-WznrTMoZnYG2PW5mnZGdD_1_mzvi955kbuTdO4mmFuBAUA@mail.gmail.com> <YWerXvr2L4uO3TBj@paquier.xyz> <CAH2-Wz=wga3=NJAiQRg0cs-wiqrvkFn6nb4ZPWJwhVV9zWpHRw@mail.gmail.com> <YWpvlCClxxDy8D1I@paquier.xyz>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox