From: Michael Paquier <michael@paquier.xyz>
To: Peter Geoghegan <pg@bowt.ie>
Cc: pgsql-committers <pgsql-committers@lists.postgresql.org>
Cc: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Subject: Re: pgsql: Add more $Test::Builder::Level in the TAP tests
Date: Sat, 16 Oct 2021 15:22:12 +0900
Message-ID: <YWpvlCClxxDy8D1I@paquier.xyz> (raw)
In-Reply-To: <CAH2-Wz=wga3=NJAiQRg0cs-wiqrvkFn6nb4ZPWJwhVV9zWpHRw@mail.gmail.com>
References: <E1ma7Ll-0006i4-3r@gemulon.postgresql.org>
<CAH2-Wzn8ftvcDPwomn+y04JJzbT=TG7TN=QsmSEATUOW-ZuvQQ@mail.gmail.com>
<YWepL/jDBoidu/Ek@paquier.xyz>
<CAH2-WznrTMoZnYG2PW5mnZGdD_1_mzvi955kbuTdO4mmFuBAUA@mail.gmail.com>
<YWerXvr2L4uO3TBj@paquier.xyz>
<CAH2-Wz=wga3=NJAiQRg0cs-wiqrvkFn6nb4ZPWJwhVV9zWpHRw@mail.gmail.com>
On Wed, Oct 13, 2021 at 09:04:11PM -0700, Peter Geoghegan wrote:
> On Wed, Oct 13, 2021 at 9:00 PM Michael Paquier <michael@paquier.xyz> wrote:
>> It does not change the fact that the test needs to be fixed. TAP
>> tests should be designed to not be influenced by the external world.
>
> Right.
Hearing nothing, I have just poked at this problem. And I'd rather
make all the tests go through PostgresNode::psql from which it is
possible to grab stdout, stderr and the result code, as we could miss
local changes in this test if we need a centralized fix, for problems
like the one with SIGPIPE where we don't use -c in a psql command.
Peter, does it take care of your problem?
--
Michael
Attachments:psql-tap-fix.patch (text/x-diff, 2.6K) downloaddiff --git a/src/bin/psql/t/001_basic.pl b/src/bin/psql/t/001_basic.plindex 685744744a..eb9041d1f3 100644--- a/src/bin/psql/t/001_basic.pl+++ b/src/bin/psql/t/001_basic.pl@@ -15,10 +15,46 @@ program_options_handling_ok('psql');
my ($stdout, $stderr);
my $result;
+# Execute a psql command and check its result patterns.+sub psql_like+{+ local $Test::Builder::Level = $Test::Builder::Level + 1;++ my $node = shift;+ my $test_name = shift;+ my $query = shift;+ my $expected_stdout = shift;+ my $expected_stderr = shift;++ die "cannot specify both expected stdout and stderr here"+ if (defined($expected_stdout) && defined($expected_stderr));++ # Use the context of a WAL sender, some of the tests rely on that.+ my ($ret, $stdout, $stderr) = $node->psql(+ 'postgres', $query,+ on_error_die => 0,+ replication => 'database');++ if (defined($expected_stdout))+ {+ is($ret, 0, "$test_name: expected result code");+ is($stderr, '', "$test_name: no stderr");+ like($stdout, $expected_stdout, "$test_name: stdout matches");+ }+ if (defined($expected_stderr))+ {+ isnt($ret, 0, "$test_name: expected result code");+ like($stderr, $expected_stderr, "$test_name: stderr matches");+ }++ return;+}+
# test --help=foo, analogous to program_help_ok()
foreach my $arg (qw(commands variables))
{
- $result = IPC::Run::run [ 'psql', "--help=$arg" ], '>', \$stdout, '2>', \$stderr;+ $result = IPC::Run::run [ 'psql', "--help=$arg" ], '>', \$stdout, '2>',+ \$stderr;
ok($result, "psql --help=$arg exit code 0");
isnt($stdout, '', "psql --help=$arg goes to stdout");
is($stderr, '', "psql --help=$arg nothing to stderr");
@@ -34,11 +70,13 @@ max_wal_senders = 4
});
$node->start;
-$node->command_like([ 'psql', '-c', '\copyright' ], qr/Copyright/, '\copyright');-$node->command_like([ 'psql', '-c', '\help' ], qr/ALTER/, '\help without arguments');-$node->command_like([ 'psql', '-c', '\help SELECT' ], qr/SELECT/, '\help');-+psql_like($node, '\copyright', '\copyright', qr/Copyright/, undef);+psql_like($node, '\help without arguments', '\help', qr/ALTER/, undef);+psql_like($node, '\help with argument', '\help SELECT', qr/SELECT/, undef);
# Test clean handling of unsupported replication command responses
-$node->command_fails_like([ 'psql', '-d', 'replication=database', '-c', 'START_REPLICATION 0/0' ],- qr/^unexpected PQresultStatus: 8$/, 'handling of unexpected PQresultStatus');+psql_like(+ $node,+ 'handling of unexpected PQresultStatus',+ 'START_REPLICATION 0/0',+ undef, qr/unexpected PQresultStatus: 8$/);
signature.asc (application/pgp-signature, 832B) download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-committers@postgresql.org
Cc: michael@paquier.xyz, pg@bowt.ie, peter.eisentraut@enterprisedb.com
Subject: Re: pgsql: Add more $Test::Builder::Level in the TAP tests
In-Reply-To: <YWpvlCClxxDy8D1I@paquier.xyz>
References: <E1ma7Ll-0006i4-3r@gemulon.postgresql.org> <CAH2-Wzn8ftvcDPwomn+y04JJzbT=TG7TN=QsmSEATUOW-ZuvQQ@mail.gmail.com> <YWepL/jDBoidu/Ek@paquier.xyz> <CAH2-WznrTMoZnYG2PW5mnZGdD_1_mzvi955kbuTdO4mmFuBAUA@mail.gmail.com> <YWerXvr2L4uO3TBj@paquier.xyz> <CAH2-Wz=wga3=NJAiQRg0cs-wiqrvkFn6nb4ZPWJwhVV9zWpHRw@mail.gmail.com> <YWpvlCClxxDy8D1I@paquier.xyz>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox