From: Tom Lane <tgl@sss.pgh.pa.us>
To: pgsql-hackers@lists.postgresql.org
Subject: PSA: we lack TAP test coverage on NetBSD and OpenBSD
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <9426.1547701450.1@sss.pgh.pa.us>
Content-Transfer-Encoding: quoted-printable
Date: Thu, 17 Jan 2019 00:04:10 -0500
Message-ID: <9427.1547701450@sss.pgh.pa.us>
Precedence: bulk

Although we've got a few NetBSD and OpenBSD buildfarm critters,
none of them are doing --enable-tap-tests.  If they were, we'd
have noticed the pgbench regression tests falling over:

not ok 3 - pgbench option error: bad option stderr /(?^:(unrecognized|ille=
gal) option)/
#   Failed test 'pgbench option error: bad option stderr /(?^:(unrecognize=
d|illegal) option)/'
#   at t/002_pgbench_no_server.pl line 190.
#                   'pgbench: unknown option -- bad-option
# Try "pgbench --help" for more information.
# '
#     doesn't match '(?^:(unrecognized|illegal) option)'

Sure enough, manual testing confirms that on these platforms
that error message is spelled differently:

$ pgbench --bad-option
pgbench: unknown option -- bad-option
Try "pgbench --help" for more information.


I am, TBH, inclined to fix this by removing that test case rather
than teaching it another spelling to accept.  I think it's very
hard to make the case that tests like this one are anything but
a waste of developer and buildfarm time.  When they are also a
portability hazard, it's time to cut our losses.  (I also note
that this test has caused us problems before, cf 869aa40a2 and
933851033.)

			regards, tom lane