Content-Type: text/plain;
	charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.100.31\))
Subject: Re: TAP output format in pg_regress
From: Daniel Gustafsson <daniel@yesql.se>
In-Reply-To: <1558373.1656945577@sss.pgh.pa.us>
Date: Tue, 5 Jul 2022 00:11:13 +0200
Cc: Peter Eisentraut <peter.eisentraut@enterprisedb.com>,
 Andres Freund <andres@anarazel.de>,
 PostgreSQL Developers <pgsql-hackers@lists.postgresql.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <0F8DA66F-3604-4ED4-B2EE-57815652449C@yesql.se>
References: <BD4B107D-7E53-4794-ACBA-275BEB4327C9@yesql.se>
 <49A54862-C682-4B60-9155-05F178FA79DC@yesql.se>
 <20220321234924.kcc3q26o7sx6rnfa@alap3.anarazel.de>
 <E35C4A74-7265-4106-908E-5D3E5B9D8C2E@yesql.se>
 <489D2B5A-B26F-4918-BC07-B426329169E3@yesql.se>
 <5e6bc6a6-0a4e-109d-2b7d-c286dc58d609@enterprisedb.com>
 <1558373.1656945577@sss.pgh.pa.us>
To: Tom Lane <tgl@sss.pgh.pa.us>
Archived-At: 
 <https://www.postgresql.org/message-id/0F8DA66F-3604-4ED4-B2EE-57815652449C%40yesql.se>
Precedence: bulk

> On 4 Jul 2022, at 16:39, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>=20
> Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
>> I'm not sure what to make of all these options.  I think providing a =
TAP=20
>> output for pg_regress is a good idea.  But then do we still need the =
old=20
>> output?  Is it worth maintaining two output formats that display =
exactly=20
>> the same thing in slightly different ways?
>=20
> Probably is, because this is bad:
>=20
>> ... The proposed default format now hides the=20
>> fact that some tests are started in parallel.  I remember the last =
time=20
>> I wanted to tweak the output of the parallel tests, people were very=20=

>> attached to the particular timing and spacing of the current output.  =
So=20
>> I'm not sure people will like this.
>=20
> and so is this:
>=20
>> The timing output is very popular.  Where is that in the TAP output?
>=20
> Both of those things are fairly critical for test development.  You
> need to know what else might be running in parallel with a test case,
> and you need to know whether you just bloated the runtime =
unreasonably.
>=20
> More generally, I'm unhappy about the proposal that TAP should become
> the default output. =20

That's not my proposal though, my proposal is that the traditional =
format
should be the default output (with the parallel test info added back, =
that was
my bad), and that TAP is used in automated test runners like in meson.  =
Hiding
the timing in TAP was (as mentioned upthread) since TAP test runners =
generally
never show that anyways, but I'll add it back since it clearly doesn't =
hurt to
have even there.

> There is nothing particularly human-friendly
> about it, whereas the existing format is something we have tuned to
> our liking over literally decades.  I don't mind if there's a way to
> get TAP when you're actually intending to feed it into a TAP-parsing
> tool, but I am not a TAP-parsing tool and I don't see why I should
> have to put up with it.

I totally agree, and that's why the patch has the traditional format - =
without
all the boilerplate - as the default.  Unless opting-in there is no =
change over
today, apart from the boilerplate.

--
Daniel Gustafsson		https://vmware.com/