Message-ID: <d5305046-d733-a567-5164-b40094cea407@enterprisedb.com>
Date: Fri, 25 Feb 2022 14:39:15 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0)
 Gecko/20100101 Thunderbird/91.6.0
Subject: Re: Readd use of TAP subtests
Content-Language: en-US
To: Andres Freund <andres@anarazel.de>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>,
 Andrew Dunstan <andrew@dunslane.net>, Daniel Gustafsson <daniel@yesql.se>,
 Tom Lane <tgl@sss.pgh.pa.us>, =?UTF-8?Q?Dagfinn_Ilmari_Manns=c3=a5ker?=
 <ilmari@ilmari.org>
References: <9878e1e2-2ce3-6df0-3498-07d967534f5e@enterprisedb.com>
 <87zgpbnr59.fsf@wibble.ilmari.org>
 <6131CFD1-EC47-41ED-999E-AB28B40DDEAA@yesql.se>
 <87wnkfnq9m.fsf@wibble.ilmari.org>
 <7e67b831-9eae-cd19-c96c-c1e59d916498@dunslane.net>
 <87tufjnodg.fsf@wibble.ilmari.org> <3662434.1638984660@sss.pgh.pa.us>
 <b7e1cbbf-d4dd-8c33-42be-c1a81b162363@enterprisedb.com>
 <e8a8768c-81a6-2eca-db96-f6e741b4ffad@enterprisedb.com>
 <20220224150033.5lql2lkiv7y5kkme@alap3.anarazel.de>
From: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
In-Reply-To: <20220224150033.5lql2lkiv7y5kkme@alap3.anarazel.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Archived-At: <https://www.postgresql.org/message-id/d5305046-d733-a567-5164-b40094cea407%40enterprisedb.com>
Precedence: bulk

On 24.02.22 16:00, Andres Freund wrote:
> I've incidentally played with subtests yesterdays, when porting
> src/interfaces/libpq/test/regress.pl to a tap test. Unfortunately it seems
> that subtests aren't actually specified in the tap format, and that different
> libraries generate different output formats. The reason this matters somewhat
> is that meson's testrunner can parse tap and give nicer progress / error
> reports. But since subtests aren't in the spec it can't currently parse
> them...

Ok that's good to know.  What exactly happens when it tries to parse 
them?  Does it not count them or does it fail somehow?  The way the 
output is structured

t/001_basic.pl ..
# Subtest: vacuumlo --help
     ok 1 - exit code 0
     ok 2 - goes to stdout
     ok 3 - nothing to stderr
     1..3
ok 1 - vacuumlo --help

it appears that it should be able to parse it nonetheless and should 
just count the non-indented lines.