Feedback-ID: i131946ab:Fastmail
Message-ID: <5e6bc6a6-0a4e-109d-2b7d-c286dc58d609@enterprisedb.com>
Date: Mon, 4 Jul 2022 16:27:20 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0)
 Gecko/20100101 Thunderbird/91.11.0
Subject: Re: TAP output format in pg_regress
Content-Language: en-US
To: Daniel Gustafsson <daniel@yesql.se>, Andres Freund <andres@anarazel.de>
Cc: PostgreSQL Developers <pgsql-hackers@lists.postgresql.org>
References: <BD4B107D-7E53-4794-ACBA-275BEB4327C9@yesql.se>
 <49A54862-C682-4B60-9155-05F178FA79DC@yesql.se>
 <20220321234924.kcc3q26o7sx6rnfa@alap3.anarazel.de>
 <E35C4A74-7265-4106-908E-5D3E5B9D8C2E@yesql.se>
 <489D2B5A-B26F-4918-BC07-B426329169E3@yesql.se>
From: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
In-Reply-To: <489D2B5A-B26F-4918-BC07-B426329169E3@yesql.se>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/5e6bc6a6-0a4e-109d-2b7d-c286dc58d609%40enterprisedb.com>
Precedence: bulk

On 29.06.22 21:50, Daniel Gustafsson wrote:
> Attached is a new version of this patch, which completes the TAP output format
> option such that all codepaths emitting output are TAP compliant.  The verbose
> option is fixed to to not output extraneous newlines which the previous PoC
> did.  The output it made to conform to the original TAP spec since v13/14 TAP
> parsers seem less common than those that can handle the original spec.  Support
> for the new format additions should be quite simple to add should we want that.
> 
> Running pg_regress --verbose should give the current format output.
> 
> I did end up combining TAP and --verbose into a single patch, as the TAP format
> sort of depends on the verbose flag as TAP has no verbose mode.  I can split it
> into two separate should a reviewer prefer that.

I'm not sure what to make of all these options.  I think providing a TAP 
output for pg_regress is a good idea.  But then do we still need the old 
output?  Is it worth maintaining two output formats that display exactly 
the same thing in slightly different ways?

What is the purpose of the --verbose option?  When and how is one 
supposed to activate that?  The proposed default format now hides the 
fact that some tests are started in parallel.  I remember the last time 
I wanted to tweak the output of the parallel tests, people were very 
attached to the particular timing and spacing of the current output.  So 
I'm not sure people will like this.

The timing output is very popular.  Where is that in the TAP output?

More generally, what do you envision we do with this feature?  Who is it 
for, what are the tradeoffs, etc.?