Feedback-ID: i131946ab:Fastmail
Message-ID: <28cced17-a21b-84c0-2de2-e470d97089e5@enterprisedb.com>
Date: Wed, 29 Mar 2023 09:08:05 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0)
 Gecko/20100101 Thunderbird/102.9.0
Subject: Re: TAP output format in pg_regress
Content-Language: en-US
To: Daniel Gustafsson <daniel@yesql.se>
Cc: vignesh C <vignesh21@gmail.com>, Nikolay Shaplov <dhyan@nataraj.su>,
 Andres Freund <andres@anarazel.de>,
 =?UTF-8?Q?Dagfinn_Ilmari_Manns=c3=a5ker?= <ilmari@ilmari.org>,
 PostgreSQL Developers <pgsql-hackers@lists.postgresql.org>,
 Andrew Dunstan <andrew@dunslane.net>, Tom Lane <tgl@sss.pgh.pa.us>
References: <5e6bc6a6-0a4e-109d-2b7d-c286dc58d609@enterprisedb.com>
 <E3FDEFE1-6460-4714-BC8F-0A1693B2BED5@yesql.se>
 <20221128182848.rxcmaxtw3mclmyr4@awork3.anarazel.de>
 <5348601.u5redSiXY4@thinkpad-pgpro>
 <655E7296-24A1-4481-A992-B5CD55B7F7DB@yesql.se>
 <CALDaNm0k6iOhzwxCi16B9gHCVpHUsh_AM85ugX4KJdMTTxq8xg@mail.gmail.com>
 <CALDaNm0WtGHpU5iZNfd8nDJ-avEw+btFNJvv7qVOPLy6BZG=dw@mail.gmail.com>
 <CALDaNm21roPooMrok6u6jxPciWyeLGen9qBnavD67=gJdswNrw@mail.gmail.com>
 <4B2CBB72-9B1D-4028-9CAD-E53DF303AB8D@yesql.se>
 <0CBE542D-F333-45DE-9D6B-533CCDB13CDE@yesql.se>
 <42883A10-AA8C-4044-B559-3E741F011B4E@yesql.se>
 <34735bb9-ca5e-8b3d-1ccd-c4ba252bbfa4@enterprisedb.com>
 <66CE8D6C-1344-4B55-9CB5-2E144DD3CCF1@yesql.se>
 <E29A82D4-F150-46ED-94B3-ACFE373B3A86@yesql.se>
From: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
In-Reply-To: <E29A82D4-F150-46ED-94B3-ACFE373B3A86@yesql.se>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Archived-At: <https://www.postgresql.org/message-id/28cced17-a21b-84c0-2de2-e470d97089e5%40enterprisedb.com>
Precedence: bulk

On 28.03.23 15:56, Daniel Gustafsson wrote:
>> On 28 Mar 2023, at 15:26, Daniel Gustafsson <daniel@yesql.se> wrote:
> 
>> I think the attached is a good candidate for going in, but I would like to see it
>> for another spin in the CF bot first.
> 
> Another candidate due to a thinko which raised a compiler warning.

This is incorrect:

-echo "+++ tap install-check in $(subdir) +++" && \
+echo "\# +++ tap install-check in $(subdir) +++" && \

It actually prints the backslash.

But this appears to be correct:

  pg_regress_check = \
-    echo "+++ regress check in $(subdir) +++" && \
+    echo "\# +++ regress check in $(subdir) +++" && \

(Maybe because it's in a variable definition?)


I'm confused why all the messages at the end lost their period ("All 
tests passed", "A copy of the test summary ...", etc.).  As long as they 
are written as sentences, they should have a period.


There is a comment "Testnames are indented 8 spaces" but you made that 
into a macro now, which is currently defined to 3.


There is an unexplained #if 0.


The additions of

+       free(difffilename);
+       difffilename = NULL;
+       free(logfilename);
+       logfilename = NULL;

in the success case are not clear.  The program is going to end soon anyway.


On the indentation of the test names:  If you look at the output of 
meson test verbose mode (e.g., meson test -C _build --suite regress 
--verbose), it reproduces the indentation in a weirdly backwards way, e.g.,

▶ 1/1 stats                                  981 ms OK
▶ 1/1 event_trigger                          122 ms OK
▶ 1/1 oidjoins                               172 ms OK
▶ 1/1 fast_default                              137 ms OK
▶ 1/1 tablespace                                285 ms OK

Not sure by which logic it arrives at that, but you can clearly see 3 
additional spaces for the single tests.


One thing I have noticed while playing around with this is that it's 
quite hard to tell casually whether a test run has failed or is failing, 
if you just keep half an eye on it in another Window.  The display of 
"ok"/"not ok" as well as the summaries at the end are much less 
prominent than the previous "ok"/"FAILED" and the summary box at the 
end.  I'm not sure what to do about that, or if it's just something to 
get used to.