Subject: Re: Ensure that STDERR is empty during connect_ok

From: Daniel Gustafsson <[email-masked]>
Date: Wed, 2 Feb 2022 16:42:09 +0100

> On 2 Feb 2022, at 16:01, Alvaro Herrera <[email-masked]> wrote:
> 
> On 2022-Feb-02, Daniel Gustafsson wrote:
> 
>> As part of the NSS patchset, quite a few bugs (and NSS quirks) were found by
>> inspecting STDERR in connect_ok and require it to be empty.  This is not really
>> NSS specific, and could help find issues in other libraries as well so I
>> propose to apply it regardless of the fate of the NSS patchset.
>> 
>> (The change in the SCRAM tests stems from this now making all testruns have the
>> same number of tests.  While I prefer to not plan at all and instead run
>> done_testing(), doing that consistently is for another patch, keeping this with
>> the remainder of the suites.)
> 
> Since commit 405f32fc4960 we can rely on subtests for this, so perhaps
> we should group all the tests in connect_ok() (including your new one)
> into a subtest; then each connect_ok() calls count as a single test, and
> we can add more tests in it without having to change the callers.

Disclaimer: longer term I would prefer to remove test plan counting like above
and Toms comment downthread.  This is just to make this patch more palatable on
the way there.

Making this a subtest in order to not having to change the callers, turns the
patch into the attached.  For this we must group the new test with one already
existing test, if we group more into it (which would make more sense) then we
need to change callers as that reduce the testcount across the tree.

This makes the patch smaller, but not more readable IMO (given that we can't
make it fully use subtests), so my preference is still the v1 patch.

Or did I misunderstand you?

--
Daniel Gustafsson		https://vmware.com/

---

From: Alvaro Herrera <[email-masked]>
Date: Wed, 2 Feb 2022 12:48:22 -0300

> need to change callers as that reduce the testcount across the tree.

Well, it wasn't my intention that this patch would not have to change
the callers.  What I was thinking about is making connect_ok() have a
subtest for *all* the tests it contains (and changing the callers to
match), so that *in the future* we can add more tests there without
having to change the callers *again*.

> Or did I misunderstand you?

I think you did.

-- 
Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/
"Cómo ponemos nuestros dedos en la arcilla del otro. Eso es la amistad; jugar
al alfarero y ver qué formas se pueden sacar del otro" (C. Halloway en
La Feria de las Tinieblas, R. Bradbury)