Date: Wed, 12 Jun 2024 13:56:57 -0700
From: Noah Misch <noah@leadboat.com>
To: Jelte Fennema-Nio <postgres@jeltef.nl>
Cc: Andres Freund <andres@anarazel.de>,
	Andrew Dunstan <andrew@dunslane.net>,
	Jacob Champion <jacob.champion@enterprisedb.com>,
	PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Subject: Re: RFC: adding pytest as a supported test framework
Message-ID: <20240612205657.2c@rfd.leadboat.com>
References: <CAOYmi+kThkM9Z87u=R_Wi7fCor2i+UZKAyq0UCyprzCwTQvqgA@mail.gmail.com>
 <20240610200411.byj6sv2vpgol6wcf@awork3.anarazel.de>
 <00c5efe6-1abb-42f8-a9a7-ed3a055c4b92@dunslane.net>
 <20240611014911.khxuai5fy2ssbmiw@awork3.anarazel.de>
 <20240611234829.9d@rfd.leadboat.com>
 <CAGECzQTEPcLH-Gjr8aTkK=R36_jHU_ikhhan6hO42zCoUNv=yA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAGECzQTEPcLH-Gjr8aTkK=R36_jHU_ikhhan6hO42zCoUNv=yA@mail.gmail.com>
User-Agent: Mutt/2.2.12 (2023-09-09)
Archived-At: <https://www.postgresql.org/message-id/20240612205657.2c%40rfd.leadboat.com>
Precedence: bulk

On Wed, Jun 12, 2024 at 01:40:30PM +0200, Jelte Fennema-Nio wrote:
> On Wed, 12 Jun 2024 at 01:48, Noah Misch <noah@leadboat.com> wrote:
> > I also want the initial scope to be the new language coexisting with the
> > existing Perl tests.  If a bulk translation ever happens, it should happen
> > long after the debut of the new framework.  That said, I don't much trust a
> > human-written bulk language translation to go through without some tests
> > accidentally ceasing to test what they test in Perl today.
> 
> I definitely don't think we should rewrite all the tests that we have
> in Perl today into some other language. But I do think that whatever
> language we choose, that language should make it as least as easy to
> write tests, as easy to read them and as easy to see that they are
> testing the intended thing, as is currently the case for Perl.
> Rewriting a few Perl tests into the new language, even if not merging
> the rewrite, is a good way of validating that imho.

Agreed.

> PS. For PgBouncer I actually hand-rewrote all the tests that we had in
> bash (which is the worst testing language ever) in Python and doing so
> actually found more bugs in PgBouncer code that our bash tests
> wouldn't catch. So it's not necessarily the case that you lose
> coverage by rewriting tests.

Yep.