public inbox for [email protected]
help / color / mirror / Atom feedFrom: Jacob Champion <[email protected]>
To: Robert Haas <[email protected]>
Cc: Jelte Fennema-Nio <[email protected]>
Cc: Daniel Gustafsson <[email protected]>
Cc: Andres Freund <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Subject: Re: RFC: adding pytest as a supported test framework
Date: Tue, 18 Jun 2024 07:34:17 -0700
Message-ID: <CAOYmi+=T1t6yn7seNjUR74wFwwtvysp6bgnOMwJ2AK0txU=C6A@mail.gmail.com> (raw)
In-Reply-To: <CA+TgmoZV8B35Mvx8DqHtJjHDQxscJhwOWzNWiPj+dsUhwaUfSg@mail.gmail.com>
References: <CAOYmi+kThkM9Z87u=R_Wi7fCor2i+UZKAyq0UCyprzCwTQvqgA@mail.gmail.com>
<[email protected]>
<CAOYmi+niQdwFdX7srOiD8zdme_rxEp2m4JGdqzK=+dS6dpV2Og@mail.gmail.com>
<[email protected]>
<[email protected]>
<CA+Tgmobhsqh8c+ySXaPf5MdVwFYW6ceQ05N7pav6dzecxJGJoQ@mail.gmail.com>
<CAGECzQTtGoPGQMZactCAytbPvNcA59Zk7RA7SEmzObyAm7K=5w@mail.gmail.com>
<CA+TgmoZCQtUu4u7XJ1umD8k5pTuM3h2zLDyQ1uoVOP-==gJz_Q@mail.gmail.com>
<CAGECzQTzjVaMrKi5SCqL+H63pOx8r1DP0JpJ0v2SfaE0fMaNmg@mail.gmail.com>
<CA+TgmoYvLLAJthL9ev-qc3gVLYX5WC3y3T5hsObaEnVOAx+e6g@mail.gmail.com>
<CAGECzQQ6nHqhwBOr3fW_+_nxn82TJxMPZGAeXD-WKKJPYHYQqw@mail.gmail.com>
<CA+TgmoboC3zPe5uzY58_QxGaGexGGpSBTNSDMZ4_Tzo-nrne=w@mail.gmail.com>
<CAOYmi+moGFuBaaAHq=kixGk0YuOZan09Wn5O4WAzA-xLTEumaA@mail.gmail.com>
<CA+TgmoY4DRMVEV7CVo1D8p==ExsN69W89rF7-Ax4V=M9HGWL7g@mail.gmail.com>
<CAOYmi+ks2X+35EOi6GS2J304T5AW53EoAz+s5vkwrGAaPixS3w@mail.gmail.com>
<CA+TgmoZV8B35Mvx8DqHtJjHDQxscJhwOWzNWiPj+dsUhwaUfSg@mail.gmail.com>
(slowly catching up from the weekend email backlog)
On Fri, Jun 14, 2024 at 5:10 AM Robert Haas <[email protected]> wrote:
> I mean, both Perl and Python are Turing-complete.
Tom responded to this better than I could have, but I don't think this
is a helpful statement. In fact I opened the unconference session with
it and immediately waved it away as not-the-point.
> I just don't believe in the idea that we're going to write one
> category of tests in one language and another category in another
> language.
You and I will probably not agree, then, because IMO we already do
that. SQL behavior is tested in SQL via pg_regress characterization
tests. End-to-end tests are written in Perl. Lower-level tests are
often written in C (and, unfortunately, then driven in Perl instead of
C; see above mail to Noah).
I'm fundamentally a polyglot tester by philosophy, so I don't see
careful use of multiple languages as an inherent problem to be solved.
They increase complexity (considerably so!) but generally provide
sufficient value to offset the cost.
> As soon as we open the door to Python tests, people are
> going to start writing the TAP tests that they would have written in
> Perl in Python instead.
There's a wide spectrum of opinions between yours (which I will
cheekily paraphrase as "people will love testing in Python so much
they'll be willing to reinvent all of the wheels" -- which in the
short term would increase maintenance cost but in the long term sounds
like a very good problem to have), and people who seem to think that
new test suite infrastructure would sit unused because no one wants to
write tests anyway (to pull a strawman out of some hallway
conversations at PGConf.dev). I think the truth is probably somewhere
in the middle?
My prior mail was an attempt to bridge the gap between today and the
medium term, by introducing a series of compromises and incremental
steps in response to specific fears. We can jump forward to the end
state and try to talk about it, but I don't control the end state and
I don't have a crystal ball.
> So as I see
> it, the only reasonable plan here if we want to introduce testing in
> Python (or C#, or Ruby, or Go, or JavaScript, or Lua, or LOLCODE) is
> to try to achieve a reasonable degree of parity between that language
> and Perl. Because then we can at least review the new infrastructure
> all at once, instead of incrementally spread across many patches
> written, reviewed, and committed by many different people.
I don't at all believe that a test API which is ported en masse as a
horizontal layer, without motivating vertical slices of test
functionality, is going to be fit for purpose.
And "written, reviewed, and committed by many different people" is a
feature for me, not a bug. One of the goals of the thread is to
encourage more community test writing than we currently have.
Otherwise, I could have kept silent (I am very happy with my personal
suite and have been comfortably maintaining it for a while). I am
trying to build community momentum around a pain point that is
currently rusted in place.
> Consider the meson build system project. To get that committed, Andres
> had to make it do pretty much everything MSVC could do and mostly
> everything that configure could do
I think some lessons can be pulled from that, but fundamentally that's
a port of the build infrastructure done by a person with a commit bit.
There are some pretty considerable differences. (And even then, it
wasn't "done" with the first volley of patches, right?)
Thanks,
--Jacob
view thread (77+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: RFC: adding pytest as a supported test framework
In-Reply-To: <CAOYmi+=T1t6yn7seNjUR74wFwwtvysp6bgnOMwJ2AK0txU=C6A@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox