public inbox for [email protected]
help / color / mirror / Atom feedFrom: Jacob Champion <[email protected]>
To: Robert Haas <[email protected]>
Cc: Jelte Fennema-Nio <[email protected]>
Cc: Daniel Gustafsson <[email protected]>
Cc: Andres Freund <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Subject: Re: RFC: adding pytest as a supported test framework
Date: Thu, 13 Jun 2024 13:06:44 -0700
Message-ID: <CAOYmi+moGFuBaaAHq=kixGk0YuOZan09Wn5O4WAzA-xLTEumaA@mail.gmail.com> (raw)
In-Reply-To: <CA+TgmoboC3zPe5uzY58_QxGaGexGGpSBTNSDMZ4_Tzo-nrne=w@mail.gmail.com>
References: <CAOYmi+kThkM9Z87u=R_Wi7fCor2i+UZKAyq0UCyprzCwTQvqgA@mail.gmail.com>
<[email protected]>
<CAOYmi+niQdwFdX7srOiD8zdme_rxEp2m4JGdqzK=+dS6dpV2Og@mail.gmail.com>
<[email protected]>
<[email protected]>
<CA+Tgmobhsqh8c+ySXaPf5MdVwFYW6ceQ05N7pav6dzecxJGJoQ@mail.gmail.com>
<CAGECzQTtGoPGQMZactCAytbPvNcA59Zk7RA7SEmzObyAm7K=5w@mail.gmail.com>
<CA+TgmoZCQtUu4u7XJ1umD8k5pTuM3h2zLDyQ1uoVOP-==gJz_Q@mail.gmail.com>
<CAGECzQTzjVaMrKi5SCqL+H63pOx8r1DP0JpJ0v2SfaE0fMaNmg@mail.gmail.com>
<CA+TgmoYvLLAJthL9ev-qc3gVLYX5WC3y3T5hsObaEnVOAx+e6g@mail.gmail.com>
<CAGECzQQ6nHqhwBOr3fW_+_nxn82TJxMPZGAeXD-WKKJPYHYQqw@mail.gmail.com>
<CA+TgmoboC3zPe5uzY58_QxGaGexGGpSBTNSDMZ4_Tzo-nrne=w@mail.gmail.com>
On Thu, Jun 13, 2024 at 12:20 PM Robert Haas <[email protected]> wrote:
> I interpreted Jacob's original email as articulating a goal ("For the
> v18 cycle, I would like to try to get pytest [1] in as a supported
> test driver, in addition to the current offerings") rather than a
> plan.
That's the first part of it.
> There's no patch set yet and, as I understand it, no detailed
> plan for a patch set: that email seemed to focus on the question of
> desirability, rather than on outlining a plan of work, which I assume
> is still to come.
There was a four-step plan sketch at the end of that email, titled "A
Plan". That was not intended to be "the final detailed plan", because
I was soliciting feedback on the exact pieces people wanted to try to
implement first, and I am not the God Emperor of Pytest. But it was
definitely A Plan.
> Some things I'd like to see when a patch set does
> show up are:
>
> - good documentation for people who have no previous experience with
> Python and/or pytest e.g. here's how to set up your environment on
> Linux, Windows, macOS, *BSD so you can run the tests, here's how to
> run the tests, here's how it's different from the Perl framework we
> have now
+1
> - no external dependencies on PostgreSQL connectors. psql or libpq
> foreign function interface. the latter would be a cool increment of
> progress over the status quo.
If this is a -1 for psycopg, then I will cast my vote for ctypes/CFFI
and against psql.
> - at least as much in-tree support for writing tests as we have today
> with PostgreSQL::Test::whatever, but not necessarily a 1:1 reinvention
> of the stuff we have now, and documentation of those facilities that
> is as good or, ideally, better than what we have today.
I think this is way too much expectation for a v1 patch. If you were
committing this by yourself, would you agree to develop the entirety
of PostgreSQL::Test in a single commit, without the benefit of the
buildfarm checking you as you went, and other people trying to write
tests with it?
> - high overall code quality and level of maturity, not just something
> someone threw together for parity with the Perl system.
+1
> - enough tests written for or converted to the new system to give
> reviewers confidence that it's truly usable and fit for purpose.
This is that "know everything up front" tax that I think is not
reasonable for a test framework. If the thing you're trying to avoid
is the foot-in-the-door phenomenon, I would agree with you for a
Postgres feature. But these are tests; we don't ship them, we have
different rules for backporting them, they are developed in a very
different way.
> The important thing to me here (as it so often is) is to think like a
> maintainer. Imagine that immediately after the patches for this
> feature are committed, the developers who did the work all disappear
> from the community and are never heard from again. How much pain does
> that end us causing? The answer doesn't need to be zero; that is
> unrealistic. But it also shouldn't be "well, if that happens we're
> going to have to rip the feature out"
Can you elaborate on why that's not an okay outcome?
> or "well, a bunch of committers
> who didn't want to write tests in Python in the first place are now
> going to have to do a lot of work in Python to stabilize the work
> already committed."
Is it that? If the problem is that, we should address that. Because if
that is truly the fear, I cannot assuage that fear without showing you
something, and I cannot show you something you do not want to see, if
you don't want to write tests in Python in the first place.
--Jacob
view thread (77+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: RFC: adding pytest as a supported test framework
In-Reply-To: <CAOYmi+moGFuBaaAHq=kixGk0YuOZan09Wn5O4WAzA-xLTEumaA@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox