MIME-Version: 1.0
References: <CAOYmi+moGFuBaaAHq=kixGk0YuOZan09Wn5O4WAzA-xLTEumaA@mail.gmail.com>
 <CA+TgmoY4DRMVEV7CVo1D8p==ExsN69W89rF7-Ax4V=M9HGWL7g@mail.gmail.com>
 <CA+TgmoZV8B35Mvx8DqHtJjHDQxscJhwOWzNWiPj+dsUhwaUfSg@mail.gmail.com>
 <2302192.1718380169@sss.pgh.pa.us> <CAOYmi+k214g1tgREvQkUcr=OCiVKyaTDu1ja+OGAygG1y=jhPQ@mail.gmail.com>
 <CAOYmi+nEqA2LmetcJKUDmctypPLLumkVwj3vQ3idYd8yAGza5Q@mail.gmail.com>
 <CAOYmi+kebAt6wSX7ee0c0kMzV7r0hp93bAt10V5a88yHHUKwog@mail.gmail.com>
 <CAOYmi+=CrBFBRX-foRRES2tx2wXBJJhbsJGjgFbWvPcmBJuK-Q@mail.gmail.com>
 <DDOUZE2TCU8O.89TN0AS4GLP7@jeltef.nl> <DE5BODEZ607Y.3LNII22O5N27O@jeltef.nl> <g4gdtfedwwdgu5sbcopjt3djtqk6p2q7n5nymp7ppapfwukoyd@ev2v4jtsbure>
In-Reply-To: <g4gdtfedwwdgu5sbcopjt3djtqk6p2q7n5nymp7ppapfwukoyd@ev2v4jtsbure>
From: Jacob Champion <jacob.champion@enterprisedb.com>
Date: Mon, 5 Jan 2026 12:19:00 -0800
Message-ID: <CAOYmi+mV97+FC=ME0EZj+3LBFHZ3g_xGmTGA_3u+TwJ_pvpFWw@mail.gmail.com>
Subject: Re: RFC: adding pytest as a supported test framework
To: Andres Freund <andres@anarazel.de>, Jelte Fennema-Nio <postgres@jeltef.nl>
Cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>, Robert Haas <robertmhaas@gmail.com>, 
	Daniel Gustafsson <daniel@yesql.se>, Tom Lane <tgl@sss.pgh.pa.us>, 
	Peter Eisentraut <peter@eisentraut.org>, Nazir Bilal Yavuz <byavuz81@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://www.postgresql.org/message-id/CAOYmi%2BmV97%2BFC%3DME0EZj%2B3LBFHZ3g_xGmTGA_3u%2BTwJ_pvpFWw%40mail.gmail.com>
Precedence: bulk

On Wed, Dec 17, 2025 at 8:10=E2=80=AFAM Andres Freund <andres@anarazel.de> =
wrote:
> I assume this intentionally doesn't pass CI:
> https://cirrus-ci.com/github/postgresql-cfbot/postgresql/cf%2F6045

v2 intentionally doesn't pass CI to show what the failure modes are.
Jelte rightly pointed out that this masks known 32-bit failures -- I
hadn't yet found a way to get 32-bit and 64-bit Python to behave well
together on Debian. (Not a huge fan of how v4 is working around the
problem, though; if we can't load libpq into Python then why run
pytest?)

Before it gets too far away from me: note that I have not yet been
able to get up to speed with the combined refactoring+feature patch
that Jelte added in v3, and it's now up to v7, and this patchset has
been moved out of Drafts. That's fine by me, but I plan to focus on
things that need to get into PG19 before I focus on this, since
nothing is really blocked on it.

> I agree with adding tap to the configuration summary, but I don't underst=
and
> the prove part, that just seems like a waste of vertical space.

I don't have a strong opinion, if no one finds it useful.

> Yes, that needs to be baked into the image. Chocolatey is catastrophicall=
y
> slow and unreliable. It's also just bad form to hit any service with such
> repeated downloads.

Yep.

> Why do we need pytest the program at all?  Running the tests one-by-one w=
ith
> pytest as a runner doesn't seem to make a whole lot of sense to me.

Even if we find reasons to implement our own custom runner for mtest
-- which I don't think we should do for the sake of architectural
purity alone; it doesn't seem to be a big deal in practice -- using
pytest directly ensures that the CI is using the same code paths as
individual developers who choose to use pytest as the top-level
runner.

> > -SUBDIRS =3D perl postmaster regress isolation modules authentication r=
ecovery subscription
> > +SUBDIRS =3D \
> > +     authentication \
> > +     isolation \
> > +     modules \
> > +     perl \
> > +     postmaster \
> > +     pytest \
> > +     recovery \
> > +     regress \
> > +     subscription
>
> I'm onboard with that, but we should do it separately and probably check =
for
> other cases where we should do it at the same time.

I'm not sure what context this is referring to? What are you on board with?

> I think it'd be a seriously bad idea to start with no central infrastruct=
ure,
> we'd be force to duplicate that all over.

Right, I just want central infra to be pulled out of the new tests
that need them rather than the other way around.

> Eventually we'll be forced to
> introduce some central infrastructure, but we'll probably not go around a=
nd
> carefully go through the existing tests for stuff that should now use the
> common infrastructure.

Ehh... I don't want to encourage "regular" refactoring of test
fixtures, since that would be incredibly disruptive and cause backport
difficulties, but I think it's fine to expect some careful suite-wide
improvements to be made as we introduce a completely new way of
testing. (And I find composed tests in Python a lot easier to refactor
than Test::More scripts, since they've declared their logical
dependencies.)

Thanks!
--Jacob