MIME-Version: 1.0
References: <CAOYmi+kThkM9Z87u=R_Wi7fCor2i+UZKAyq0UCyprzCwTQvqgA@mail.gmail.com>
 <20240612155040.u6cvatdb5tiwcxci@awork3.anarazel.de> <FAC2FCFE-8C70-4996-9598-34DD7596BB41@yesql.se>
 <CA+Tgmobhsqh8c+ySXaPf5MdVwFYW6ceQ05N7pav6dzecxJGJoQ@mail.gmail.com>
 <CAGECzQTtGoPGQMZactCAytbPvNcA59Zk7RA7SEmzObyAm7K=5w@mail.gmail.com>
 <CA+TgmoZCQtUu4u7XJ1umD8k5pTuM3h2zLDyQ1uoVOP-==gJz_Q@mail.gmail.com>
 <CAGECzQTzjVaMrKi5SCqL+H63pOx8r1DP0JpJ0v2SfaE0fMaNmg@mail.gmail.com>
 <CA+TgmoYvLLAJthL9ev-qc3gVLYX5WC3y3T5hsObaEnVOAx+e6g@mail.gmail.com>
 <CAGECzQQ6nHqhwBOr3fW_+_nxn82TJxMPZGAeXD-WKKJPYHYQqw@mail.gmail.com>
 <CA+TgmoboC3zPe5uzY58_QxGaGexGGpSBTNSDMZ4_Tzo-nrne=w@mail.gmail.com>
 <CAOYmi+moGFuBaaAHq=kixGk0YuOZan09Wn5O4WAzA-xLTEumaA@mail.gmail.com>
 <CA+TgmoY4DRMVEV7CVo1D8p==ExsN69W89rF7-Ax4V=M9HGWL7g@mail.gmail.com>
 <CA+TgmoZV8B35Mvx8DqHtJjHDQxscJhwOWzNWiPj+dsUhwaUfSg@mail.gmail.com>
 <2302192.1718380169@sss.pgh.pa.us> <CAGECzQTOE1KdTBwq3vfXFH4dtXSGt+bcP5zU8aXVA9XJEfQUog@mail.gmail.com>
 <a38765d0-da19-4121-aaf9-cbf111da5461@dunslane.net> <CAGECzQQCG+HM-5+prrjKjF4TotoThNSQ6zD9JjMNsbRNgK8qVA@mail.gmail.com>
In-Reply-To: <CAGECzQQCG+HM-5+prrjKjF4TotoThNSQ6zD9JjMNsbRNgK8qVA@mail.gmail.com>
From: Robert Haas <robertmhaas@gmail.com>
Date: Sat, 15 Jun 2024 13:26:57 -0400
Message-ID: <CA+Tgmoa0HryaVxwMkNEedsbVFF_DCGEE7xh0oxowh7gc3F-gRg@mail.gmail.com>
Subject: Re: RFC: adding pytest as a supported test framework
To: Jelte Fennema-Nio <postgres@jeltef.nl>
Cc: Andrew Dunstan <andrew@dunslane.net>, Tom Lane <tgl@sss.pgh.pa.us>, 
	Jacob Champion <jacob.champion@enterprisedb.com>, Daniel Gustafsson <daniel@yesql.se>, 
	Andres Freund <andres@anarazel.de>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://www.postgresql.org/message-id/CA%2BTgmoa0HryaVxwMkNEedsbVFF_DCGEE7xh0oxowh7gc3F-gRg%40mail.gmail.com>
Precedence: bulk

On Sat, Jun 15, 2024 at 12:48=E2=80=AFPM Jelte Fennema-Nio <postgres@jeltef=
.nl> wrote:
> Honestly, my primary *objective* complaint about our current test
> suite, is that when a test fails, it's very often impossible for me to
> understand why the test failed, by only looking at the output of
> "meson test". I think logging the postgres log to stderr for Perl, as
> you proposed, would significantly improve that situation. I think the
> only thing that we cannot get from Perl Test::More that we can from
> pytest, is the fancy recursive introspection of the expression that
> pytest shows on error.

This surprises me. I agree that the current state of affairs is kind
of annoying, but the contents of regress_log_whatever are usually
quite long. Printing all of that out to standard output seems like
it's just going to flood the terminal with output. I don't think I'd
be a fan of that change.

I think I basically agree with all the nearby comments about how the
advantages you cite for Python aren't, I don't know, entirely
compelling. Switching from ok() to is() or cmp_ok() or like() is minor
stuff. Where the output goes is minor stuff. The former can be fixed,
and the latter can be worked around with scripts and aliases. The one
thing I know about that *I* think is a pretty big problem about Perl
is that IPC::Run is not really maintained. But I wonder if the
solution to that is to do something ourselves instead of depending on
IPC::Run. Beyond that, I think this is just a language popularity
contest.

--=20
Robert Haas
EDB: http://www.enterprisedb.com