Date: Wed, 5 Apr 2017 09:52:12 -0400
From: Stephen Frost <sfrost@snowman.net>
To: Michael Paquier <michael.paquier@gmail.com>
Cc: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>,
	Andres Freund <andres@anarazel.de>,
	PostgreSQL mailing lists <pgsql-hackers@postgresql.org>,
	Noah Misch <noah@leadboat.com>
Subject: Re: Rewriting the test of pg_upgrade as a TAP test
Message-ID: <20170405135211.GZ9812@tamriel.snowman.net>
References: 
 <CAB7nPqRdaN1A1YNjxNL9T1jUEWct8ttqq29dNv8W_o37+e8wfA@mail.gmail.com>
 <62a4e16e-5ea3-89c9-c94a-ee3be953b94a@2ndquadrant.com>
 <20170403153200.jqntjo7zat6x5rhh@alap3.anarazel.de>
 <42d412a2-9cd7-7ee1-0c1a-e8d230d568b0@2ndquadrant.com>
 <CAB7nPqR9JWCoU8z_CXovL2UkTxahcEx3TzyBYzk3qyXh+MTXMg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="VBDpdjLefZynJFCd"
Content-Disposition: inline
In-Reply-To: 
 <CAB7nPqR9JWCoU8z_CXovL2UkTxahcEx3TzyBYzk3qyXh+MTXMg@mail.gmail.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org


--VBDpdjLefZynJFCd
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Michael,

* Michael Paquier (michael.paquier@gmail.com) wrote:
> On Tue, Apr 4, 2017 at 10:52 PM, Peter Eisentraut
> <peter.eisentraut@2ndquadrant.com> wrote:
> > On 4/3/17 11:32, Andres Freund wrote:
> >> That doesn't strike as particularly future proof.  We intentionally
> >> leave objects behind pg_regress runs, but that only works if we actual=
ly
> >> run them...
> >
> > I generally agree with the sentiments expressed later in this thread.
> > But just to clarify what I meant here:  We don't need to run a, say,
> > 1-minute serial test to load a few "left behind" objects for the
> > pg_upgrade test, if we can load the same set of objects using dedicated
> > scripting in say 2 seconds.  This would make both the pg_upgrade tests
> > faster and would reduce the hidden dependencies in the main tests about
> > which kinds of objects need to be left behind.
>=20
> Making the tests run shorter while maintaining the current code
> coverage is nice. But this makes more complicated the test suite
> maintenance as this needs either a dedicated regression schedule or an
> extra test suite where objects are created just for the sake of
> pg_upgrade. This increases the risks of getting a rotten test suite
> with the time if patch makers and reviewers are not careful.

I believe that what Peter was getting at is that the pg_dump TAP tests
create a whole slew of objects in just a few seconds and are able to
then exercise those code-paths in pg_dump, without needing to run the
entire serial regression test run.

I'm still not completely convinced that we actually need to
independently test pg_upgrade by creating all the objects which the
pg_dump TAP tests do, given that pg_upgrade just runs pg_dump
underneath.  If we really want to do that, however, what we should do is
abstract out the pg_dump set of tests into a place that both the pg_dump
and pg_upgrade TAP tests could use them to create all the types of
objects which are supported to perform their tests.

Thanks!

Stephen

--VBDpdjLefZynJFCd
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCgAGBQJY5PaLAAoJEO1sijiDR2RVZTsP/1EYBSmMCMaLqCmheUs6pMdh
+iXikeyVamfi1LPbSdbFTocVCHeKv5KCSXIoOGuBIbXUPwWwO1fR2mULWAATLAJv
JsHm1NrrxaKI+F1KsoZ2UdNSBsiByblxAkYRZySaOfUHD8nIwqthHykN058m7WlO
103eLancYLOvLTWlF5rpzXIFihsKPc99jDdsJ/s3yUXJbERtMPMyL1ivnblgf7To
vA82raEpNxGUTbwtnShoKHccI5Don+jkqVXxYSbzS231rYbc6sZINQ/ertsyoHlG
vk44LmFHgAR1Pyti2skz+WFqLaTpOlkYp3eFpvs76u+z1E1WNpJDQ8ovhxvAbm2+
qin0ZqmMuiyQBun5TU+iIqMwbFu+Jeg2ya4zNeaJtLjIlAP4+2fBpv8ivA0pO2v8
rVESgdBLM0R4mpwtz3dYcRBG43hkITp5ERwGbnvpj5qqixuSVYlhDe1c9I4DXE+f
ZdvAk4G0nTTfDTpvdW4ZW1x86pd76jluJJDa0L+7n9pYCXZ4zrlK5taY+GHge50g
FM3fr54x7yQbhEj5/UGtD9klRTsjEcwvAcMC0V6YEbQXs2iAe2/duWbHn41pgCOW
D+MWDtaY4fyUDFWBF8IRdberuEvr4c9+zUXiVbl3scB5PyvhOy/kw1THGCuxhEIt
9acnoFKOkZRh4pAdPdbI
=oCeO
-----END PGP SIGNATURE-----

--VBDpdjLefZynJFCd--