Date: Mon, 11 Jan 2021 22:13:52 -0600
From: Justin Pryzby <pryzby@telsasoft.com>
To: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Cc: Tomas Vondra <tomas.vondra@2ndquadrant.com>,
	Tom Lane <tgl@sss.pgh.pa.us>, Andres Freund <andres@anarazel.de>,
	buschmann@nidsa.net, pgsql-hackers@lists.postgresql.org,
	Noah Misch <noah@leadboat.com>
Subject: Re: pg_upgrade test for binary compatibility of core data types
Message-ID: <20210112041352.GY1849@telsasoft.com>
References: <20191013203848.2eksimmpux3546wr@development>
 <27596.1571062600@sss.pgh.pa.us>
 <20191014163538.cqn6j4cl7s2mk2yz@development>
 <20191015001817.zei45apgpu5dzngr@development>
 <20191015044118.GO3599@telsasoft.com>
 <20191015070725.csmdmdvrlb6klwkg@development>
 <20201206180248.GI24052@telsasoft.com>
 <20201216172222.GS30237@telsasoft.com>
 <20201227190729.GB26311@telsasoft.com>
 <be693563-def1-2f5a-89c1-f4d3cb2ae716@enterprisedb.com>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="kadn00tgSopKmJ1H"
Content-Disposition: inline
In-Reply-To: <be693563-def1-2f5a-89c1-f4d3cb2ae716@enterprisedb.com>
User-Agent: Mutt/1.9.4 (2018-02-28)
Precedence: bulk


--kadn00tgSopKmJ1H
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

On Mon, Jan 11, 2021 at 03:28:08PM +0100, Peter Eisentraut wrote:
> On 2020-12-27 20:07, Justin Pryzby wrote:
> > On Wed, Dec 16, 2020 at 11:22:23AM -0600, Justin Pryzby wrote:
> > > On Sun, Dec 06, 2020 at 12:02:48PM -0600, Justin Pryzby wrote:
> > > > I meant to notice if the binary format is accidentally changed again, which was
> > > > what happened here:
> > > > 7c15cef86 Base information_schema.sql_identifier domain on name, not varchar.
> > > > 
> > > > I added a table to the regression tests so it's processed by pg_upgrade tests,
> > > > run like:
> > > > 
> > > > | time make -C src/bin/pg_upgrade check oldsrc=`pwd`/11 oldbindir=`pwd`/11/tmp_install/usr/local/pgsql/bin
> > > 
> > > Per cfbot, this avoids testing ::xml (support for which may not be enabled)
> > > And also now tests oid types.
> > > 
> > > I think the per-version hacks should be grouped by logical change, rather than
> > > by version.  Which I've started doing here.
> > 
> > rebased on 6df7a9698bb036610c1e8c6d375e1be38cb26d5f
> 
> I think these patches could use some in-place documentation of what they are
> trying to achieve and how they do it.  The required information is spread
> over a lengthy thread.  No one wants to read that.  Add commit messages to
> the patches.

0001 patch fixes pg_upgrade/test.sh, which was disfunctional.
Portions of the first patch were independently handled by commits 52202bb39,
fa744697c, 091866724.  So this is rebased on those.
I guess updating this script should be a part of a beta-checklist somewhere,
since I guess nobody will want to backpatch changes for testing older releases.

0002 allows detecting the information_schema problem that was introduced at:
7c15cef86 Base information_schema.sql_identifier domain on name, not varchar.

+-- Create a table with different data types, to exercise binary compatibility
+-- during pg_upgrade test

If binary compatibility is changed I expect this will error, crash, at least
return wrong data, and thereby fail tests.

-- 
Justin

On Sun, Dec 06, 2020 at 12:02:48PM -0600, Justin Pryzby wrote:
> I checked that if I cherry-pick 0002 to v11, and comment out
> old_11_check_for_sql_identifier_data_type_usage(), then pg_upgrade/test.sh
> detects the original problem:
> pg_dump: error: Error message from server: ERROR:  invalid memory alloc request size 18446744073709551613
> 
> I understand the buildfarm has its own cross-version-upgrade test, which I
> think would catch this on its own.
> 
> These all seem to complicate use of pg_upgrade/test.sh, so 0001 is needed to
> allow testing upgrade from older releases.
> 
> e78900afd217fa3eaa77c51e23a94c1466af421c Create by default sql/ and expected/ for output directory in pg_regress
> 40b132c1afbb4b1494aa8e48cc35ec98d2b90777 In the pg_upgrade test suite, don't write to src/test/regress.
> fc49e24fa69a15efacd5b8958115ed9c43c48f9a Make WAL segment size configurable at initdb time.
> c37b3d08ca6873f9d4eaf24c72a90a550970cbb8 Allow group access on PGDATA
> da9b580d89903fee871cf54845ffa2b26bda2e11 Refactor dir/file permissions


--kadn00tgSopKmJ1H
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="v4-0001-WIP-pg_upgrade-test.sh-changes-needed-to-allow-te.patch"