From: Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
Subject: Re: Move regression.diffs of pg_upgrade test suite
To: Noah Misch <noah@leadboat.com>, pgsql-hackers@postgresql.org
References: <20181224034411.GA3224776@rfd.leadboat.com>
Openpgp: preference=signencrypt
Autocrypt: addr=andrew.dunstan@2ndQuadrant.com; prefer-encrypt=mutual;
 keydata=
 xsBNBE7KWFkBCAClridxur2AIc7eW2AR7izbfp3EnNefie2HbLF0izW5Ik5UjX2HBXBx4syI
 gY6b0ugohXrr274+baoAlvSbq6cAoQuEVrk5IZFzt20b1Xkx65FwGSEj526yiKLocqkJceSq
 Xr9xcA5SGY+FZv441chh5SU92v4q6z+6LPpoHOh97ptAVXZYNTtU0LevyvD5lja0TzbvJm6C
 eFXitJfnm1pLEr0DGJCR/iUOl/N62Kh4855zZC7NHIjQHPOvV5Stz/l5ilDhvGVk+xkXFPys
 SjZoUr1rXhYLpiyi5sR0X9FHXT0KnGuz1F5ERO7ZTLSSQ6fJwPj6gOk9K+vvoKvoeql5ABEB
 AAHNL0FuZHJldyBEdW5zdGFuIDxhbmRyZXcuZHVuc3RhbkAybmRRdWFkcmFudC5jb20+wsB+
 BBMBAgAoAhsDBgsJCAcDAgYVCAIJCgsEFgIDAQIeAQIXgAUCXA6fjQUJDyV6hAAKCRCZ+n/L
 Wfw7gXfrCACQF8lUFIw3JsBg8i3SKFeqSP4LUbFChCbEKTun4Gj4h42n2FVjYBjH+I4a+LIO
 EJGshytFUlIxZ5YHuJJqk7rI69fFIbsV5WB05dPM0u7UoweyVtuVBHmpyASeS0HIHViRaKGs
 OpC6MKCGYAmQZIyaCK0+6jX9HUOuNozAoTf3oVzpNEpCtP+8DlY//NEzLDZaJAdiRzY3n7P1
 X5Kl22XsDkY0fNO0JKQ4GbSYnIX0QgZP6bcUAMu0Fm7pqIn5XcJpw/LNNztnhwGC5Lqr/ZGV
 YlW+rs+sT/gePr0yqomtv+soyKtcn9AH0qqHpOKckgmM51L444RxWfDuTJZc0oMYzsBNBE7K
 WFkBCADRnOM0FCzsYW6jtncg+dWIagjUZpvaClmqn/sJluLaQ3v1VXMQJzYs3eC1gh386W+X
 BwLRpDj3jzH81lX+p73Re3d3oJW7X+ffsxuzu5ZVdMUkqBYonkAbKxr6gyJ12F/+JkUVzLco
 TN+d/7YsQvUVi7NaKH8mJgjz112O4fUe3p9wfAaFa0RXHc5SGPzRTYRRlv/XZBIho4J2tkZO
 nteZJZ+GbxQVlINt6fd8P6al3MWOvpP/ExJPguEfjOsO6Njyxjo3WfpD4lHMOR/Oc3/8mScE
 F84rF2jXbsFgelWnbPWAvXY+pD0dXOFRkagGmC/viwBDqq5b5tk76kKmUbZxABEBAAHCwGUE
 GAECAA8CGwwFAlwOn6QFCQ8lesQACgkQmfp/y1n8O4HlOAf/bxRbFZlsXqw1oDY8oivNMyD5
 pcP4sHVJVOhSLtCVizIONPd5L8QbO0TGVfYQap93pAbw+CLxbFSya8Pas0iNhR91HmMuVl6R
 LNNT6YmutDTpb0Q6l268s7r33vomKOm1T9yJK0z0zWC/wXj1Ew9gz70iPNkmxk8s7gO0ib+j
 UaqkAzCMtX1OiskBS5kCaf0CcoSls5vY1ofHnsPI1WqDGNCm6uTe0aorBMdS7vyMOuOlKYXW
 M0Dcpn/pG4lOSn+COg4c1fMPbIntQaNY7jY9yHOf3xSbJsZY252g21z3JxbLq9TqPZiBxWpI
 FoDgTLsaFn4zjYi44y4f/fmSKkki1w==
Message-ID: <ede9f7a3-2082-ce59-abb0-fe867dc03bbb@2ndQuadrant.com>
Date: Mon, 24 Dec 2018 15:11:33 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.3.0
MIME-Version: 1.0
In-Reply-To: <20181224034411.GA3224776@rfd.leadboat.com>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
Content-Language: en-MW
Precedence: bulk


On 12/23/18 10:44 PM, Noah Misch wrote:
> src/bin/pg_upgrade/test.sh runs installcheck, which writes to files in
> src/test/regress.  This has at least two disadvantages when check-world runs
> both this test suite and the "make check" suite:
>
> 1. The suite finishing second will overwrite the other's regression.{out,diffs}.
> 2. If these suites run a given test file in parallel (possible with "make -j
>    check-world"), they simultaneously edit a file in src/test/regress/results.
>    This can cause reporting of spurious failures.  On my system, the symptom
>    is a regression.diffs indicating that the .out file contained ranges of NUL
>    bytes (holes) and/or lacked expected lines.
>
> A disadvantage of any change here is that it degrades buildfarm reports, which
> recover slowly as owners upgrade to a fixed buildfarm release.  This will be
> similar to the introduction of --outputdir=output_iso.  On non-upgraded
> animals, pg_upgradeCheck failures will omit regression.diffs.
>
> I think the right fix, attached, is to use "pg_regress --outputdir" to
> redirect these files to src/bin/pg_upgrade/tmp_check/regress.  I chose that
> particular path because it will still fit naturally if we ever rewrite test.sh
> using src/test/perl.  I'm recommending that the buildfarm capture[1] files
> matching src/bin/pg_upgrade/tmp_check/*/*.diffs, which will work even if we
> make this test suite run installcheck more than once.  This revealed a few
> places where tests assume @abs_builddir@ is getcwd(), which I fixed.
>
> Thanks,
> nm
>
> [1] https://github.com/PGBuildFarm/client-code/blob/REL_9/PGBuild/Modules/TestUpgrade.pm#L126

Seems reasonable.

cheers

andrew


-- 
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services