pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
pgsql: Fix tests of pg_upgrade across different major versions
4+ messages / 1 participants
[nested] [flat]

* pgsql: Fix tests of pg_upgrade across different major versions
@ 2021-10-13 00:23 Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 4+ messages in thread

From: Michael Paquier @ 2021-10-13 00:23 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix tests of pg_upgrade across different major versions

This fixes a set of issues that cause different breakages or annoyances
when using pg_upgrade's test.sh to do upgrades across different major
versions:
- test.sh is completely broken when using v14 as new version because of
the removal of testtablespace/ as Makefile rule.  Older versions of
pg_regress don't support --make-tablespacedir, blocking the creation of
the tablespace.  In order to fix that, it is simple enough to create
those directories in the script itself, but only do that when an old
version is involved.  This fix is needed on HEAD and REL_14_STABLE.
- The script would fail when using PG <= v11 as old version because of
WITH OIDS relations not supported in v12.  In order to fix this, this
steals a method from the buildfarm that uses a DO block to change all
the relations marked as WITH OIDS, allowing pg_upgrade to pass.  This is
more portable than using ALTER TABLE queries on the relations causing
issues.  This is fixed down to v12, and authored originally by Andrew
Dunstan.
- Not using --extra-float-digits=0 with v11 as old version causes
a lot of diffs in the dumps, making the whole unreadable.  This gets
only done when using v11 as old version.  This is fixed down to v12.
The buildfarm code uses that already.

Note that the addition of --wal-segsize and --allow-group-access breaks
the script when using v10 or older at initdb time as these got added in
11.  10 would be EOL'd next year and nobody has complained about those
problems yet, so nothing is done about that.  This means that this
commit fixes upgrade tests using test.sh with v11 as minimum older
version, up to HEAD, and that it is enough to apply this change down to
12.  The old and new dumps still generate diffs, still require manual
checks, and more could be done to reduce the noise, but this allows the
tests to run with a rather minimal amount of them.

I have tested this commit and test.sh with v11 as minimum across all the
branches where this is applied.  Note that this commit has no impact on
the normal pg_upgrade test run with a simple "make check".

Author:  Justin Pryzby, Andrew Dunstan, Michael Paquier
Discussion: https://postgr.es/m/20201206180248.GI24052@telsasoft.com
Backpatch-through: 12

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/fa66b6dee0843d2bca5bf9c9b8b7be32defbffae

Modified Files
--------------
src/bin/pg_upgrade/test.sh | 66 +++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 59 insertions(+), 7 deletions(-)



^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* pgsql: Fix tests of pg_upgrade across different major versions
@ 2021-10-13 00:23 Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 4+ messages in thread

From: Michael Paquier @ 2021-10-13 00:23 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix tests of pg_upgrade across different major versions

This fixes a set of issues that cause different breakages or annoyances
when using pg_upgrade's test.sh to do upgrades across different major
versions:
- test.sh is completely broken when using v14 as new version because of
the removal of testtablespace/ as Makefile rule.  Older versions of
pg_regress don't support --make-tablespacedir, blocking the creation of
the tablespace.  In order to fix that, it is simple enough to create
those directories in the script itself, but only do that when an old
version is involved.  This fix is needed on HEAD and REL_14_STABLE.
- The script would fail when using PG <= v11 as old version because of
WITH OIDS relations not supported in v12.  In order to fix this, this
steals a method from the buildfarm that uses a DO block to change all
the relations marked as WITH OIDS, allowing pg_upgrade to pass.  This is
more portable than using ALTER TABLE queries on the relations causing
issues.  This is fixed down to v12, and authored originally by Andrew
Dunstan.
- Not using --extra-float-digits=0 with v11 as old version causes
a lot of diffs in the dumps, making the whole unreadable.  This gets
only done when using v11 as old version.  This is fixed down to v12.
The buildfarm code uses that already.

Note that the addition of --wal-segsize and --allow-group-access breaks
the script when using v10 or older at initdb time as these got added in
11.  10 would be EOL'd next year and nobody has complained about those
problems yet, so nothing is done about that.  This means that this
commit fixes upgrade tests using test.sh with v11 as minimum older
version, up to HEAD, and that it is enough to apply this change down to
12.  The old and new dumps still generate diffs, still require manual
checks, and more could be done to reduce the noise, but this allows the
tests to run with a rather minimal amount of them.

I have tested this commit and test.sh with v11 as minimum across all the
branches where this is applied.  Note that this commit has no impact on
the normal pg_upgrade test run with a simple "make check".

Author:  Justin Pryzby, Andrew Dunstan, Michael Paquier
Discussion: https://postgr.es/m/20201206180248.GI24052@telsasoft.com
Backpatch-through: 12

Branch
------
REL_14_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/f4e1c8892b9e90a9d3ccae21db04a1215b9312a5

Modified Files
--------------
src/bin/pg_upgrade/test.sh | 66 +++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 59 insertions(+), 7 deletions(-)



^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* pgsql: Fix tests of pg_upgrade across different major versions
@ 2021-10-13 00:23 Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 4+ messages in thread

From: Michael Paquier @ 2021-10-13 00:23 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix tests of pg_upgrade across different major versions

This fixes a set of issues that cause different breakages or annoyances
when using pg_upgrade's test.sh to do upgrades across different major
versions:
- test.sh is completely broken when using v14 as new version because of
the removal of testtablespace/ as Makefile rule.  Older versions of
pg_regress don't support --make-tablespacedir, blocking the creation of
the tablespace.  In order to fix that, it is simple enough to create
those directories in the script itself, but only do that when an old
version is involved.  This fix is needed on HEAD and REL_14_STABLE.
- The script would fail when using PG <= v11 as old version because of
WITH OIDS relations not supported in v12.  In order to fix this, this
steals a method from the buildfarm that uses a DO block to change all
the relations marked as WITH OIDS, allowing pg_upgrade to pass.  This is
more portable than using ALTER TABLE queries on the relations causing
issues.  This is fixed down to v12, and authored originally by Andrew
Dunstan.
- Not using --extra-float-digits=0 with v11 as old version causes
a lot of diffs in the dumps, making the whole unreadable.  This gets
only done when using v11 as old version.  This is fixed down to v12.
The buildfarm code uses that already.

Note that the addition of --wal-segsize and --allow-group-access breaks
the script when using v10 or older at initdb time as these got added in
11.  10 would be EOL'd next year and nobody has complained about those
problems yet, so nothing is done about that.  This means that this
commit fixes upgrade tests using test.sh with v11 as minimum older
version, up to HEAD, and that it is enough to apply this change down to
12.  The old and new dumps still generate diffs, still require manual
checks, and more could be done to reduce the noise, but this allows the
tests to run with a rather minimal amount of them.

I have tested this commit and test.sh with v11 as minimum across all the
branches where this is applied.  Note that this commit has no impact on
the normal pg_upgrade test run with a simple "make check".

Author:  Justin Pryzby, Andrew Dunstan, Michael Paquier
Discussion: https://postgr.es/m/20201206180248.GI24052@telsasoft.com
Backpatch-through: 12

Branch
------
REL_13_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/2a8dee6a67ccde6074f463786d9152dbb87c7eb9

Modified Files
--------------
src/bin/pg_upgrade/test.sh | 54 ++++++++++++++++++++++++++++++++++++++++------
1 file changed, 47 insertions(+), 7 deletions(-)



^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* pgsql: Fix tests of pg_upgrade across different major versions
@ 2021-10-13 00:23 Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 4+ messages in thread

From: Michael Paquier @ 2021-10-13 00:23 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix tests of pg_upgrade across different major versions

This fixes a set of issues that cause different breakages or annoyances
when using pg_upgrade's test.sh to do upgrades across different major
versions:
- test.sh is completely broken when using v14 as new version because of
the removal of testtablespace/ as Makefile rule.  Older versions of
pg_regress don't support --make-tablespacedir, blocking the creation of
the tablespace.  In order to fix that, it is simple enough to create
those directories in the script itself, but only do that when an old
version is involved.  This fix is needed on HEAD and REL_14_STABLE.
- The script would fail when using PG <= v11 as old version because of
WITH OIDS relations not supported in v12.  In order to fix this, this
steals a method from the buildfarm that uses a DO block to change all
the relations marked as WITH OIDS, allowing pg_upgrade to pass.  This is
more portable than using ALTER TABLE queries on the relations causing
issues.  This is fixed down to v12, and authored originally by Andrew
Dunstan.
- Not using --extra-float-digits=0 with v11 as old version causes
a lot of diffs in the dumps, making the whole unreadable.  This gets
only done when using v11 as old version.  This is fixed down to v12.
The buildfarm code uses that already.

Note that the addition of --wal-segsize and --allow-group-access breaks
the script when using v10 or older at initdb time as these got added in
11.  10 would be EOL'd next year and nobody has complained about those
problems yet, so nothing is done about that.  This means that this
commit fixes upgrade tests using test.sh with v11 as minimum older
version, up to HEAD, and that it is enough to apply this change down to
12.  The old and new dumps still generate diffs, still require manual
checks, and more could be done to reduce the noise, but this allows the
tests to run with a rather minimal amount of them.

I have tested this commit and test.sh with v11 as minimum across all the
branches where this is applied.  Note that this commit has no impact on
the normal pg_upgrade test run with a simple "make check".

Author:  Justin Pryzby, Andrew Dunstan, Michael Paquier
Discussion: https://postgr.es/m/20201206180248.GI24052@telsasoft.com
Backpatch-through: 12

Branch
------
REL_12_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/afa09e4a9af6dd6491a67c4e25b21a45b3a78006

Modified Files
--------------
src/bin/pg_upgrade/test.sh | 54 ++++++++++++++++++++++++++++++++++++++++------
1 file changed, 47 insertions(+), 7 deletions(-)



^ permalink  raw  reply  [nested|flat] 4+ messages in thread


end of thread, other threads:[~2021-10-13 00:23 UTC | newest]

Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-10-13 00:23 pgsql: Fix tests of pg_upgrade across different major versions Michael Paquier <michael@paquier.xyz>
2021-10-13 00:23 pgsql: Fix tests of pg_upgrade across different major versions Michael Paquier <michael@paquier.xyz>
2021-10-13 00:23 pgsql: Fix tests of pg_upgrade across different major versions Michael Paquier <michael@paquier.xyz>
2021-10-13 00:23 pgsql: Fix tests of pg_upgrade across different major versions Michael Paquier <michael@paquier.xyz>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox