Re: Rewriting the test of pg_upgrade as a TAP test - take three - remastered set

public inbox for [email protected]  
help / color / mirror / Atom feed

From: Andrew Dunstan <[email protected]>
To: Michael Paquier <[email protected]>
To: Postgres hackers <[email protected]>
Subject: Re: Rewriting the test of pg_upgrade as a TAP test - take three - remastered set
Date: Sat, 15 May 2021 14:22:24 -0400
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>

On 5/14/21 10:26 PM, Michael Paquier wrote:
> Hi,
> (Adding Andrew in CC for the buildfarm and PostgresNode parts.)
>
> $subject has been around for a couple of years now, with the following
> threads:
> https://www.postgresql.org/message-id/[email protected]
> https://www.postgresql.org/message-id/[email protected]...
>
> An advantage of moving to TAP is that we can then remove the support
> for upgrades within the MSVC scripts, and also remove pg_upgrade's
> test.sh that has accumulated tweaks that are solved by the TAP tests,
> resulting in cleanup:
>  8 files changed, 230 insertions(+), 403 deletions(-)
>
> Based on the past discussions, there were two obstacles preventing to
> do this switch:
> - Support for tests with older versions, something where the gap as
> been closed thanks to Andrew's work in 4c4eaf3d.
> - Buildfarm support, and I am not sure how things need to be extended
> there.
>
> Another thing to note is that HEAD uses oldbindir, bindir and libdir
> to track the location of the old and new libraries and binaries.  With
> the infrastructure in place, once can define only an install path for
> a PostgresNode, so this version uses only two variables:
> - oldinstall, for the installation path of the version to upgrade
> from.
> - oldsrc, to point to the source of the old version.
>
> It is not difficult to switch to one approach or the other, but
> reducing the logic to a minimum number of variables is a good deal to
> take IMO.
>
> I have been testing this patch a bit with older versions, down to 12,
> and that was logically working, and PostgresNode may need more to be
> able to work with ~11, as documented in get_new_node().  And I have
> not tested that with MSVC yet.  Anyway, attached is a new patch to
> make the discussion move on.  Even if there is still work to be done
> here, would people here still support this switch?

PostgresNode is currently able to create nodes suitable for upgrade down
to release 10. If we add '-w' to the 'pg_ctl start' flags that can
extend down to release 9.5. (Just tested) I think we should do that
forthwith. '-w' is now the default, but having it there explicitly does
no harm.

If people are interested in what's incompatible on older versions, they
can look at
<https://gitlab.com/adunstan/postgresnodeng/-/blob/master/PostgresNode.pm;
starting at about line 2764.

I don't think this will work, though, unless there is enough data to
exercise pg_upgrade fairly thoroughly. The approach taken by both
test.sh and (somewhat more comprehensively) by the buildfarm cross
version upgrade module is to test a cluster where the regression tests
have been run. That might be more difficult when testing against older
versions, so I have published a snapshot of the dumps of each of the
versions we tests against in the buildfarm animal crake. These could be
loaded into PostgresNode instances and then an upgrade attempted. See
<https://gitlab.com/adunstan/pg-old-bin/-/tree/master/data;. The data
goes back to 9.2. These compressed dumps are a couple of megabytes each,
not huge.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

view thread (113+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected]
  Subject: Re: Rewriting the test of pg_upgrade as a TAP test - take three - remastered set
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox