From: Jonathan S. Katz <jkatz@postgresql.org>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Peter Geoghegan <pg@bowt.ie>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Andres Freund <andres@anarazel.de>
Cc: Noah Misch <noah@leadboat.com>
Cc: Andrew Dunstan <andrew@dunslane.net>
Cc: Bruce Momjian <bruce@momjian.us>
Cc: Michael Paquier <michael@paquier.xyz>
Cc: Justin Pryzby <pryzby@telsasoft.com>
Cc: pgsql-hackers@postgresql.org <pgsql-hackers@postgresql.org>
Cc: Shruthi Gowda <gowdashru@gmail.com>
Subject: Re: pg15b2: large objects lost on upgrade
Date: Thu, 4 Aug 2022 07:02:28 -0700
Message-ID: <f0c60283-1c3d-0f0e-166a-b667c154fe0a@postgresql.org> (raw)
In-Reply-To: <3260692.1659568781@sss.pgh.pa.us>
References: <20220730054401.GB3644221@rfd.leadboat.com>
<351616.1659192034@sss.pgh.pa.us>
<635c8809-3525-7dae-a1a1-0d9069cd86bc@postgresql.org>
<2657259.1659460333@sss.pgh.pa.us>
<CA+TgmoastyZUAmJ6t953=yycQxu_S0x6A08QVARsbOVAWW=D2A@mail.gmail.com>
<c07d096c-a67a-9fb9-2001-7a97b36aa5a7@postgresql.org>
<2686054.1659469199@sss.pgh.pa.us>
<98286e9b-8fe6-554e-c23f-d67a3df2c30d@postgresql.org>
<2687956.1659469881@sss.pgh.pa.us>
<CA+TgmoYqSrO-dSP6up31gv5F+1-zM-5exjKw2BU6w+FtzAzNhA@mail.gmail.com>
<20220803202014.77m6z5m7v525p2b4@awork3.anarazel.de>
<CA+TgmoYJaHNBghqALt-9FonK8FRB96m_MSNr9ev5EBto-2oMYw@mail.gmail.com>
<3128525.1659559617@sss.pgh.pa.us>
<CAH2-WzkmNN0kwPBO8t=waOCikWM4KLX9fWEYyt2oyWP08u3mSg@mail.gmail.com>
<ddf43511-32fd-fc22-1b36-8e6c733a55b0@postgresql.org>
<3260692.1659568781@sss.pgh.pa.us>
On 8/3/22 4:19 PM, Tom Lane wrote:
> "Jonathan S. Katz" <jkatz@postgresql.org> writes:
>> I did rule out wanting to do the "xid + $X" check after reviewing some
>> of the output. I think that both $X could end up varying, and it really
>> feels like a bandaid.
>
> It is that. I wouldn't feel comfortable with $X less than 100 or so,
> which is probably sloppy enough to draw Robert's ire. Still, realizing
> that what we want right now is a band-aid for 15beta3, I don't think
> it's an unreasonable short-term option.
Attached is the "band-aid / sloppy" version of the patch. Given from the
test examples I kept seeing deltas over 100 for relfrozenxid, I chose
1000. The mxid delta was less, but I kept it at 1000 for consistency
(and because I hope this test is short lived in this state), but can be
talked into otherwise.
>> Andres suggested upthread using "txid_current()" -- for the comparison,
>> that's one thing I looked at. Would any of the XID info from
>> "pg_control_checkpoint()" also serve for this test?
>
> I like the idea of txid_current(), but we have no comparable
> function for mxid do we? While you could get both numbers from
> pg_control_checkpoint(), I doubt that's sufficiently up-to-date.
...unless we force a checkpoint in the test?
Jonathan
Attachments:pg_upgrade-test-gte-xid-v2.patch (text/plain, 1.7K) downloaddiff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.plindex 4cbc75644c..ced889601f 100644--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl@@ -226,6 +226,8 @@ ORDER BY c.oid::regclass::text
EOM
$horizon_query =~ s/\s+/ /g; # run it together on one line
my $horizon1 = $oldnode->safe_psql('regression', $horizon_query);
+chomp($horizon1);+my @horizon1_values = split(/\|/, $horizon1);
# In a VPATH build, we'll be started in the source directory, but we want
# to run pg_upgrade in the build directory so that any files generated finish
@@ -313,6 +315,8 @@ $newnode->command_ok(
# And record the horizons from the upgraded cluster as well.
my $horizon2 = $newnode->safe_psql('regression', $horizon_query);
+chomp($horizon2);+my @horizon2_values = split(/\|/, $horizon2);
# Compare the two dumps, there should be no differences.
my $compare_res = compare("$tempdir/dump1.sql", "$tempdir/dump2.sql");
@@ -331,8 +335,13 @@ if ($compare_res != 0)
print "=== EOF ===\n";
}
-# Compare the horizons, there should be no differences.-my $horizons_ok = $horizon1 eq $horizon2;+# Compare the horizons. There should be no change in the OID, so we will test+# for equality. However, because autovacuum may have run between the two+# horizions, we will check if the updated horizon XIDs are greater than or+# equal to the originals.+my $horizons_ok = $horizon2_values[0] eq $horizon1_values[0] &&+ int($horizon2_values[1]) <= int($horizon1_values[1]) + 1000 &&+ int($horizon2_values[2]) <= int($horizon1_values[2]) + 1000;
ok($horizons_ok, 'old and new horizons match after pg_upgrade');
# Provide more context if the horizons do not match.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: jkatz@postgresql.org, tgl@sss.pgh.pa.us, pg@bowt.ie, robertmhaas@gmail.com, andres@anarazel.de, noah@leadboat.com, andrew@dunslane.net, bruce@momjian.us, michael@paquier.xyz, pryzby@telsasoft.com, gowdashru@gmail.com
Subject: Re: pg15b2: large objects lost on upgrade
In-Reply-To: <f0c60283-1c3d-0f0e-166a-b667c154fe0a@postgresql.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox