Date: Fri, 28 Feb 2025 14:01:32 -0600
From: Nathan Bossart <nathandbossart@gmail.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: pgsql-hackers@postgresql.org
Subject: Re: optimize file transfer in pg_upgrade
Message-ID: <Z8IWHJ7OdMnQ45ur@nathan>
References: <Zyvop-LxLXBLrZil@nathan>
 <CA+Tgmobi1aiWhtfJXL2PZsEsXOHRivRykMxE1kcTDb4EwEfj-w@mail.gmail.com>
 <CA+TgmoY6VZT=HzuJ_zfrt=2Vi7EHfVfwMd3AOuEfw-f7aeR-AQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <CA+TgmoY6VZT=HzuJ_zfrt=2Vi7EHfVfwMd3AOuEfw-f7aeR-AQ@mail.gmail.com>
Archived-At: <https://www.postgresql.org/message-id/Z8IWHJ7OdMnQ45ur%40nathan>
Precedence: bulk

On Fri, Feb 28, 2025 at 02:41:22PM -0500, Robert Haas wrote:
> On Fri, Feb 28, 2025 at 2:40 PM Robert Haas <robertmhaas@gmail.com> wrote:
>> Maybe we should rethink the decision not to transfer relfilenodes for
>> sequences. Or have more than one way to do it. pg_upgrade
>> --binary-upgrade --binary-upgrade-even-for-sequences, or whatever.
> 
> Sorry, I meant: pg_dump --binary-upgrade --binary-upgrade-even-for-sequences
> 
> i.e. pg_upgrade could decide which way to ask pg_dump to do it,
> depending on versions and flags.

That's exactly where I landed (see v3-0002).  I haven't measured whether
transferring relfilenodes or dumping the sequence data is faster for the
existing modes, but for now I've left those alone, i.e., they still dump
sequence data.  The new "swap" mode just uses the old cluster's sequence
files, and I've disallowed using swap mode for upgrades from <v10 to avoid
the sequence tuple format change (along with other incompatible changes).

I'll admit I'm a bit concerned that this will cause problems if and when
someone wants to change the sequence tuple format again.  But that hasn't
happened for a while, AFAIK nobody's planning to change it, and even if it
does happen, we just need to have my proposed new mode transfer the
sequence files like it transfers the catalog files.  That will make this
mode slower, especially if you have a ton of sequences, but maybe it'll
still be a win in most cases.  Of course, we probably will need to have
pg_upgrade handle other kinds of format changes, too, but IMHO it's still
worth trying to speed up pg_upgrade despite the potential future
complexities.

-- 
nathan