From: Nathan Bossart <nathandbossart@gmail.com>
To: Bruce Momjian <bruce@momjian.us>
Cc: pgsql-hackers@postgresql.org
Subject: Re: optimize file transfer in pg_upgrade
Date: Tue, 19 Nov 2024 12:29:06 -0600
Message-ID: <ZzzY8szZRP-W0VwB@nathan> (raw)
In-Reply-To: <ZzwHKIZY5zrrfqat@momjian.us>
References: <Zyvop-LxLXBLrZil@nathan>
<ZzwHKIZY5zrrfqat@momjian.us>
On Mon, Nov 18, 2024 at 10:34:00PM -0500, Bruce Momjian wrote:
> On Wed, Nov 6, 2024 at 04:07:35PM -0600, Nathan Bossart wrote:
>> For clusters with many relations, the file transfer step of pg_upgrade can
>> take the longest. This step clones, copies, or links the user relation
>> files from the older cluster to the new cluster, so the amount of time it
>> takes is closely related to the number of relations. However, since v15,
>> we've preserved the relfilenodes during pg_upgrade, which means that all of
>> these user relation files will have the same name. Therefore, it can be
>> much faster to instead move the entire data directory from the old cluster
>> to the new cluster and to then swap the catalog relation files.
>
> That is certainly a creative idea. I am surprised the links take so
> long. Obviously rollback would be hard, as you mentioned, while now you
> can rollback --link until you start. I think it clearly should be
> considered.
I've yet to try, but I'm cautiously optimistic that it will be possible to
generate simple scripts that can unwind things by just looking at the
directory entries, even if pg_upgrade crashed halfway through the linking
stage.
> The patch is smaller than I expected.
I was surprised by this, too. Obviously, this one is a bit smaller than
the "real" patches will be because it's just a proof-of-concept, but it
should still be pretty manageable.
--
nathan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: nathandbossart@gmail.com, bruce@momjian.us
Subject: Re: optimize file transfer in pg_upgrade
In-Reply-To: <ZzzY8szZRP-W0VwB@nathan>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox