Message-ID: <663393ca-b2ff-26f0-2e2d-adc942aff4fd@timescale.com>
Date: Wed, 7 Sep 2022 14:42:05 -0700
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.11.0
Subject: Re: pg_upgrade failing for 200+ million Large Objects
Content-Language: en-US
To: Nathan Bossart <nathandbossart@gmail.com>, Jan Wieck <jan@wi3ck.info>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Bruce Momjian <bruce@momjian.us>,
 Zhihong Yu <zyu@yugabyte.com>, Andrew Dunstan <andrew@dunslane.net>,
 Magnus Hagander <magnus@hagander.net>, Robins Tharakan <tharakan@gmail.com>,
 Peter Eisentraut <peter.eisentraut@enterprisedb.com>,
 "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
References: <91b02dc1-f0d9-e50d-849c-18d9a66484fb@wi3ck.info>
 <985941.1616524546@sss.pgh.pa.us>
 <c0f5fde5-5cad-65ed-dd30-cd852301d074@wi3ck.info>
 <986904.1616525964@sss.pgh.pa.us>
 <6cccaa33-c263-b8a2-b064-985605d33d25@wi3ck.info>
 <988415.1616528159@sss.pgh.pa.us>
 <872315a8-99fc-da4e-463d-784cfb5a025d@wi3ck.info>
 <1010642.1616532950@sss.pgh.pa.us>
 <802b96e9-f5e1-015c-dfb9-8756974b11fc@wi3ck.info>
 <0263bf35-05d6-02a1-519b-b7895a918314@wi3ck.info>
 <20220825003227.GA1456581@nathanxps13>
From: Jacob Champion <jchampion@timescale.com>
In-Reply-To: <20220825003227.GA1456581@nathanxps13>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/663393ca-b2ff-26f0-2e2d-adc942aff4fd%40timescale.com>
Precedence: bulk

On 8/24/22 17:32, Nathan Bossart wrote:
> I'd like to revive this thread, so I've created a commitfest entry [0] and
> attached a hastily rebased patch that compiles and passes the tests.  I am
> aiming to spend some more time on this in the near future.

Just to clarify, was Justin's statement upthread (that the XID problem
is fixed) correct? And is this patch just trying to improve the
remaining memory and lock usage problems?

I took a quick look at the pg_upgrade diffs. I agree with Jan that the
escaping problem is a pretty bad smell, but even putting that aside for
a bit, is it safe to expose arbitrary options to pg_dump/restore during
upgrade? It's super flexible, but I can imagine that some of those flags
might really mess up the new cluster...

And yeah, if you choose to do that then you get to keep both pieces, I
guess, but I like that pg_upgrade tries to be (IMO) fairly bulletproof.

--Jacob