From: Tom Lane <tgl@sss.pgh.pa.us>
To: Laurenz Albe <laurenz.albe@cybertec.at>
cc: Nathan Bossart <nathandbossart@gmail.com>, vignesh C <vignesh21@gmail.com>,
        "Kumar,
    Sachin" <ssetiya@amazon.com>,
        Robins Tharakan <tharakan@gmail.com>, Jan Wieck <jan@wi3ck.info>,
        Bruce Momjian <bruce@momjian.us>, Andrew Dunstan <andrew@dunslane.net>,
        Magnus Hagander <magnus@hagander.net>,
        Peter Eisentraut <peter.eisentraut@enterprisedb.com>,
        "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Subject: Re: pg_upgrade failing for 200+ million Large Objects
In-reply-to: <4a3ebf7d81bfc6dd4d545e5b27d6e8f6c32d8937.camel@cybertec.at>
References: <c0f5fde5-5cad-65ed-dd30-cd852301d074@wi3ck.info> <986904.1616525964@sss.pgh.pa.us> <6cccaa33-c263-b8a2-b064-985605d33d25@wi3ck.info> <988415.1616528159@sss.pgh.pa.us> <872315a8-99fc-da4e-463d-784cfb5a025d@wi3ck.info> <1010642.1616532950@sss.pgh.pa.us> <802b96e9-f5e1-015c-dfb9-8756974b11fc@wi3ck.info> <0263bf35-05d6-02a1-519b-b7895a918314@wi3ck.info> <20220825003227.GA1456581@nathanxps13> <663393ca-b2ff-26f0-2e2d-adc942aff4fd@timescale.com> <20220908231807.GA2242918@nathanxps13> <CAAWbhmgUb8p7ff_ZX5jCvqM=ipPxbbDJTXMNVzH-Ho_CXVkRHA@mail.gmail.com> <0643CC11-223A-4039-AC34-94E127462796@amazon.com> <1152134.1699555261@sss.pgh.pa.us> <83D44BE5-0088-4D41-8AE6-20A05D026F46@amazon.com> <81D13E16-BA04-43CF-9B89-B8924300B211@amazon.com> <240D05EC-8B28-4112-BEAB-85ECBAF3F871@amazon.com> <2055911.1702258962@sss.pgh.pa.us> <557FD681-3929-44A1-87B2-6B5E10C4A66B@amazon.com> <CALDaNm08DLqkr6LQfB=AJ-wEOGtASW+rYGsD6Q56-bYBYoHFdA@mail.gmail.com> <842242.1706287466@sss.pgh.pa.us> <!
 2751089.1710544721@sss.pgh.pa.us> <4a3ebf7d81bfc6dd4d545e5b27d6e8f6c32d8937.camel@cybertec.at>
Comments: In-reply-to Laurenz Albe <laurenz.albe@cybertec.at>
	message dated "Sat, 16 Mar 2024 22:59:45 +0100"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <3023816.1710629175.1@sss.pgh.pa.us>
Date: Sat, 16 Mar 2024 18:46:15 -0400
Message-ID: <3023817.1710629175@sss.pgh.pa.us>
Archived-At: <https://www.postgresql.org/message-id/3023817.1710629175%40sss.pgh.pa.us>
Precedence: bulk

Laurenz Albe <laurenz.albe@cybertec.at> writes:
> On Fri, 2024-03-15 at 19:18 -0400, Tom Lane wrote:
>> This patch seems to have stalled out again.  In hopes of getting it
>> over the finish line, I've done a bit more work to address the two
>> loose ends I felt were probably essential to deal with:

> Applies and builds fine.
> I didn't scrutinize the code, but I gave it a spin on a database with
> 15 million (small) large objects.  I tried pg_upgrade --link with and
> without the patch on a debug build with the default configuration.

Thanks for looking at it!

> Without the patch:
> Runtime: 74.5 minutes

> With the patch:
> Runtime: 70 minutes

Hm, I'd have hoped for a bit more runtime improvement.  But perhaps
not --- most of the win we saw upthread was from parallelism, and
I don't think you'd get any parallelism in a pg_upgrade with all
the data in one database.  (Perhaps there is more to do there later,
but I'm still not clear on how this should interact with the existing
cross-DB parallelism; so I'm content to leave that question for
another patch.)

			regards, tom lane