Subject: Re: pg_upgrade failing for 200+ million Large Objects
To: Bruce Momjian <bruce@momjian.us>
Cc: Zhihong Yu <zyu@yugabyte.com>, Andrew Dunstan <andrew@dunslane.net>,
 Tom Lane <tgl@sss.pgh.pa.us>, Magnus Hagander <magnus@hagander.net>,
 Robins Tharakan <tharakan@gmail.com>,
 Peter Eisentraut <peter.eisentraut@enterprisedb.com>,
 "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
References: <1743618.1615222719@sss.pgh.pa.us>
 <4fbf92f2-ec91-f4fa-a259-f0968e34f3d7@wi3ck.info>
 <181907.1616253799@sss.pgh.pa.us>
 <147fa478-510b-18ef-5323-9c1725b2493c@wi3ck.info>
 <c2a43a97-e551-ea6d-7a4f-a4709b4e0cbd@dunslane.net>
 <5bdcb010-ecdd-c69a-b441-68002fc38483@wi3ck.info>
 <ee7d96b8-7b0e-bb76-9724-900606efe69a@dunslane.net>
 <CALNJ-vSPnudYcS8YowHeq7VAf7PBRyu0vpsbmb_VvTJgguVyHA@mail.gmail.com>
 <3886649c-c77d-dfd7-08a4-d1606bc71254@wi3ck.info>
 <91ccdb0d-42fd-7413-4e7c-3d6445655d2e@wi3ck.info>
 <20210323145628.GD579@momjian.us>
From: Jan Wieck <jan@wi3ck.info>
Message-ID: <8d8d3961-8e8b-3dbe-f911-6f418c5fb1d3@wi3ck.info>
Date: Tue, 23 Mar 2021 13:25:15 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.8.0
MIME-Version: 1.0
In-Reply-To: <20210323145628.GD579@momjian.us>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Archived-At: 
 <https://www.postgresql.org/message-id/8d8d3961-8e8b-3dbe-f911-6f418c5fb1d3%40wi3ck.info>
Precedence: bulk

On 3/23/21 10:56 AM, Bruce Momjian wrote:
> On Tue, Mar 23, 2021 at 08:51:32AM -0400, Jan Wieck wrote:
>> On 3/22/21 7:18 PM, Jan Wieck wrote:
>> > On 3/22/21 5:36 PM, Zhihong Yu wrote:
>> > >     Hi,
>> > > 
>> > > w.r.t. pg_upgrade_improvements.v2.diff.
>> > > 
>> > > +       blobBatchCount = 0;
>> > > +       blobInXact = false;
>> > > 
>> > > The count and bool flag are always reset in tandem. It seems
>> > > variable blobInXact is not needed.
>> > 
>> > You are right. I will fix that.
>> 
>> New patch v3 attached.
> 
> Would it be better to allow pg_upgrade to pass arbitrary arguments to
> pg_restore, instead of just these specific ones?
> 

That would mean arbitrary parameters to pg_dump as well as pg_restore. 
But yes, that would probably be better in the long run.

Any suggestion as to how that would actually look like? Unfortunately 
pg_restore has -[dDoOr] already used, so it doesn't look like there will 
be any naturally intelligible short options for that.


Regards, Jan

-- 
Jan Wieck
Principle Database Engineer
Amazon Web Services