public inbox for [email protected]
help / color / mirror / Atom feedFrom: Alexander Korotkov <[email protected]>
To: Tom Lane <[email protected]>
Cc: Justin Pryzby <[email protected]>
Cc: Nathan Bossart <[email protected]>
Cc: Michael Banck <[email protected]>
Cc: Laurenz Albe <[email protected]>
Cc: vignesh C <[email protected]>
Cc: Kumar, Sachin <[email protected]>
Cc: Robins Tharakan <[email protected]>
Cc: Jan Wieck <[email protected]>
Cc: Bruce Momjian <[email protected]>
Cc: Andrew Dunstan <[email protected]>
Cc: Magnus Hagander <[email protected]>
Cc: Peter Eisentraut <[email protected]>
Cc: [email protected]
Subject: Re: pg_upgrade failing for 200+ million Large Objects
Date: Sat, 27 Jul 2024 06:00:47 +0300
Message-ID: <CAPpHfdu63MK5k4qBNc4YV14MFdNq8rCsvz0x7+Z=ijbFm1y6wQ@mail.gmail.com> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
<[email protected]>
<[email protected]>
<[email protected]>
<[email protected]>
<20240327150826.GB3994937@nathanxps13>
<20240401191930.GA2302032@nathanxps13>
<[email protected]>
<ZqEND4ZcTDBmcv31@pryzbyj2023>
<CAPpHfduUSA7c0YrTxN443SwXXCL41vmXkavYJjN8GS3Wc5Z42w@mail.gmail.com>
<ZqQIxIsaPWO6coil@pryzbyj2023>
<[email protected]>
<CAPpHfdtpr2FYVYqmARZdSp=2zsndgJgmSsHFxFFNTiYRfX8uaA@mail.gmail.com>
<[email protected]>
On Sat, Jul 27, 2024 at 2:06 AM Tom Lane <[email protected]> wrote:
> Alexander Korotkov <[email protected]> writes:
> > On Sat, Jul 27, 2024 at 1:37 AM Tom Lane <[email protected]> wrote:
> >> It's fairly easy to fix things so that this example doesn't cause
> >> that to happen: we just need to issue these updates as one command
> >> not N commands per table.
>
> > I was thinking about counting actual number of queries, not TOC
> > entries for transaction number as a more universal solution. But that
> > would require usage of psql_scan() or writing simpler alternative for
> > this particular purpose. That looks quite annoying. What do you
> > think?
>
> The assumption underlying what we're doing now is that the number
> of SQL commands per TOC entry is limited. I'd prefer to fix the
> code so that that assumption is correct, at least in normal cases.
> I confess I'd not looked closely enough at the binary-upgrade support
> code to realize it wasn't correct already :-(. If we go that way,
> we can fix this while also making pg_upgrade faster rather than
> slower. I also expect that it'll be a lot simpler than putting
> a full SQL parser in pg_restore.
I'm good with that as soon as we're not going to meet many cases of
high number SQL commands per TOC entry.
J4F, I have an idea to count number of ';' sings and use it for
transaction size counter, since it is as upper bound estimate of
number of SQL commands :-)
------
Regards,
Alexander Korotkov
Supabase
view thread (15+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: pg_upgrade failing for 200+ million Large Objects
In-Reply-To: <CAPpHfdu63MK5k4qBNc4YV14MFdNq8rCsvz0x7+Z=ijbFm1y6wQ@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox