From: Antonin Houska <ah@cybertec.at>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>
cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>,
    Pg Hackers <pgsql-hackers@lists.postgresql.org>,
    Robert Treat <rob@xzilla.net>
Subject: Re: Adding REPACK [concurrently]
In-reply-to: <202602251344.gtuwb7nbk7m3@alvherre.pgsql>
References: <202602251344.gtuwb7nbk7m3@alvherre.pgsql>
Comments: In-reply-to Alvaro Herrera <alvherre@alvh.no-ip.org>
   message dated "Wed, 25 Feb 2026 14:54:09 +0100."
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <93329.1772035399.1@localhost>
Content-Transfer-Encoding: quoted-printable
Date: Wed, 25 Feb 2026 17:03:19 +0100
Message-ID: <93330.1772035399@localhost>
Archived-At: <https://www.postgresql.org/message-id/93330.1772035399%40localhost>
Precedence: bulk

Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:

> On 2026-Feb-25, Antonin Houska wrote:

> > For REPACK, I suggest a variant of toast_flatten_tuple() that writes t=
he
> > output to a file, and a corresponding function that reads it while all=
ocating
> > separate chunks of memory for the individual TOASTed attributes - the =
restored
> > tuple would reference the chunks using the "external indirect" TOAST p=
ointers,
> > as if it had been processed by ReorderBufferToastReplace(). Does that =
make
> > sense to you?
> =

> Hmm, so on the apply side when reading the file, we would first reach
> each toast attribute value, which we know to insert directly to the
> toast table (keeping track of each individually toast pointer as we do
> so); then we reach the heap tuple itself, we [... somehow ...] interpret
> these external indirect toast pointers and substitute the toast pointers
> that we created.  So we never have to construct the entire tuple, or
> indeed do anything else with the toasted values other than insert them
> into the toast table.

Yes, that's what I mean.

> Actually, can't we simply insert the toasted values directly in the
> decoding worker into the new toast table?  That could save a lot of
> writing to the file, since we only save the raw heap tuples with no
> toasted contents; but it's not clear to me that this is valid.  (And we
> might create extra bloat if a tuple is inserted and later deleted
> concurrently with the repack; but that would happen with the original
> approach as well, no?)

The problem I see here is that for UPDATE you need the old tuple to determ=
ine
if its TOAST value should be deleted or if the new tuple should reuse it -
this is how I understand toast_tuple_init(). So the worker would have to s=
tore
all the changes somewhere temporarily until it can fully apply the changes
(i.e. until the initial copy and index build is complete).

Besides that, if the worker had to switch between the past (for the decodi=
ng)
and present (for the TOAST operations), it would have to invalidate system
caches repeatedly. 0004 does that, but 0005 makes that unnecessary. (I don=
't
know if the repeated cache invalidation would be a serious performance
problem, but from coding perspective I find it more convenient if the work=
er
only deals with decoding and does not have to do this time travel and
invalidations at all.)

-- =

Antonin Houska
Web: https://www.cybertec-postgresql.com