Feedback-ID: ia2694551:Fastmail
Date: Wed, 25 Feb 2026 14:54:09 +0100
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
To: Antonin Houska <ah@cybertec.at>
Cc: Mihail Nikalayeu <mihailnikalayeu@gmail.com>, 
	Pg Hackers <pgsql-hackers@lists.postgresql.org>, Robert Treat <rob@xzilla.net>
Subject: Re: Adding REPACK [concurrently]
Message-ID: <202602251344.gtuwb7nbk7m3@alvherre.pgsql>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <9116.1772009759@localhost>
Archived-At: <https://www.postgresql.org/message-id/202602251344.gtuwb7nbk7m3%40alvherre.pgsql>
Precedence: bulk

On 2026-Feb-25, Antonin Houska wrote:

> In 0005 ("Use background worker to do logical decoding"), the function is a
> bit simpler because here the decoding worker uses temporary file to send the
> data to the REPACKing backend, rather than tuplestore.

Ah, that patch changes the implementation rather substantially then; I
didn't realize that because I haven't been looking at it yet, as I
wanted to have a good idea of the status of 0004 before proceeding
further with the next one.  However, given that these changes are so
extensive, I think it might be better to review 0004+0005 squashed as
one unit, rather than separately.

> For REPACK, I suggest a variant of toast_flatten_tuple() that writes the
> output to a file, and a corresponding function that reads it while allocating
> separate chunks of memory for the individual TOASTed attributes - the restored
> tuple would reference the chunks using the "external indirect" TOAST pointers,
> as if it had been processed by ReorderBufferToastReplace(). Does that make
> sense to you?

Hmm, so on the apply side when reading the file, we would first reach
each toast attribute value, which we know to insert directly to the
toast table (keeping track of each individually toast pointer as we do
so); then we reach the heap tuple itself, we [... somehow ...] interpret
these external indirect toast pointers and substitute the toast pointers
that we created.  So we never have to construct the entire tuple, or
indeed do anything else with the toasted values other than insert them
into the toast table.

Actually, can't we simply insert the toasted values directly in the
decoding worker into the new toast table?  That could save a lot of
writing to the file, since we only save the raw heap tuples with no
toasted contents; but it's not clear to me that this is valid.  (And we
might create extra bloat if a tuple is inserted and later deleted
concurrently with the repack; but that would happen with the original
approach as well, no?)

-- 
Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/
"We’ve narrowed the problem down to the customer’s pants being in a situation
 of vigorous combustion" (Robert Haas, Postgres expert extraordinaire)