[PATCH 2/7] Do not dereference varattrib_4b in VARSIZE

agora inbox for [email protected]  
help / color / mirror / Atom feed

[PATCH 2/7] Do not dereference varattrib_4b in VARSIZE_4B.
130+ messages / 1 participants
[nested] [flat]

* [PATCH 2/7] Do not dereference varattrib_4b in VARSIZE_4B.
@ 2026-03-11 13:53  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-03-11 13:53 UTC (permalink / raw)

Since VARSIZE_ANY() may call VARSIZE_4B(), it's possible that the compiler
(when invoked with -Warray-bounds) complains if the argument of VARSIZE_ANY()
is actually smaller than what VARSIZE_4B() expects. This patch adjusts the
VARSIZE_4B() macro so that it does not have to dereference the varattrib_4b
structure. We assume that varlena value always starts with the length word.

The problem does not exist in the tree at the moment since the current users
of VARSIZE_ANY() pass a pointer to a dynamically allocated memory, so the
compiler has no idea about the memory available. However, in an upcoming
patch, it makes sense to pass a pointer to a local variable of "varlena"
type. In such a case, the compiler warning might appear because
sizeof(varlena) is lower than sizeof(varattrib_4b).
---
 src/include/varatt.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/include/varatt.h b/src/include/varatt.h
index 000bdf33b92..31063e5d4f1 100644
--- a/src/include/varatt.h
+++ b/src/include/varatt.h
@@ -207,7 +207,7 @@ typedef struct
 
 /* VARSIZE_4B() should only be used on known-aligned data */
 #define VARSIZE_4B(PTR) \
-	(((const varattrib_4b *) (PTR))->va_4byte.va_header & 0x3FFFFFFF)
+	(*((const uint32 *) (PTR)) & 0x3FFFFFFF)
 #define VARSIZE_1B(PTR) \
 	(((const varattrib_1b *) (PTR))->va_header & 0x7F)
 #define VARTAG_1B_E(PTR) \
@@ -240,7 +240,7 @@ typedef struct
 
 /* VARSIZE_4B() should only be used on known-aligned data */
 #define VARSIZE_4B(PTR) \
-	((((const varattrib_4b *) (PTR))->va_4byte.va_header >> 2) & 0x3FFFFFFF)
+	((*((const uint32 *) (PTR)) >> 2) & 0x3FFFFFFF)
 #define VARSIZE_1B(PTR) \
 	((((const varattrib_1b *) (PTR))->va_header >> 1) & 0x7F)
 #define VARTAG_1B_E(PTR) \
-- 
2.47.3


--=-=-=
Content-Type: text/plain
Content-Disposition: attachment;
 filename=v42-0003-Add-CONCURRENTLY-option-to-REPACK-command.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-06-16 11:54  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2268cc277bc..127d4415084 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -609,11 +608,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -627,11 +623,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -705,6 +696,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -881,11 +874,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -901,8 +892,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -922,6 +912,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -932,33 +924,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2346,21 +2341,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2372,6 +2391,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2380,76 +2402,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v01-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

* [PATCH 1/8] Use tuple slot to pass tuples for rewriting.
@ 2026-07-15 08:37  Antonin Houska <[email protected]>
  0 siblings, 0 replies; 130+ messages in thread

From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw)

This patch tries to adopt the preferable way of handling tuples, i.e. pass the
containing tuple slot rather than the actual tuple. The motivation is that
heap_insert_for_repack() will need a slot in the near future, in order to call
ExecInsertIndexTuples(). Also, in order to set values of dropped attributes to
to NULL, it seems more compact to pass a tuple slot as a workspace than two
arrays (one for values and one for nulls).
---
 src/backend/access/heap/heapam_handler.c | 199 +++++++++++++----------
 1 file changed, 110 insertions(+), 89 deletions(-)

diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..4db8a16f691 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -48,14 +48,13 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
-static void reform_and_rewrite_tuple(HeapTuple tuple,
-									 Relation OldHeap, Relation NewHeap,
-									 Datum *values, bool *isnull, RewriteState rwstate);
-static void heap_insert_for_repack(HeapTuple tuple, Relation OldHeap,
-								   Relation NewHeap, Datum *values, bool *isnull,
-								   BulkInsertState bistate);
-static HeapTuple reform_tuple(HeapTuple tuple, Relation OldHeap,
-							  Relation NewHeap, Datum *values, bool *isnull);
+static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+									 RewriteState rwstate);
+static void heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+								   TupleTableSlot *reform,
+								   BulkInsertStateData *bistate);
+static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc);
+static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform);
 
 static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
 								   HeapTuple tuple,
@@ -603,11 +602,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	bool		is_system_catalog;
 	Tuplesortstate *tuplesort;
 	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
 	TupleTableSlot *slot;
-	int			natts;
-	Datum	   *values;
-	bool	   *isnull;
+	TupleTableSlot *reform_slot;
 	BufferHeapTupleTableSlot *hslot;
 	BlockNumber prev_cblock = InvalidBlockNumber;
 	bool		concurrent = snapshot != NULL;
@@ -621,11 +617,6 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	 */
 	Assert(RelationGetTargetBlock(NewHeap) == InvalidBlockNumber);
 
-	/* Preallocate values/isnull arrays */
-	natts = newTupDesc->natts;
-	values = palloc_array(Datum, natts);
-	isnull = palloc_array(bool, natts);
-
 	/*
 	 * In non-concurrent mode, initialize the rewrite operation.  This is not
 	 * needed in concurrent mode.
@@ -699,6 +690,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 
 	slot = table_slot_create(OldHeap, NULL);
 	hslot = (BufferHeapTupleTableSlot *) slot;
+	reform_slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										   &TTSOpsVirtual);
 
 	/*
 	 * Scan through the OldHeap, either in OldIndex order or sequentially;
@@ -875,11 +868,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			int64		ct_val[2];
 
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple, OldHeap, NewHeap,
-										 values, isnull, rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/*
 			 * In indexscan mode and also VACUUM FULL, report increase in
@@ -895,8 +886,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		index_endscan(indexScan);
 	if (tableScan != NULL)
 		table_endscan(tableScan);
-	if (slot)
-		ExecDropSingleTupleTableSlot(slot);
+	ExecDropSingleTupleTableSlot(slot);
 
 	/*
 	 * In scan-and-sort mode, complete the sort, then read out all live tuples
@@ -916,6 +906,8 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 		pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
 									 PROGRESS_REPACK_PHASE_WRITE_NEW_HEAP);
 
+		slot = MakeSingleTupleTableSlot(RelationGetDescr(OldHeap),
+										&TTSOpsHeapTuple);
 		for (;;)
 		{
 			HeapTuple	tuple;
@@ -926,33 +918,36 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 			if (tuple == NULL)
 				break;
 
+			/*
+			 * XXX Ideally we should use tuplesort_gettupleslot() above, but
+			 * it retrieves minimal tuples and tuplesort_puttupleslot() cannot
+			 * get the tuple descriptor from tuplesort created by
+			 * tuplesort_begin_cluster().
+			 */
+			ExecStoreHeapTuple(tuple, slot, false);
+
 			n_tuples += 1;
 			if (!concurrent)
-				reform_and_rewrite_tuple(tuple,
-										 OldHeap, NewHeap,
-										 values, isnull,
-										 rwstate);
+				reform_and_rewrite_tuple(slot, reform_slot, rwstate);
 			else
-				heap_insert_for_repack(tuple, OldHeap, NewHeap,
-									   values, isnull, bistate);
+				heap_insert_for_repack(NewHeap, slot, reform_slot, bistate);
 
 			/* Report n_tuples */
 			pgstat_progress_update_param(PROGRESS_REPACK_HEAP_TUPLES_INSERTED,
 										 n_tuples);
 		}
 
+		ExecDropSingleTupleTableSlot(slot);
 		tuplesort_end(tuplesort);
 	}
 
+	ExecDropSingleTupleTableSlot(reform_slot);
+
 	/* Write out any remaining tuples, and fsync if needed */
 	if (rwstate)
 		end_heap_rewrite(rwstate);
 	if (bistate)
 		FreeBulkInsertState(bistate);
-
-	/* Clean up */
-	pfree(values);
-	pfree(isnull);
 }
 
 /*
@@ -2340,21 +2335,45 @@ heapam_scan_sample_next_tuple(TableScanDesc scan, SampleScanState *scanstate,
  * currently only known to happen as an after-effect of ALTER TABLE
  * SET WITHOUT OIDS.
  *
- * So, we must reconstruct the tuple from component Datums.
+ * So, we must reconstruct the tuple from component Datums. 'reform' slot
+ * is a workspace for this reconstruction.
  */
 static void
-reform_and_rewrite_tuple(HeapTuple tuple,
-						 Relation OldHeap, Relation NewHeap,
-						 Datum *values, bool *isnull, RewriteState rwstate)
+reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform,
+						 RewriteState rwstate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple,
+				newtuple;
+	bool		shouldFree,
+				shouldFreeNew;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	/*
+	 * The old tuple will not be modified, so do not request materialization.
+	 * (A copy can be created for specific slot type though, e.g.
+	 * TTSOpsMinimalTuple.)
+	 */
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+
+		/* No need to materialize, copy will be created anyway. */
+		Assert(TTS_IS_VIRTUAL(reform));
+		newtuple = ExecFetchSlotHeapTuple(reform, false, &shouldFreeNew);
+	}
+	else
+	{
+		newtuple = heap_copytuple(tuple);
+		shouldFreeNew = true;
+	}
 
 	/* The heap rewrite module does the rest */
 	rewrite_heap_tuple(rwstate, tuple, newtuple);
 
-	heap_freetuple(newtuple);
+	if (shouldFree)
+		heap_freetuple(tuple);
+	if (shouldFreeNew)
+		heap_freetuple(newtuple);
 }
 
 /*
@@ -2366,6 +2385,9 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * information). Thus we must use heap_insert() both during the
  * catch-up and here.
  *
+ * 'reform' is a slot to use for tuple "reforming", typically to get set
+ * values of dropped columns to NULL.
+ *
  * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical
  * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops
  * this relation, so no logical replication subscription should need the data.
@@ -2374,76 +2396,75 @@ reform_and_rewrite_tuple(HeapTuple tuple,
  * case.
  */
 static void
-heap_insert_for_repack(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-					   Datum *values, bool *isnull, BulkInsertState bistate)
+heap_insert_for_repack(Relation rel, TupleTableSlot *src,
+					   TupleTableSlot *reform, BulkInsertStateData *bistate)
 {
-	HeapTuple	newtuple;
+	HeapTuple	tuple;
+	bool		shouldFree;
+	TupleTableSlot *slot;
 
-	newtuple = reform_tuple(tuple, OldHeap, NewHeap, values, isnull);
+	tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree);
+	if (tuple_needs_reform(tuple, src->tts_tupleDescriptor))
+	{
+		clear_dropped_attributes(tuple, reform);
+		slot = reform;
+	}
+	else
+		slot = src;
 
-	heap_insert(NewHeap, newtuple, GetCurrentCommandId(true),
-				HEAP_INSERT_NO_LOGICAL, bistate);
+	/*
+	 * clear_dropped_attributes() should have deformed the tuple, so nothing
+	 * should depend on it now.
+	 */
+	if (shouldFree)
+		heap_freetuple(tuple);
 
-	heap_freetuple(newtuple);
+	table_tuple_insert(rel, slot, GetCurrentCommandId(true),
+					   TABLE_INSERT_NO_LOGICAL, bistate);
 }
 
-/*
- * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
- *
- * Deform the given tuple, set values of dropped columns to NULL, and fill in
- * any values from attmissingval; then form a new tuple and return it.  If no
- * attributes need to be changed, a copy of the original tuple is returned.
- * Caller is responsible for freeing the returned tuple.
- *
- * XXX this coding assumes that both relations have the same tupledesc.
- */
-static HeapTuple
-reform_tuple(HeapTuple tuple, Relation OldHeap, Relation NewHeap,
-			 Datum *values, bool *isnull)
+static bool
+tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc)
 {
-	TupleDesc	oldTupDesc = RelationGetDescr(OldHeap);
-	TupleDesc	newTupDesc = RelationGetDescr(NewHeap);
-	bool		needs_reform = false;
-
 	/*
 	 * A short tuple might require values from attmissing val, so activate the
 	 * coding unconditionally in that case.  The value might legitimally be
 	 * NULL otherwise, so this is slightly wasteful, but it probably beats
 	 * having to test each attribute for presence of attmissingval each time.
 	 */
-	if (HeapTupleHeaderGetNatts(tuple->t_data) < newTupDesc->natts)
-		needs_reform = true;
+	if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts)
+		return true;
 
-	/*
-	 * If the column has been dropped but a value is still present, we can
-	 * optimize storage now by getting rid of it.
-	 */
-	if (!needs_reform)
+	/* Does it have dropped attributes? */
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		for (int i = 0; i < newTupDesc->natts; i++)
-		{
-			if (TupleDescCompactAttr(newTupDesc, i)->attisdropped &&
-				!heap_attisnull(tuple, i + 1, newTupDesc))
-			{
-				needs_reform = true;
-				break;
-			}
-		}
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped &&
+			!heap_attisnull(tuple, i + 1, tupDesc))
+			return true;
 	}
 
-	/* Skip work if no changes are needed */
-	if (!needs_reform)
-		return heap_copytuple(tuple);
+	return false;
+}
 
-	heap_deform_tuple(tuple, oldTupDesc, values, isnull);
+/*
+ * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack.
+ *
+ * Set values of dropped columns to NULL,
+ */
+static void
+clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform)
+{
+	TupleDesc	tupDesc = reform->tts_tupleDescriptor;
+
+	/* Assuming 'reform' is virtual, this deforms the tuple. */
+	Assert(TTS_IS_VIRTUAL(reform));
+	ExecForceStoreHeapTuple(tuple, reform, false);
 
-	for (int i = 0; i < newTupDesc->natts; i++)
+	for (int i = 0; i < tupDesc->natts; i++)
 	{
-		if (TupleDescCompactAttr(newTupDesc, i)->attisdropped)
-			isnull[i] = true;
+		if (TupleDescCompactAttr(tupDesc, i)->attisdropped)
+			reform->tts_isnull[i] = true;
 	}
-
-	return heap_form_tuple(newTupDesc, values, isnull);
 }
 
 /*
-- 
2.52.0


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v02-0002-Move-functions-to-repack.c.patch



^ permalink  raw  reply  [nested|flat] 130+ messages in thread

end of thread, other threads:[~2026-07-15 08:37 UTC | newest]

Thread overview: 130+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-03-11 13:53 [PATCH 2/7] Do not dereference varattrib_4b in VARSIZE_4B. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-06-16 11:54 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>
2026-07-15 08:37 [PATCH 1/8] Use tuple slot to pass tuples for rewriting. Antonin Houska <[email protected]>

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox