agora inbox for [email protected]help / color / mirror / Atom feed
[PATCH v7 06/16] Provide examples of listing all settings 116+ messages / 2 participants [nested] [flat]
* [PATCH v7 06/16] Provide examples of listing all settings @ 2023-09-25 20:36 Karl O. Pinc <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Karl O. Pinc @ 2023-09-25 20:36 UTC (permalink / raw) This commit is problematic in that it does something that is not done elsewhere in the documentation, it provides example SELECTs as a stand-in for a regular tabular documentation element. It is almost something that should go in its own appendix, but I don't think that is warranted. Likewise, I think that having another tabular documentation element that must be maintained and kept up-to-date is also not appropriate. It is useful to have tabular overviews of all available system settings, as is having an overview of the setting values of your particular cluster. This commit is an attempt in that direction. --- doc/src/sgml/config.sgml | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index 97f9838bfb..0af4e6dcae 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -11,6 +11,43 @@ <para> There are many configuration parameters that affect the behavior of the database system. + A single master list of all parameters and their characteristics is not + provided in this document. + Use the <link linkend="bookindex">index</link> (or web search) to find + configuration parameter documentation by name. + </para> + + <indexterm> + <primary>pg_settings</primary> + <secondary>example queries</secondary> + </indexterm> + + <indexterm> + <primary>configuration</primary> + <secondary>query the current settings</secondary> + </indexterm> + + <para> + The <link linkend="view-pg-settings">pg_settings</link> + <link linkend="tutorial-views">view</link> into + the <link linkend="catalogs">system catalogs</link> provides summaries of + all, or selected, configuration parameters, e.g: + </para> + +<programlisting> +-- Describe all configuration parameters. +SELECT name, short_desc FROM pg_settings ORDER BY name; + +-- Show the current configuration of the connected cluster, database, +-- and session. +SELECT name, setting, unit FROM pg_settings ORDER BY name; + +-- Show the means available to change the setting; whether the setting is +-- per-cluster, per-database, per-session, etc. +SELECT name, context FROM pg_settings ORDER BY name; +</programlisting> + + <para> The first section of this chapter describes how to interact with configuration parameters. Subsequent sections discuss each parameter in detail. -- 2.30.2 --MP_/VSGM3xNEmY7iJyL2wuWRCjV Content-Type: text/x-patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=v7-0007-Cleanup-summary-of-role-powers.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-06-16 11:54 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-06-16 11:54 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 127d4415084..2866a5e696e 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2382,97 +2378,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index ec100e3eef5..fa98b8b9247 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1268,6 +1268,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v01-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
* [PATCH 2/8] Move functions to repack.c. @ 2026-07-15 08:37 Antonin Houska <[email protected]> 0 siblings, 0 replies; 116+ messages in thread From: Antonin Houska @ 2026-07-15 08:37 UTC (permalink / raw) In the next patches, the functions will be called from other modules. Use a separate diff for the move so that the following diffs are a bit easier to read. --- src/backend/access/heap/heapam_handler.c | 97 +----------------------- src/backend/commands/repack.c | 91 ++++++++++++++++++++++ src/include/commands/repack.h | 7 ++ 3 files changed, 99 insertions(+), 96 deletions(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 4db8a16f691..97f850ebf73 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -34,6 +34,7 @@ #include "catalog/storage.h" #include "catalog/storage_xlog.h" #include "commands/progress.h" +#include "commands/repack.h" #include "executor/executor.h" #include "miscadmin.h" #include "pgstat.h" @@ -50,11 +51,6 @@ static void reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, RewriteState rwstate); -static void heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, - BulkInsertStateData *bistate); -static bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); -static void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer, HeapTuple tuple, @@ -2376,97 +2372,6 @@ reform_and_rewrite_tuple(TupleTableSlot *src, TupleTableSlot *reform, heap_freetuple(newtuple); } -/* - * Insert tuple when processing REPACK CONCURRENTLY. - * - * rewriteheap.c is not used in the CONCURRENTLY case because it'd be - * difficult to do the same in the catch-up phase (as the logical - * decoding does not provide us with sufficient visibility - * information). Thus we must use heap_insert() both during the - * catch-up and here. - * - * 'reform' is a slot to use for tuple "reforming", typically to get set - * values of dropped columns to NULL. - * - * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical - * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops - * this relation, so no logical replication subscription should need the data. - * - * BulkInsertState is used because many tuples are inserted in the typical - * case. - */ -static void -heap_insert_for_repack(Relation rel, TupleTableSlot *src, - TupleTableSlot *reform, BulkInsertStateData *bistate) -{ - HeapTuple tuple; - bool shouldFree; - TupleTableSlot *slot; - - tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); - if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) - { - clear_dropped_attributes(tuple, reform); - slot = reform; - } - else - slot = src; - - /* - * clear_dropped_attributes() should have deformed the tuple, so nothing - * should depend on it now. - */ - if (shouldFree) - heap_freetuple(tuple); - - table_tuple_insert(rel, slot, GetCurrentCommandId(true), - TABLE_INSERT_NO_LOGICAL, bistate); -} - -static bool -tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) -{ - /* - * A short tuple might require values from attmissing val, so activate the - * coding unconditionally in that case. The value might legitimally be - * NULL otherwise, so this is slightly wasteful, but it probably beats - * having to test each attribute for presence of attmissingval each time. - */ - if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) - return true; - - /* Does it have dropped attributes? */ - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped && - !heap_attisnull(tuple, i + 1, tupDesc)) - return true; - } - - return false; -} - -/* - * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. - * - * Set values of dropped columns to NULL, - */ -static void -clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) -{ - TupleDesc tupDesc = reform->tts_tupleDescriptor; - - /* Assuming 'reform' is virtual, this deforms the tuple. */ - Assert(TTS_IS_VIRTUAL(reform)); - ExecForceStoreHeapTuple(tuple, reform, false); - - for (int i = 0; i < tupDesc->natts; i++) - { - if (TupleDescCompactAttr(tupDesc, i)->attisdropped) - reform->tts_isnull[i] = true; - } -} - /* * Check visibility of the tuple. */ diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 02883fe34a4..19927c15bf3 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -1288,6 +1288,97 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, return OIDNewHeap; } +/* + * Insert tuple when processing REPACK CONCURRENTLY. + * + * rewriteheap.c is not used in the CONCURRENTLY case because it'd be + * difficult to do the same in the catch-up phase (as the logical + * decoding does not provide us with sufficient visibility + * information). Thus we must use heap_insert() both during the + * catch-up and here. + * + * 'reform' is a slot to use for tuple "reforming", typically to get set + * values of dropped columns to NULL. + * + * We pass the NO_LOGICAL flag to heap_insert() in order to skip logical + * decoding: as soon as REPACK CONCURRENTLY swaps the relation files, it drops + * this relation, so no logical replication subscription should need the data. + * + * BulkInsertState is used because many tuples are inserted in the typical + * case. + */ +void +heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, BulkInsertStateData *bistate) +{ + HeapTuple tuple; + bool shouldFree; + TupleTableSlot *slot; + + tuple = ExecFetchSlotHeapTuple(src, false, &shouldFree); + if (tuple_needs_reform(tuple, src->tts_tupleDescriptor)) + { + clear_dropped_attributes(tuple, reform); + slot = reform; + } + else + slot = src; + + /* + * clear_dropped_attributes() should have deformed the tuple, so nothing + * should depend on it now. + */ + if (shouldFree) + heap_freetuple(tuple); + + table_tuple_insert(rel, slot, GetCurrentCommandId(true), + TABLE_INSERT_NO_LOGICAL, bistate); +} + +bool +tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc) +{ + /* + * A short tuple might require values from attmissing val, so activate the + * coding unconditionally in that case. The value might legitimally be + * NULL otherwise, so this is slightly wasteful, but it probably beats + * having to test each attribute for presence of attmissingval each time. + */ + if (HeapTupleHeaderGetNatts(tuple->t_data) < tupDesc->natts) + return true; + + /* Does it have dropped attributes? */ + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped && + !heap_attisnull(tuple, i + 1, tupDesc)) + return true; + } + + return false; +} + +/* + * Subroutine for reform_and_rewrite_tuple and heap_insert_for_repack. + * + * Set values of dropped columns to NULL, + */ +void +clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform) +{ + TupleDesc tupDesc = reform->tts_tupleDescriptor; + + /* Assuming 'reform' is virtual, this deforms the tuple. */ + Assert(TTS_IS_VIRTUAL(reform)); + ExecForceStoreHeapTuple(tuple, reform, false); + + for (int i = 0; i < tupDesc->natts; i++) + { + if (TupleDescCompactAttr(tupDesc, i)->attisdropped) + reform->tts_isnull[i] = true; + } +} + /* * Do the physical copying of table data. * diff --git a/src/include/commands/repack.h b/src/include/commands/repack.h index 45e5440a311..27105c10591 100644 --- a/src/include/commands/repack.h +++ b/src/include/commands/repack.h @@ -15,6 +15,8 @@ #include <signal.h> +#include "access/hio.h" +#include "nodes/execnodes.h" #include "nodes/parsenodes.h" #include "parser/parse_node.h" #include "storage/lockdefs.h" @@ -48,6 +50,11 @@ extern void mark_index_clustered(Relation rel, Oid indexOid, bool is_internal); extern Oid make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, char relpersistence, LOCKMODE lockmode); +extern void heap_insert_for_repack(Relation rel, TupleTableSlot *src, + TupleTableSlot *reform, + BulkInsertStateData *bistate); +extern bool tuple_needs_reform(HeapTuple tuple, TupleDesc tupDesc); +extern void clear_dropped_attributes(HeapTuple tuple, TupleTableSlot *reform); extern void finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap, bool is_system_catalog, bool swap_toast_by_content, -- 2.52.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v02-0003-Introduce-RepackDest-structure.patch ^ permalink raw reply [nested|flat] 116+ messages in thread
end of thread, other threads:[~2026-07-15 08:37 UTC | newest] Thread overview: 116+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2023-09-25 20:36 [PATCH v7 06/16] Provide examples of listing all settings Karl O. Pinc <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-06-16 11:54 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]> 2026-07-15 08:37 [PATCH 2/8] Move functions to repack.c. Antonin Houska <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox