public inbox for [email protected]
help / color / mirror / Atom feedFrom: Melanie Plageman <[email protected]>
To: Andres Freund <[email protected]>
Cc: Kirill Reshke <[email protected]>
Cc: Chao Li <[email protected]>
Cc: Andrey Borodin <[email protected]>
Cc: Xuneng Zhou <[email protected]>
Cc: Robert Haas <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Cc: Heikki Linnakangas <[email protected]>
Subject: Re: eliminate xl_heap_visible to reduce WAL (and eventually set VM on-access)
Date: Mon, 23 Mar 2026 17:54:13 -0400
Message-ID: <CAAKRu_bgP-DMZs=D2j2N0+U9+uWU5cVagw-yZLOuhYbWj_KwnA@mail.gmail.com> (raw)
In-Reply-To: <CAAKRu_Zj8G4T=HN3QSY7iQvkKSQk-k1fq+eJkjCBNqoSg63z+Q@mail.gmail.com>
References: <CAAKRu_a7ByG2LipFADR7UYnLP4BWQKOV1sajqJC4=R37iO05+A@mail.gmail.com>
<[email protected]>
<[email protected]>
<[email protected]>
<CAAKRu_a+hO4PCptyaPR7AMZd7FjcHfOFKKJT8ouU3KedMud0tQ@mail.gmail.com>
<CAAKRu_Z8Ry_ynNBPAzs_Ry3MQi9NaBgt1ccLgwRsDbxWpocaBg@mail.gmail.com>
<CAAKRu_ZbOp52rnkjf63h5mf94raEKBH7AAbz6QTx-xdH9yLfmQ@mail.gmail.com>
<CAAKRu_b8m+iuupm4ZX+2_V5Xj5u4jCTrU=Tv=epg6p4H2SMkFQ@mail.gmail.com>
<CALdSSPh9hVXNiPwdntWqbMzu5upKy0jBDDe7Un0_Nf2A54R2VQ@mail.gmail.com>
<CAAKRu_a6Cd7JnxhY4A=b_Paxc3UDUDOPeqV3GbzMh=R2KkD-uQ@mail.gmail.com>
<jlsvov4o5xswxjvjhvuchz6y55ncvoc457grvxct7cub5gcxuj@4e2toesujnpr>
<CAAKRu_bsHbvt+VqbjHXsdphKf8fqwBEutRhH3fmo+qUVe=yBKw@mail.gmail.com>
<CAAKRu_ZhHtEaucO--SdYrCjq0zgqk_LPztUD+-QS74A2htXgKw@mail.gmail.com>
<CAAKRu_Zj8G4T=HN3QSY7iQvkKSQk-k1fq+eJkjCBNqoSg63z+Q@mail.gmail.com>
On Sun, Mar 22, 2026 at 3:58 PM Melanie Plageman
<[email protected]> wrote:
>
> I've pushed the first two patches. Attached are the remaining 10. No
> changes were made to those from the previous version.
I'm planning on pushing 0001-0005 in the morning.
I've made some significant changes to 0006 and realized I need some
help. 0006 tracks what relations are modified by a query. This new
version (v44) uses relation oids instead of rt indexes to handle cases
where the same relation appears more than once in the range table
(e.g. INSERT INTO foo SELECT * FROM foo; foo appears twice). It
computes modifiedRelOids (a list of relation OIDs modified by the
query) in the planner and stores them in the PlannedStmt. There is one
big issue I'm not sure how to solve:
For queries like INSERT INTO ptable SELECT * FROM ptable, where ptable
is a partitioned table, though we scan ptable, we don't know when
executing that scan that we will then modify ptable with the insert.
In my patch, I've added find_all_inheritors() when populating
modifiedRelOids, but I realize this probably isn't acceptable to add
to planner from a performance perspective.
I'm looking for other ways to solve the problem. Now, for my use case
(setting the VM), we don't mind setting the VM during the table scan
part of the query. Whatever page gets the inserted tuple will clear
all-visible -- but that is just one page out of many. However, future
users of modifiedRelOids will likely expect it to contain all modified
relation oids.
I could also check when setting up the scan descriptor if the leaf
partition's parents (would have to check full ancestry) are in
modifiedRelOids. This also doesn't address the problem of future users
thinking modifiedRelOids is complete.
Note that it also means partitions that aren't modified will be
included in modifiedRelOids if one of the partitions is being
modified.
I could also just change the name of the modifiedRelOids to something
that doesn't make future users think it's exhaustive.
- Melanie
Attachments:
[text/x-patch] v44-0001-Use-GlobalVisState-in-vacuum-to-determine-page-l.patch (18.4K, 2-v44-0001-Use-GlobalVisState-in-vacuum-to-determine-page-l.patch)
download | inline diff:
From cb89cc1c911f74f66d7febe69cbef95cef5c614e Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Wed, 17 Dec 2025 16:51:05 -0500
Subject: [PATCH v44 01/10] Use GlobalVisState in vacuum to determine page
level visibility
During vacuum's first and third phases, we examine tuples' visibility to
determine if we can set the page all-visible in the visibility map.
Previously, this check compared tuple xmins against a single XID chosen
at the start of vacuum (OldestXmin). We now use GlobalVisState, which
also enables future work to set the VM during on-access pruning, since
ordinary queries have access to GlobalVisState but not OldestXmin.
This also benefits vacuum: in some cases, GlobalVisState may advance
during a vacuum, allowing more pages to become considered all-visible.
And, in the future, we could easily add a heuristic to update
GlobalVisState more frequently during vacuums of large tables.
OldestXmin is still used for freezing and as a backstop to ensure we
don't freeze a dead tuple that wasn't yet prunable according to
GlobalVisState in the rare occurrences where GlobalVisState moves
backwards.
Because comparing a transaction ID against GlobalVisState is more
expensive than comparing against a single XID, we defer this check until
after scanning all tuples on the page. Therefore, we perform the
GlobalVisState check only once per page. This is safe because
visibility_cutoff_xid records the newest live xmin on the page; if it is
globally visible, then the entire page is all-visible.
Using GlobalVisState means on-access pruning can also maintain
visibility_cutoff_xid, which is required to set the visibility map
on-access in the future.
Author: Melanie Plageman <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Chao Li <[email protected]>
Reviewed-by: Kirill Reshke <[email protected]>
Discussion: https://postgr.es/m/flat/bqc4kh5midfn44gnjiqez3bjqv4zogydguvdn446riw45jcf3y%404ez66il7ebvk#c755ef151507aba58471ffaca607e493
---
src/backend/access/heap/heapam_visibility.c | 31 ++++++++++-
src/backend/access/heap/pruneheap.c | 54 ++++++++++---------
src/backend/access/heap/vacuumlazy.c | 60 ++++++++++++++-------
src/backend/access/spgist/spgvacuum.c | 2 +-
src/backend/storage/ipc/procarray.c | 19 ++++---
src/include/access/heapam.h | 4 ++
src/include/utils/snapmgr.h | 8 ++-
7 files changed, 123 insertions(+), 55 deletions(-)
diff --git a/src/backend/access/heap/heapam_visibility.c b/src/backend/access/heap/heapam_visibility.c
index fc64f4343ce..9a7bf331df7 100644
--- a/src/backend/access/heap/heapam_visibility.c
+++ b/src/backend/access/heap/heapam_visibility.c
@@ -1131,6 +1131,32 @@ HeapTupleSatisfiesVacuum(HeapTuple htup, TransactionId OldestXmin,
return res;
}
+/*
+ * Wrapper around GlobalVisTestIsRemovableXid() for use when examining live
+ * tuples. Returns true if the given XID may be considered running by at least
+ * one snapshot.
+ *
+ * This function alone is insufficient to determine tuple visibility; callers
+ * must also consider the XID's commit status. Its purpose is purely semantic:
+ * when applied to live tuples, GlobalVisTestIsRemovableXid() is checking
+ * whether the inserting transaction is still considered running, not whether
+ * the tuple is removable. Live tuples are, by definition, not removable, but
+ * the snapshot criteria for “transaction still running” are identical to
+ * those used for removal XIDs.
+ *
+ * If allow_update is true, the GlobalVisState boundaries may be updated. If
+ * it is false, they definitely will not be updated.
+ *
+ * See the comment above GlobalVisTestIsRemovable[Full]Xid() for details on
+ * the required preconditions for calling this function.
+ */
+bool
+GlobalVisTestXidConsideredRunning(GlobalVisState *state, TransactionId xid,
+ bool allow_update)
+{
+ return !GlobalVisTestIsRemovableXid(state, xid, allow_update);
+}
+
/*
* Work horse for HeapTupleSatisfiesVacuum and similar routines.
*
@@ -1354,7 +1380,7 @@ HeapTupleSatisfiesNonVacuumable(HeapTuple htup, Snapshot snapshot,
{
Assert(TransactionIdIsValid(dead_after));
- if (GlobalVisTestIsRemovableXid(snapshot->vistest, dead_after))
+ if (GlobalVisTestIsRemovableXid(snapshot->vistest, dead_after, true))
res = HEAPTUPLE_DEAD;
}
else
@@ -1420,7 +1446,8 @@ HeapTupleIsSurelyDead(HeapTuple htup, GlobalVisState *vistest)
/* Deleter committed, so tuple is dead if the XID is old enough. */
return GlobalVisTestIsRemovableXid(vistest,
- HeapTupleHeaderGetRawXmax(tuple));
+ HeapTupleHeaderGetRawXmax(tuple),
+ true);
}
/*
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index b383b0fca8b..8eb3afda4bf 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -160,10 +160,13 @@ typedef struct
* all-frozen bits in the visibility map can be set for this page after
* pruning.
*
- * visibility_cutoff_xid is the newest xmin of live tuples on the page.
- * The caller can use it as the conflict horizon, when setting the VM
- * bits. It is only valid if we froze some tuples, and set_all_frozen is
- * true.
+ * visibility_cutoff_xid is the newest xmin of live tuples on the page. It
+ * is used after processing all tuples to determine if the page can be
+ * considered all-visible (if the newest xmin is still considered running
+ * by some snapshot, it cannot be). It is also used by the caller as the
+ * conflict horizon when setting the VM bits, unless we froze all tuples
+ * on the page (in which case the conflict xid was already included in the
+ * WAL record).
*
* NOTE: set_all_visible and set_all_frozen initially don't include
* LP_DEAD items. That's convenient for heap_page_prune_and_freeze() to
@@ -281,7 +284,7 @@ heap_page_prune_opt(Relation relation, Buffer buffer, Buffer *vmbuffer)
*/
vistest = GlobalVisTestFor(relation);
- if (!GlobalVisTestIsRemovableXid(vistest, prune_xid))
+ if (!GlobalVisTestIsRemovableXid(vistest, prune_xid, true))
return;
/*
@@ -1081,6 +1084,19 @@ heap_page_prune_and_freeze(PruneFreezeParams *params,
*/
prune_freeze_plan(&prstate, off_loc);
+ /*
+ * After processing all the live tuples on the page, if the newest xmin
+ * amongst them may be considered running by any snapshot, the page cannot
+ * be all-visible. This should be done before determining whether or not
+ * to opportunistically freeze.
+ */
+ if (prstate.set_all_visible &&
+ TransactionIdIsNormal(prstate.visibility_cutoff_xid) &&
+ GlobalVisTestXidConsideredRunning(prstate.vistest,
+ prstate.visibility_cutoff_xid,
+ true))
+ prstate.set_all_visible = prstate.set_all_frozen = false;
+
/*
* If checksums are enabled, calling heap_prune_satisfies_vacuum() while
* checking tuple visibility information in prune_freeze_plan() may have
@@ -1283,7 +1299,7 @@ heap_prune_satisfies_vacuum(PruneState *prstate, HeapTuple tup)
* if the GlobalVisState has been updated since the beginning of vacuuming
* the relation.
*/
- if (GlobalVisTestIsRemovableXid(prstate->vistest, dead_after))
+ if (GlobalVisTestIsRemovableXid(prstate->vistest, dead_after, true))
return HEAPTUPLE_DEAD;
return res;
@@ -1749,29 +1765,15 @@ heap_prune_record_unchanged_lp_normal(PruneState *prstate, OffsetNumber offnum)
}
/*
- * The inserter definitely committed. But is it old enough
- * that everyone sees it as committed? A FrozenTransactionId
- * is seen as committed to everyone. Otherwise, we check if
- * there is a snapshot that considers this xid to still be
- * running, and if so, we don't consider the page all-visible.
+ * The inserter definitely committed. But we don't know if it
+ * is old enough that everyone sees it as committed. Later,
+ * after processing all the tuples on the page, we'll check if
+ * there is any snapshot that still considers the newest xid
+ * on the page to be running. If so, we don't consider the
+ * page all-visible.
*/
xmin = HeapTupleHeaderGetXmin(htup);
- /*
- * For now always use prstate->cutoffs for this test, because
- * we only update 'set_all_visible' and 'set_all_frozen' when
- * freezing is requested. We could use
- * GlobalVisTestIsRemovableXid instead, if a non-freezing
- * caller wanted to set the VM bit.
- */
- Assert(prstate->cutoffs);
- if (!TransactionIdPrecedes(xmin, prstate->cutoffs->OldestXmin))
- {
- prstate->set_all_visible = false;
- prstate->set_all_frozen = false;
- break;
- }
-
/* Track newest xmin on page. */
if (TransactionIdFollows(xmin, prstate->visibility_cutoff_xid) &&
TransactionIdIsNormal(xmin))
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 1a446050d85..797973d7bd0 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -468,13 +468,14 @@ static void dead_items_cleanup(LVRelState *vacrel);
#ifdef USE_ASSERT_CHECKING
static bool heap_page_is_all_visible(Relation rel, Buffer buf,
- TransactionId OldestXmin,
+ GlobalVisState *vistest,
bool *all_frozen,
TransactionId *visibility_cutoff_xid,
OffsetNumber *logging_offnum);
#endif
static bool heap_page_would_be_all_visible(Relation rel, Buffer buf,
- TransactionId OldestXmin,
+ GlobalVisState *vistest,
+ bool allow_update_vistest,
OffsetNumber *deadoffsets,
int ndeadoffsets,
bool *all_frozen,
@@ -2089,7 +2090,7 @@ lazy_scan_prune(LVRelState *vacrel,
Assert(presult.lpdead_items == 0);
Assert(heap_page_is_all_visible(vacrel->rel, buf,
- vacrel->cutoffs.OldestXmin, &debug_all_frozen,
+ vacrel->vistest, &debug_all_frozen,
&debug_cutoff, &vacrel->offnum));
Assert(presult.set_all_frozen == debug_all_frozen);
@@ -2852,7 +2853,7 @@ lazy_vacuum_heap_page(LVRelState *vacrel, BlockNumber blkno, Buffer buffer,
* done outside the critical section.
*/
if (heap_page_would_be_all_visible(vacrel->rel, buffer,
- vacrel->cutoffs.OldestXmin,
+ vacrel->vistest, true,
deadoffsets, num_offsets,
&all_frozen, &visibility_cutoff_xid,
&vacrel->offnum))
@@ -3614,14 +3615,19 @@ dead_items_cleanup(LVRelState *vacrel)
*/
static bool
heap_page_is_all_visible(Relation rel, Buffer buf,
- TransactionId OldestXmin,
+ GlobalVisState *vistest,
bool *all_frozen,
TransactionId *visibility_cutoff_xid,
OffsetNumber *logging_offnum)
{
-
+ /*
+ * Pass allow_update_vistest as false so that the GlobalVisState
+ * boundaries used here match those used by the pruning code we are
+ * cross-checking. Allowing an update could move the boundaries between
+ * the two calls, causing a spurious assertion failure.
+ */
return heap_page_would_be_all_visible(rel, buf,
- OldestXmin,
+ vistest, false,
NULL, 0,
all_frozen,
visibility_cutoff_xid,
@@ -3642,7 +3648,9 @@ heap_page_is_all_visible(Relation rel, Buffer buf,
* Returns true if the page is all-visible other than the provided
* deadoffsets and false otherwise.
*
- * OldestXmin is used to determine visibility.
+ * vistest is used to determine visibility. If allow_update_vistest is true,
+ * the boundaries of the GlobalVisState may be updated when checking the
+ * visibility of the newest live XID on the page.
*
* Output parameters:
*
@@ -3661,7 +3669,8 @@ heap_page_is_all_visible(Relation rel, Buffer buf,
*/
static bool
heap_page_would_be_all_visible(Relation rel, Buffer buf,
- TransactionId OldestXmin,
+ GlobalVisState *vistest,
+ bool allow_update_vistest,
OffsetNumber *deadoffsets,
int ndeadoffsets,
bool *all_frozen,
@@ -3742,7 +3751,7 @@ heap_page_would_be_all_visible(Relation rel, Buffer buf,
{
TransactionId xmin;
- /* Check comments in lazy_scan_prune. */
+ /* Check heap_prune_record_unchanged_lp_normal comments */
if (!HeapTupleHeaderXminCommitted(tuple.t_data))
{
all_visible = false;
@@ -3751,16 +3760,17 @@ heap_page_would_be_all_visible(Relation rel, Buffer buf,
}
/*
- * The inserter definitely committed. But is it old enough
- * that everyone sees it as committed?
+ * The inserter definitely committed. But we don't know if
+ * it is old enough that everyone sees it as committed.
+ * Don't check that now.
+ *
+ * If we scan all tuples without finding one that prevents
+ * the page from being all-visible, we then check whether
+ * any snapshot still considers the newest XID on the page
+ * to be running. In that case, the page is not considered
+ * all-visible.
*/
xmin = HeapTupleHeaderGetXmin(tuple.t_data);
- if (!TransactionIdPrecedes(xmin, OldestXmin))
- {
- all_visible = false;
- *all_frozen = false;
- break;
- }
/* Track newest xmin on page. */
if (TransactionIdFollows(xmin, *visibility_cutoff_xid) &&
@@ -3789,6 +3799,20 @@ heap_page_would_be_all_visible(Relation rel, Buffer buf,
}
} /* scan along page */
+ /*
+ * After processing all the live tuples on the page, if the newest xmin
+ * among them may still be considered running by any snapshot, the page
+ * cannot be all-visible.
+ */
+ if (all_visible &&
+ TransactionIdIsNormal(*visibility_cutoff_xid) &&
+ GlobalVisTestXidConsideredRunning(vistest, *visibility_cutoff_xid,
+ allow_update_vistest))
+ {
+ all_visible = false;
+ *all_frozen = false;
+ }
+
/* Clear the offset information once we have processed the given page. */
*logging_offnum = InvalidOffsetNumber;
diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c
index 6b7117b56b2..c461f8dc02d 100644
--- a/src/backend/access/spgist/spgvacuum.c
+++ b/src/backend/access/spgist/spgvacuum.c
@@ -536,7 +536,7 @@ vacuumRedirectAndPlaceholder(Relation index, Relation heaprel, Buffer buffer)
*/
if (dt->tupstate == SPGIST_REDIRECT &&
(!TransactionIdIsValid(dt->xid) ||
- GlobalVisTestIsRemovableXid(vistest, dt->xid)))
+ GlobalVisTestIsRemovableXid(vistest, dt->xid, true)))
{
dt->tupstate = SPGIST_PLACEHOLDER;
Assert(opaque->nRedirection > 0);
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 0f913897acc..27e5adeebfb 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -4223,11 +4223,17 @@ GlobalVisUpdate(void)
* The state passed needs to have been initialized for the relation fxid is
* from (NULL is also OK), otherwise the result may not be correct.
*
+ * If allow_update is false, the GlobalVisState boundaries will not be updated
+ * even if it would otherwise be beneficial. This is useful for callers that
+ * do not want GlobalVisState to advance at all, for example because they need
+ * a conservative answer based on the current boundaries.
+ *
* See comment for GlobalVisState for details.
*/
bool
GlobalVisTestIsRemovableFullXid(GlobalVisState *state,
- FullTransactionId fxid)
+ FullTransactionId fxid,
+ bool allow_update)
{
/*
* If fxid is older than maybe_needed bound, it definitely is visible to
@@ -4248,7 +4254,7 @@ GlobalVisTestIsRemovableFullXid(GlobalVisState *state,
* might not exist a snapshot considering fxid running. If it makes sense,
* update boundaries and recheck.
*/
- if (GlobalVisTestShouldUpdate(state))
+ if (allow_update && GlobalVisTestShouldUpdate(state))
{
GlobalVisUpdate();
@@ -4268,7 +4274,8 @@ GlobalVisTestIsRemovableFullXid(GlobalVisState *state,
* relfrozenxid).
*/
bool
-GlobalVisTestIsRemovableXid(GlobalVisState *state, TransactionId xid)
+GlobalVisTestIsRemovableXid(GlobalVisState *state, TransactionId xid,
+ bool allow_update)
{
FullTransactionId fxid;
@@ -4282,7 +4289,7 @@ GlobalVisTestIsRemovableXid(GlobalVisState *state, TransactionId xid)
*/
fxid = FullXidRelativeTo(state->definitely_needed, xid);
- return GlobalVisTestIsRemovableFullXid(state, fxid);
+ return GlobalVisTestIsRemovableFullXid(state, fxid, allow_update);
}
/*
@@ -4296,7 +4303,7 @@ GlobalVisCheckRemovableFullXid(Relation rel, FullTransactionId fxid)
state = GlobalVisTestFor(rel);
- return GlobalVisTestIsRemovableFullXid(state, fxid);
+ return GlobalVisTestIsRemovableFullXid(state, fxid, true);
}
/*
@@ -4310,7 +4317,7 @@ GlobalVisCheckRemovableXid(Relation rel, TransactionId xid)
state = GlobalVisTestFor(rel);
- return GlobalVisTestIsRemovableXid(state, xid);
+ return GlobalVisTestIsRemovableXid(state, xid, true);
}
/*
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 305ecc31a9e..ca5e8d1794f 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -480,6 +480,10 @@ extern TM_Result HeapTupleSatisfiesUpdate(HeapTuple htup, CommandId curcid,
Buffer buffer);
extern HTSV_Result HeapTupleSatisfiesVacuum(HeapTuple htup, TransactionId OldestXmin,
Buffer buffer);
+
+extern bool GlobalVisTestXidConsideredRunning(GlobalVisState *state,
+ TransactionId xid,
+ bool allow_update);
extern HTSV_Result HeapTupleSatisfiesVacuumHorizon(HeapTuple htup, Buffer buffer,
TransactionId *dead_after);
extern void HeapTupleSetHintBits(HeapTupleHeader tuple, Buffer buffer,
diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h
index 8c919d2640e..c7a869bc2b2 100644
--- a/src/include/utils/snapmgr.h
+++ b/src/include/utils/snapmgr.h
@@ -115,8 +115,12 @@ extern char *ExportSnapshot(Snapshot snapshot);
*/
typedef struct GlobalVisState GlobalVisState;
extern GlobalVisState *GlobalVisTestFor(Relation rel);
-extern bool GlobalVisTestIsRemovableXid(GlobalVisState *state, TransactionId xid);
-extern bool GlobalVisTestIsRemovableFullXid(GlobalVisState *state, FullTransactionId fxid);
+extern bool GlobalVisTestIsRemovableXid(GlobalVisState *state,
+ TransactionId xid,
+ bool allow_update);
+extern bool GlobalVisTestIsRemovableFullXid(GlobalVisState *state,
+ FullTransactionId fxid,
+ bool allow_update);
extern bool GlobalVisCheckRemovableXid(Relation rel, TransactionId xid);
extern bool GlobalVisCheckRemovableFullXid(Relation rel, FullTransactionId fxid);
--
2.43.0
[text/x-patch] v44-0002-Keep-newest-live-XID-up-to-date-even-if-page-not.patch (15.6K, 3-v44-0002-Keep-newest-live-XID-up-to-date-even-if-page-not.patch)
download | inline diff:
From 0968ef2bc8aabd448a3ce97365f74859d83cb68d Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Sat, 28 Feb 2026 16:06:51 -0500
Subject: [PATCH v44 02/10] Keep newest live XID up-to-date even if page not
all-visible
During pruning, we keep track of the newest xmin of live tuples on the
page visible to all running and future transactions so that we can use
it later as the snapshot conflict horizon when setting the VM if the
page turns out to be all-visible.
Previously, we stopped updating this value once we determined the page
was not all-visible. However, maintaining it even when the page is not
all-visible is inexpensive and makes the snapshot conflict horizon
calculation clearer. This guarantees it won't contain a stale value.
Since we'll keep it up to date all the time now anyway, there's no
reason not to maintain set_all_visible for on-access pruning. This will
allow us to set the VM on-access in the future.
Author: Melanie Plageman <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Chao Li <[email protected]>
Discussion: https://postgr.es/m/bqc4kh5midfn44gnjiqez3bjqv4zogydguvdn446riw45jcf3y%404ez66il7ebvk
---
src/backend/access/heap/pruneheap.c | 138 +++++++++++----------------
src/backend/access/heap/vacuumlazy.c | 30 +++---
2 files changed, 73 insertions(+), 95 deletions(-)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index 8eb3afda4bf..301fcfe7024 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -129,6 +129,9 @@ typedef struct
/* Bits in the vmbuffer for this heap page */
uint8 old_vmbits;
+ /* The newest xmin of live tuples on the page */
+ TransactionId newest_live_xid;
+
/*-------------------------------------------------------
* Information about what was done
*
@@ -160,14 +163,6 @@ typedef struct
* all-frozen bits in the visibility map can be set for this page after
* pruning.
*
- * visibility_cutoff_xid is the newest xmin of live tuples on the page. It
- * is used after processing all tuples to determine if the page can be
- * considered all-visible (if the newest xmin is still considered running
- * by some snapshot, it cannot be). It is also used by the caller as the
- * conflict horizon when setting the VM bits, unless we froze all tuples
- * on the page (in which case the conflict xid was already included in the
- * WAL record).
- *
* NOTE: set_all_visible and set_all_frozen initially don't include
* LP_DEAD items. That's convenient for heap_page_prune_and_freeze() to
* use them to decide whether to freeze the page or not. The
@@ -177,7 +172,6 @@ typedef struct
*/
bool set_all_visible;
bool set_all_frozen;
- TransactionId visibility_cutoff_xid;
} PruneState;
/*
@@ -458,53 +452,43 @@ prune_freeze_setup(PruneFreezeParams *params,
prstate->deadoffsets = presult->deadoffsets;
/*
- * Vacuum may update the VM after we're done. We can keep track of
- * whether the page will be all-visible and all-frozen after pruning and
- * freezing to help the caller to do that.
- *
- * Currently, only VACUUM sets the VM bits. To save the effort, only do
- * the bookkeeping if the caller needs it. Currently, that's tied to
- * HEAP_PAGE_PRUNE_FREEZE, but it could be a separate flag if you wanted
- * to update the VM bits without also freezing or freeze without also
- * setting the VM bits.
+ * We track whether the page will be all-visible/all-frozen at the end of
+ * pruning and freezing. While examining tuple visibility, we'll set
+ * set_all_visible to false if there are tuples on the page not visible to
+ * all running and future transactions. set_all_visible is always
+ * maintained but only VACUUM will set the VM if the page ends up being
+ * all-visible.
*
- * In addition to telling the caller whether it can set the VM bit, we
- * also use 'set_all_visible' and 'set_all_frozen' for our own
- * decision-making. If the whole page would become frozen, we consider
- * opportunistically freezing tuples. We will not be able to freeze the
- * whole page if there are tuples present that are not visible to everyone
- * or if there are dead tuples which are not yet removable. However, dead
- * tuples which will be removed by the end of vacuuming should not
- * preclude us from opportunistically freezing. Because of that, we do
- * not immediately clear set_all_visible and set_all_frozen when we see
- * LP_DEAD items. We fix that after scanning the line pointers. We must
- * correct set_all_visible and set_all_frozen before we return them to the
- * caller, so that the caller doesn't set the VM bits incorrectly.
+ * We also keep track of the newest live XID, which is used to calculate
+ * the snapshot conflict horizon for a WAL record setting the VM.
*/
- if (prstate->attempt_freeze)
- {
- prstate->set_all_visible = true;
- prstate->set_all_frozen = true;
- }
- else
- {
- /*
- * Initializing to false allows skipping the work to update them in
- * heap_prune_record_unchanged_lp_normal().
- */
- prstate->set_all_visible = false;
- prstate->set_all_frozen = false;
- }
+ prstate->set_all_visible = true;
+ prstate->newest_live_xid = InvalidTransactionId;
/*
- * The visibility cutoff xid is the newest xmin of live tuples on the
- * page. In the common case, this will be set as the conflict horizon the
- * caller can use for updating the VM. If, at the end of freezing and
- * pruning, the page is all-frozen, there is no possibility that any
- * running transaction on the standby does not see tuples on the page as
- * all-visible, so the conflict horizon remains InvalidTransactionId.
+ * Currently, only VACUUM performs freezing, but other callers may in the
+ * future. We must initialize set_all_frozen based on whether or not the
+ * caller passed HEAP_PAGE_PRUNE_FREEZE, because if they did not, we won't
+ * call heap_prepare_freeze_tuple() for each tuple, and set_all_frozen
+ * will never be cleared for tuples that need freezing. This would lead to
+ * incorrectly setting the visibility map all-frozen for this page.
+ *
+ * When freezing is not required (no XIDs/MXIDs older than the freeze
+ * cutoff), we may still choose to "opportunistically" freeze if doing so
+ * would make the page all-frozen.
+ *
+ * We will not be able to freeze the whole page at the end of vacuum if
+ * there are tuples present that are not visible to everyone or if there
+ * are dead tuples which will not be removable. However, dead tuples that
+ * will be removed by the end of vacuum should not prevent this
+ * opportunistic freezing.
+ *
+ * Therefore, we do not clear set_all_visible and set_all_frozen when we
+ * encounter LP_DEAD items. Instead, we correct them after deciding
+ * whether to freeze, but before updating the VM, to avoid setting the VM
+ * bits incorrectly.
*/
- prstate->visibility_cutoff_xid = InvalidTransactionId;
+ prstate->set_all_frozen = prstate->attempt_freeze;
}
/*
@@ -734,7 +718,6 @@ heap_page_will_freeze(bool did_tuple_hint_fpi,
if (!prstate->attempt_freeze)
{
Assert(!prstate->set_all_frozen && prstate->nfrozen == 0);
- Assert(prstate->lpdead_items == 0 || !prstate->set_all_visible);
return false;
}
@@ -1012,9 +995,8 @@ prune_freeze_fast_path(PruneState *prstate, PruneFreezeResult *presult)
* HEAP_PAGE_PRUNE_FREEZE option is passed, we also set
* presult->set_all_visible and presult->set_all_frozen after determining
* whether or not to opportunistically freeze, to indicate if the VM bits can
- * be set. They are always set to false when the HEAP_PAGE_PRUNE_FREEZE
- * option is not passed, because at the moment only callers that also freeze
- * need that information.
+ * be set. 'set_all_frozen' is always set to false when the
+ * HEAP_PAGE_PRUNE_FREEZE option is not passed.
*
* presult contains output parameters needed by callers, such as the number of
* tuples removed and the offsets of dead items on the page after pruning.
@@ -1091,9 +1073,9 @@ heap_page_prune_and_freeze(PruneFreezeParams *params,
* to opportunistically freeze.
*/
if (prstate.set_all_visible &&
- TransactionIdIsNormal(prstate.visibility_cutoff_xid) &&
+ TransactionIdIsNormal(prstate.newest_live_xid) &&
GlobalVisTestXidConsideredRunning(prstate.vistest,
- prstate.visibility_cutoff_xid,
+ prstate.newest_live_xid,
true))
prstate.set_all_visible = prstate.set_all_frozen = false;
@@ -1245,7 +1227,7 @@ heap_page_prune_and_freeze(PruneFreezeParams *params,
if (presult->set_all_frozen)
presult->vm_conflict_horizon = InvalidTransactionId;
else
- presult->vm_conflict_horizon = prstate.visibility_cutoff_xid;
+ presult->vm_conflict_horizon = prstate.newest_live_xid;
presult->lpdead_items = prstate.lpdead_items;
/* the presult->deadoffsets array was already filled in */
@@ -1706,6 +1688,7 @@ static void
heap_prune_record_unchanged_lp_normal(PruneState *prstate, OffsetNumber offnum)
{
HeapTupleHeader htup;
+ TransactionId xmin;
Page page = prstate->page;
Assert(!prstate->processed[offnum]);
@@ -1753,32 +1736,27 @@ heap_prune_record_unchanged_lp_normal(PruneState *prstate, OffsetNumber offnum)
* See SetHintBits for more info. Check that the tuple is hinted
* xmin-committed because of that.
*/
- if (prstate->set_all_visible)
+ if (!HeapTupleHeaderXminCommitted(htup))
{
- TransactionId xmin;
+ prstate->set_all_visible = false;
+ prstate->set_all_frozen = false;
+ break;
+ }
- if (!HeapTupleHeaderXminCommitted(htup))
- {
- prstate->set_all_visible = false;
- prstate->set_all_frozen = false;
- break;
- }
+ /*
+ * The inserter definitely committed. But we don't know if it is
+ * old enough that everyone sees it as committed. Later, after
+ * processing all the tuples on the page, we'll check if there is
+ * any snapshot that still considers the newest xid on the page to
+ * be running. If so, we don't consider the page all-visible.
+ */
+ xmin = HeapTupleHeaderGetXmin(htup);
- /*
- * The inserter definitely committed. But we don't know if it
- * is old enough that everyone sees it as committed. Later,
- * after processing all the tuples on the page, we'll check if
- * there is any snapshot that still considers the newest xid
- * on the page to be running. If so, we don't consider the
- * page all-visible.
- */
- xmin = HeapTupleHeaderGetXmin(htup);
+ /* Track newest xmin on page. */
+ if (TransactionIdFollows(xmin, prstate->newest_live_xid) &&
+ TransactionIdIsNormal(xmin))
+ prstate->newest_live_xid = xmin;
- /* Track newest xmin on page. */
- if (TransactionIdFollows(xmin, prstate->visibility_cutoff_xid) &&
- TransactionIdIsNormal(xmin))
- prstate->visibility_cutoff_xid = xmin;
- }
break;
case HEAPTUPLE_RECENTLY_DEAD:
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 797973d7bd0..696919e35dd 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -470,7 +470,7 @@ static void dead_items_cleanup(LVRelState *vacrel);
static bool heap_page_is_all_visible(Relation rel, Buffer buf,
GlobalVisState *vistest,
bool *all_frozen,
- TransactionId *visibility_cutoff_xid,
+ TransactionId *newest_live_xid,
OffsetNumber *logging_offnum);
#endif
static bool heap_page_would_be_all_visible(Relation rel, Buffer buf,
@@ -479,7 +479,7 @@ static bool heap_page_would_be_all_visible(Relation rel, Buffer buf,
OffsetNumber *deadoffsets,
int ndeadoffsets,
bool *all_frozen,
- TransactionId *visibility_cutoff_xid,
+ TransactionId *newest_live_xid,
OffsetNumber *logging_offnum);
static void update_relstats_all_indexes(LVRelState *vacrel);
static void vacuum_error_callback(void *arg);
@@ -2829,7 +2829,7 @@ lazy_vacuum_heap_page(LVRelState *vacrel, BlockNumber blkno, Buffer buffer,
Page page = BufferGetPage(buffer);
OffsetNumber unused[MaxHeapTuplesPerPage];
int nunused = 0;
- TransactionId visibility_cutoff_xid;
+ TransactionId newest_live_xid;
TransactionId conflict_xid = InvalidTransactionId;
bool all_frozen;
LVSavedErrInfo saved_err_info;
@@ -2855,14 +2855,14 @@ lazy_vacuum_heap_page(LVRelState *vacrel, BlockNumber blkno, Buffer buffer,
if (heap_page_would_be_all_visible(vacrel->rel, buffer,
vacrel->vistest, true,
deadoffsets, num_offsets,
- &all_frozen, &visibility_cutoff_xid,
+ &all_frozen, &newest_live_xid,
&vacrel->offnum))
{
vmflags |= VISIBILITYMAP_ALL_VISIBLE;
if (all_frozen)
{
vmflags |= VISIBILITYMAP_ALL_FROZEN;
- Assert(!TransactionIdIsValid(visibility_cutoff_xid));
+ Assert(!TransactionIdIsValid(newest_live_xid));
}
/*
@@ -2903,7 +2903,7 @@ lazy_vacuum_heap_page(LVRelState *vacrel, BlockNumber blkno, Buffer buffer,
visibilitymap_set_vmbits(blkno,
vmbuffer, vmflags,
vacrel->rel->rd_locator);
- conflict_xid = visibility_cutoff_xid;
+ conflict_xid = newest_live_xid;
}
/*
@@ -3617,7 +3617,7 @@ static bool
heap_page_is_all_visible(Relation rel, Buffer buf,
GlobalVisState *vistest,
bool *all_frozen,
- TransactionId *visibility_cutoff_xid,
+ TransactionId *newest_live_xid,
OffsetNumber *logging_offnum)
{
/*
@@ -3630,7 +3630,7 @@ heap_page_is_all_visible(Relation rel, Buffer buf,
vistest, false,
NULL, 0,
all_frozen,
- visibility_cutoff_xid,
+ newest_live_xid,
logging_offnum);
}
#endif
@@ -3655,7 +3655,7 @@ heap_page_is_all_visible(Relation rel, Buffer buf,
* Output parameters:
*
* - *all_frozen: true if every tuple on the page is frozen
- * - *visibility_cutoff_xid: newest xmin; valid only if page is all-visible
+ * - *newest_live_xid: newest xmin of live tuples on the page
* - *logging_offnum: OffsetNumber of current tuple being processed;
* used by vacuum's error callback system.
*
@@ -3674,7 +3674,7 @@ heap_page_would_be_all_visible(Relation rel, Buffer buf,
OffsetNumber *deadoffsets,
int ndeadoffsets,
bool *all_frozen,
- TransactionId *visibility_cutoff_xid,
+ TransactionId *newest_live_xid,
OffsetNumber *logging_offnum)
{
Page page = BufferGetPage(buf);
@@ -3684,7 +3684,7 @@ heap_page_would_be_all_visible(Relation rel, Buffer buf,
bool all_visible = true;
int matched_dead_count = 0;
- *visibility_cutoff_xid = InvalidTransactionId;
+ *newest_live_xid = InvalidTransactionId;
*all_frozen = true;
Assert(ndeadoffsets == 0 || deadoffsets);
@@ -3773,9 +3773,9 @@ heap_page_would_be_all_visible(Relation rel, Buffer buf,
xmin = HeapTupleHeaderGetXmin(tuple.t_data);
/* Track newest xmin on page. */
- if (TransactionIdFollows(xmin, *visibility_cutoff_xid) &&
+ if (TransactionIdFollows(xmin, *newest_live_xid) &&
TransactionIdIsNormal(xmin))
- *visibility_cutoff_xid = xmin;
+ *newest_live_xid = xmin;
/* Check whether this tuple is already frozen or not */
if (all_visible && *all_frozen &&
@@ -3805,8 +3805,8 @@ heap_page_would_be_all_visible(Relation rel, Buffer buf,
* cannot be all-visible.
*/
if (all_visible &&
- TransactionIdIsNormal(*visibility_cutoff_xid) &&
- GlobalVisTestXidConsideredRunning(vistest, *visibility_cutoff_xid,
+ TransactionIdIsNormal(*newest_live_xid) &&
+ GlobalVisTestXidConsideredRunning(vistest, *newest_live_xid,
allow_update_vistest))
{
all_visible = false;
--
2.43.0
[text/x-patch] v44-0003-WAL-log-VM-setting-during-vacuum-phase-I-in-XLOG.patch (23.2K, 4-v44-0003-WAL-log-VM-setting-during-vacuum-phase-I-in-XLOG.patch)
download | inline diff:
From c440d93887ed97ee8ab42004da76417e29fa2a92 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Tue, 2 Dec 2025 15:07:42 -0500
Subject: [PATCH v44 03/10] WAL log VM setting during vacuum phase I in
XLOG_HEAP2_PRUNE_VACUUM_SCAN
Vacuum no longer emits a separate WAL record for each page set
all-visible or all-frozen during phase I. Instead, visibility map
updates are now included in the XLOG_HEAP2_PRUNE_VACUUM_SCAN record that
is already emitted for pruning and freezing.
Previously, heap_page_prune_and_freeze() determined whether a page was
all-visible, but the corresponding VM bits were only set later in
lazy_scan_prune(). Now the VM is updated immediately in
heap_page_prune_and_freeze(), at the same time as the heap
modifications. This reduces WAL volume produced by vacuum.
For now, this change applies only to vacuum phase I, not to pruning
performed during normal page access.
Author: Melanie Plageman <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Kirill Reshke <[email protected]>
Reviewed-by: Chao Li <[email protected]>
Earlier version Reviewed-by: Robert Haas <[email protected]>
Discussion: https://postgr.es/m/flat/CAAKRu_ZMw6Npd_qm2KM%2BFwQ3cMOMx1Dh3VMhp8-V7SOLxdK9-g%40mail.gmail.com
---
src/backend/access/heap/pruneheap.c | 245 +++++++++++++++++++--------
src/backend/access/heap/vacuumlazy.c | 113 ++----------
src/include/access/heapam.h | 37 ++--
3 files changed, 205 insertions(+), 190 deletions(-)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index 301fcfe7024..4d6d5e92773 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -72,6 +72,21 @@ typedef struct
OffsetNumber nowunused[MaxHeapTuplesPerPage];
HeapTupleFreeze frozen[MaxHeapTuplesPerPage];
+ /*
+ * set_all_visible and set_all_frozen indicate if the all-visible and
+ * all-frozen bits in the visibility map can be set for this page after
+ * pruning.
+ *
+ * NOTE: set_all_visible and set_all_frozen initially don't include
+ * LP_DEAD items. That's convenient for heap_page_prune_and_freeze() to
+ * use them to decide whether to opportunistically freeze the page or not.
+ * The set_all_visible and set_all_frozen values ultimately used to set
+ * the VM are adjusted to include LP_DEAD items after we determine whether
+ * or not to opportunistically freeze.
+ */
+ bool set_all_visible;
+ bool set_all_frozen;
+
/*-------------------------------------------------------
* Working state for HOT chain processing
*-------------------------------------------------------
@@ -122,12 +137,16 @@ typedef struct
/*
* Caller must provide a pinned vmbuffer corresponding to the heap block
* passed to heap_page_prune_and_freeze(). We will fix any corruption
- * found in the VM.
+ * found in the VM and set the VM if the page is all-visible/all-frozen.
*/
Buffer vmbuffer;
- /* Bits in the vmbuffer for this heap page */
+ /*
+ * The state of the VM bits at the beginning of pruning and the state they
+ * will be in at the end.
+ */
uint8 old_vmbits;
+ uint8 new_vmbits;
/* The newest xmin of live tuples on the page */
TransactionId newest_live_xid;
@@ -157,21 +176,6 @@ typedef struct
*/
int lpdead_items; /* number of items in the array */
OffsetNumber *deadoffsets; /* points directly to presult->deadoffsets */
-
- /*
- * set_all_visible and set_all_frozen indicate if the all-visible and
- * all-frozen bits in the visibility map can be set for this page after
- * pruning.
- *
- * NOTE: set_all_visible and set_all_frozen initially don't include
- * LP_DEAD items. That's convenient for heap_page_prune_and_freeze() to
- * use them to decide whether to freeze the page or not. The
- * set_all_visible and set_all_frozen values returned to the caller are
- * adjusted to include LP_DEAD items after we determine whether to
- * opportunistically freeze.
- */
- bool set_all_visible;
- bool set_all_frozen;
} PruneState;
/*
@@ -228,6 +232,7 @@ static void page_verify_redirects(Page page);
static bool heap_page_will_freeze(bool did_tuple_hint_fpi, bool do_prune, bool do_hint_prune,
PruneState *prstate);
+static bool heap_page_will_set_vm(PruneState *prstate, PruneReason reason);
/*
@@ -395,6 +400,7 @@ prune_freeze_setup(PruneFreezeParams *params,
Assert(BufferIsValid(params->vmbuffer));
prstate->vmbuffer = params->vmbuffer;
+ prstate->new_vmbits = 0;
prstate->old_vmbits = visibilitymap_get_status(prstate->relation,
prstate->block,
&prstate->vmbuffer);
@@ -907,6 +913,42 @@ heap_page_fix_vm_corruption(PruneState *prstate, OffsetNumber offnum,
}
}
+/*
+ * Decide whether to set the visibility map bits (all-visible and all-frozen)
+ * for heap_blk using information from the PruneState and VM.
+ *
+ * This function does not actually set the VM bits or page-level visibility
+ * hint, PD_ALL_VISIBLE.
+ *
+ * Returns true if one or both VM bits should be set and false otherwise.
+ */
+static bool
+heap_page_will_set_vm(PruneState *prstate, PruneReason reason)
+{
+ /*
+ * Though on-access pruning maintains prstate->set_all_visible, we don't
+ * set the VM on-access for now.
+ */
+ if (reason == PRUNE_ON_ACCESS)
+ return false;
+
+ if (!prstate->set_all_visible)
+ return false;
+
+ prstate->new_vmbits = VISIBILITYMAP_ALL_VISIBLE;
+
+ if (prstate->set_all_frozen)
+ prstate->new_vmbits |= VISIBILITYMAP_ALL_FROZEN;
+
+ if (prstate->new_vmbits == prstate->old_vmbits)
+ {
+ prstate->new_vmbits = 0;
+ return false;
+ }
+
+ return true;
+}
+
/*
* If the page is already all-frozen, or already all-visible and freezing
* won't be attempted, there is no remaining work and we can use the fast path
@@ -940,8 +982,6 @@ prune_freeze_fast_path(PruneState *prstate, PruneFreezeResult *presult)
/* We'll fill in presult for the caller */
memset(presult, 0, sizeof(PruneFreezeResult));
- presult->old_vmbits = prstate->old_vmbits;
-
/* Clear any stale prune hint */
if (TransactionIdIsValid(PageGetPruneXid(page)))
{
@@ -976,7 +1016,8 @@ prune_freeze_fast_path(PruneState *prstate, PruneFreezeResult *presult)
/*
* Prune and repair fragmentation and potentially freeze tuples on the
- * specified page.
+ * specified page. If the page's visibility status has changed, update it in
+ * the VM.
*
* Caller must have pin and buffer cleanup lock on the page. Note that we
* don't update the FSM information for page on caller's behalf. Caller might
@@ -991,12 +1032,10 @@ prune_freeze_fast_path(PruneState *prstate, PruneFreezeResult *presult)
* tuples if it's required in order to advance relfrozenxid / relminmxid, or
* if it's considered advantageous for overall system performance to do so
* now. The 'params.cutoffs', 'presult', 'new_relfrozen_xid' and
- * 'new_relmin_mxid' arguments are required when freezing. When
- * HEAP_PAGE_PRUNE_FREEZE option is passed, we also set
- * presult->set_all_visible and presult->set_all_frozen after determining
- * whether or not to opportunistically freeze, to indicate if the VM bits can
- * be set. 'set_all_frozen' is always set to false when the
- * HEAP_PAGE_PRUNE_FREEZE option is not passed.
+ * 'new_relmin_mxid' arguments are required when freezing.
+ *
+ * A vmbuffer corresponding to the heap page is also passed and if the page is
+ * found to be all-visible/all-frozen, we will set it in the VM.
*
* presult contains output parameters needed by callers, such as the number of
* tuples removed and the offsets of dead items on the page after pruning.
@@ -1024,8 +1063,10 @@ heap_page_prune_and_freeze(PruneFreezeParams *params,
bool do_freeze;
bool do_prune;
bool do_hint_prune;
+ bool do_set_vm;
bool did_tuple_hint_fpi;
int64 fpi_before = pgWalUsage.wal_fpi;
+ TransactionId conflict_xid;
/* Initialize prstate */
prune_freeze_setup(params,
@@ -1124,6 +1165,31 @@ heap_page_prune_and_freeze(PruneFreezeParams *params,
prstate.set_all_visible = prstate.set_all_frozen = false;
Assert(!prstate.set_all_frozen || prstate.set_all_visible);
+ Assert(!prstate.set_all_visible || (prstate.lpdead_items == 0));
+
+ do_set_vm = heap_page_will_set_vm(&prstate, params->reason);
+
+ /*
+ * new_vmbits should be 0 regardless of whether or not the page is
+ * all-visible if we do not intend to set the VM.
+ */
+ Assert(do_set_vm || prstate.new_vmbits == 0);
+
+ /*
+ * The snapshot conflict horizon for the whole record is the most
+ * conservative (newest) horizon required by any change in the record.
+ */
+ conflict_xid = InvalidTransactionId;
+ if (do_set_vm)
+ conflict_xid = prstate.newest_live_xid;
+ if (do_freeze && TransactionIdFollows(prstate.pagefrz.FreezePageConflictXid, conflict_xid))
+ conflict_xid = prstate.pagefrz.FreezePageConflictXid;
+ if (do_prune && TransactionIdFollows(prstate.latest_xid_removed, conflict_xid))
+ conflict_xid = prstate.latest_xid_removed;
+
+ /* Lock vmbuffer before entering a critical section */
+ if (do_set_vm)
+ LockBuffer(prstate.vmbuffer, BUFFER_LOCK_EXCLUSIVE);
/* Any error while applying the changes is critical */
START_CRIT_SECTION();
@@ -1145,14 +1211,17 @@ heap_page_prune_and_freeze(PruneFreezeParams *params,
/*
* If that's all we had to do to the page, this is a non-WAL-logged
- * hint. If we are going to freeze or prune the page, we will mark
- * the buffer dirty below.
+ * hint. If we are going to freeze or prune the page or set
+ * PD_ALL_VISIBLE, we will mark the buffer dirty below.
+ *
+ * Setting PD_ALL_VISIBLE is fully WAL-logged because it is forbidden
+ * for the VM to be set and PD_ALL_VISIBLE to be clear.
*/
- if (!do_freeze && !do_prune)
+ if (!do_freeze && !do_prune && !do_set_vm)
MarkBufferDirtyHint(prstate.buffer, true);
}
- if (do_prune || do_freeze)
+ if (do_prune || do_freeze || do_set_vm)
{
/* Apply the planned item changes and repair page fragmentation. */
if (do_prune)
@@ -1166,6 +1235,27 @@ heap_page_prune_and_freeze(PruneFreezeParams *params,
if (do_freeze)
heap_freeze_prepared_tuples(prstate.buffer, prstate.frozen, prstate.nfrozen);
+ /* Set the visibility map and page visibility hint */
+ if (do_set_vm)
+ {
+ /*
+ * While it is valid for PD_ALL_VISIBLE to be set when the
+ * corresponding VM bit is clear, we strongly prefer to keep them
+ * in sync.
+ *
+ * The heap buffer must be marked dirty before adding it to the
+ * WAL chain when setting the VM. We don't worry about
+ * unnecessarily dirtying the heap buffer if PD_ALL_VISIBLE is
+ * already set, though. It is extremely rare to have a clean heap
+ * buffer with PD_ALL_VISIBLE already set and the VM bits clear,
+ * so there is no point in optimizing it.
+ */
+ PageSetAllVisible(prstate.page);
+ PageClearPrunable(prstate.page);
+ visibilitymap_set_vmbits(prstate.block, prstate.vmbuffer, prstate.new_vmbits,
+ prstate.relation->rd_locator);
+ }
+
MarkBufferDirty(prstate.buffer);
/*
@@ -1173,29 +1263,12 @@ heap_page_prune_and_freeze(PruneFreezeParams *params,
*/
if (RelationNeedsWAL(prstate.relation))
{
- /*
- * The snapshotConflictHorizon for the whole record should be the
- * most conservative of all the horizons calculated for any of the
- * possible modifications. If this record will prune tuples, any
- * queries on the standby older than the newest xid of the most
- * recently removed tuple this record will prune will conflict. If
- * this record will freeze tuples, any queries on the standby with
- * xids older than the newest tuple this record will freeze will
- * conflict.
- */
- TransactionId conflict_xid;
-
- if (TransactionIdFollows(prstate.pagefrz.FreezePageConflictXid,
- prstate.latest_xid_removed))
- conflict_xid = prstate.pagefrz.FreezePageConflictXid;
- else
- conflict_xid = prstate.latest_xid_removed;
-
log_heap_prune_and_freeze(prstate.relation, prstate.buffer,
- InvalidBuffer, /* vmbuffer */
- 0, /* vmflags */
+ do_set_vm ? prstate.vmbuffer : InvalidBuffer,
+ do_set_vm ? prstate.new_vmbits : 0,
conflict_xid,
- true, params->reason,
+ true, /* cleanup lock */
+ params->reason,
prstate.frozen, prstate.nfrozen,
prstate.redirected, prstate.nredirected,
prstate.nowdead, prstate.ndead,
@@ -1205,33 +1278,71 @@ heap_page_prune_and_freeze(PruneFreezeParams *params,
END_CRIT_SECTION();
+ if (do_set_vm)
+ LockBuffer(prstate.vmbuffer, BUFFER_LOCK_UNLOCK);
+
+ /*
+ * During its second pass over the heap, VACUUM calls
+ * heap_page_would_be_all_visible() to determine whether a page is
+ * all-visible and all-frozen. The logic here is similar. After completing
+ * pruning and freezing, use an assertion to verify that our results
+ * remain consistent with heap_page_would_be_all_visible(). It's also a
+ * valuable cross-check of the page state after pruning and freezing.
+ */
+#ifdef USE_ASSERT_CHECKING
+ if (prstate.set_all_visible)
+ {
+ TransactionId debug_cutoff;
+ bool debug_all_frozen;
+
+ Assert(prstate.lpdead_items == 0);
+
+ Assert(heap_page_is_all_visible(prstate.relation, prstate.buffer,
+ prstate.vistest,
+ &debug_all_frozen,
+ &debug_cutoff, off_loc));
+
+ Assert(!TransactionIdIsValid(debug_cutoff) ||
+ debug_cutoff == prstate.newest_live_xid);
+
+ /*
+ * It's possible the page is composed entirely of frozen tuples but is
+ * not set all-frozen in the VM and did not pass
+ * HEAP_PAGE_PRUNE_FREEZE. In this case, it's possible
+ * heap_page_is_all_visible() finds the page completely frozen, even
+ * though prstate.set_all_frozen is false.
+ */
+ Assert(!prstate.set_all_frozen || debug_all_frozen);
+ }
+#endif
+
/* Copy information back for caller */
presult->ndeleted = prstate.ndeleted;
presult->nnewlpdead = prstate.ndead;
presult->nfrozen = prstate.nfrozen;
presult->live_tuples = prstate.live_tuples;
presult->recently_dead_tuples = prstate.recently_dead_tuples;
- presult->set_all_visible = prstate.set_all_visible;
- presult->set_all_frozen = prstate.set_all_frozen;
presult->hastup = prstate.hastup;
- presult->old_vmbits = prstate.old_vmbits;
-
- /*
- * For callers planning to update the visibility map, the conflict horizon
- * for that record must be the newest xmin on the page. However, if the
- * page is completely frozen, there can be no conflict and the
- * vm_conflict_horizon should remain InvalidTransactionId. This includes
- * the case that we just froze all the tuples; the prune-freeze record
- * included the conflict XID already so the caller doesn't need it.
- */
- if (presult->set_all_frozen)
- presult->vm_conflict_horizon = InvalidTransactionId;
- else
- presult->vm_conflict_horizon = prstate.newest_live_xid;
presult->lpdead_items = prstate.lpdead_items;
/* the presult->deadoffsets array was already filled in */
+ presult->newly_all_visible = false;
+ presult->newly_all_frozen = false;
+ presult->newly_all_visible_frozen = false;
+ if (do_set_vm)
+ {
+ if ((prstate.old_vmbits & VISIBILITYMAP_ALL_VISIBLE) == 0)
+ {
+ presult->newly_all_visible = true;
+ if (prstate.set_all_frozen)
+ presult->newly_all_visible_frozen = true;
+ }
+ else if ((prstate.old_vmbits & VISIBILITYMAP_ALL_FROZEN) == 0 &&
+ prstate.set_all_frozen)
+ presult->newly_all_frozen = true;
+ }
+
if (prstate.attempt_freeze)
{
if (presult->nfrozen > 0)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 696919e35dd..23deabd8c01 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -466,13 +466,6 @@ static void dead_items_add(LVRelState *vacrel, BlockNumber blkno, OffsetNumber *
static void dead_items_reset(LVRelState *vacrel);
static void dead_items_cleanup(LVRelState *vacrel);
-#ifdef USE_ASSERT_CHECKING
-static bool heap_page_is_all_visible(Relation rel, Buffer buf,
- GlobalVisState *vistest,
- bool *all_frozen,
- TransactionId *newest_live_xid,
- OffsetNumber *logging_offnum);
-#endif
static bool heap_page_would_be_all_visible(Relation rel, Buffer buf,
GlobalVisState *vistest,
bool allow_update_vistest,
@@ -2022,8 +2015,6 @@ lazy_scan_prune(LVRelState *vacrel,
.vistest = vacrel->vistest,
.cutoffs = &vacrel->cutoffs,
};
- uint8 old_vmbits = 0;
- uint8 new_vmbits = 0;
Assert(BufferGetBlockNumber(buf) == blkno);
@@ -2074,32 +2065,6 @@ lazy_scan_prune(LVRelState *vacrel,
vacrel->new_frozen_tuple_pages++;
}
- /*
- * VACUUM will call heap_page_is_all_visible() during the second pass over
- * the heap to determine all_visible and all_frozen for the page -- this
- * is a specialized version of the logic from this function. Now that
- * we've finished pruning and freezing, make sure that we're in total
- * agreement with heap_page_is_all_visible() using an assertion.
- */
-#ifdef USE_ASSERT_CHECKING
- if (presult.set_all_visible)
- {
- TransactionId debug_cutoff;
- bool debug_all_frozen;
-
- Assert(presult.lpdead_items == 0);
-
- Assert(heap_page_is_all_visible(vacrel->rel, buf,
- vacrel->vistest, &debug_all_frozen,
- &debug_cutoff, &vacrel->offnum));
-
- Assert(presult.set_all_frozen == debug_all_frozen);
-
- Assert(!TransactionIdIsValid(debug_cutoff) ||
- debug_cutoff == presult.vm_conflict_horizon);
- }
-#endif
-
/*
* Now save details of the LP_DEAD items from the page in vacrel
*/
@@ -2120,6 +2085,17 @@ lazy_scan_prune(LVRelState *vacrel,
}
/* Finally, add page-local counts to whole-VACUUM counts */
+ if (presult.newly_all_visible)
+ vacrel->new_all_visible_pages++;
+ if (presult.newly_all_visible_frozen)
+ vacrel->new_all_visible_all_frozen_pages++;
+ if (presult.newly_all_frozen)
+ vacrel->new_all_frozen_pages++;
+
+ /* Capture if the page was newly set frozen */
+ *vm_page_frozen = presult.newly_all_visible_frozen ||
+ presult.newly_all_frozen;
+
vacrel->tuples_deleted += presult.ndeleted;
vacrel->tuples_frozen += presult.nfrozen;
vacrel->lpdead_items += presult.lpdead_items;
@@ -2133,71 +2109,6 @@ lazy_scan_prune(LVRelState *vacrel,
/* Did we find LP_DEAD items? */
*has_lpdead_items = (presult.lpdead_items > 0);
- Assert(!presult.set_all_visible || !(*has_lpdead_items));
- Assert(!presult.set_all_frozen || presult.set_all_visible);
-
- if (!presult.set_all_visible)
- return presult.ndeleted;
-
- /* Set the visibility map and page visibility hint */
- old_vmbits = presult.old_vmbits;
- new_vmbits = VISIBILITYMAP_ALL_VISIBLE;
- if (presult.set_all_frozen)
- new_vmbits |= VISIBILITYMAP_ALL_FROZEN;
-
- /* Nothing to do */
- if (old_vmbits == new_vmbits)
- return presult.ndeleted;
-
- /*
- * It should never be the case that the visibility map page is set while
- * the page-level bit is clear (and if so, we cleared it above), but the
- * reverse is allowed (if checksums are not enabled). Regardless, set both
- * bits so that we get back in sync.
- *
- * The heap buffer must be marked dirty before adding it to the WAL chain
- * when setting the VM. We don't worry about unnecessarily dirtying the
- * heap buffer if PD_ALL_VISIBLE is already set, though. It is extremely
- * rare to have a clean heap buffer with PD_ALL_VISIBLE already set and
- * the VM bits clear, so there is no point in optimizing it.
- */
- PageSetAllVisible(page);
- PageClearPrunable(page);
- MarkBufferDirty(buf);
-
- /*
- * If the page is being set all-frozen, we pass InvalidTransactionId as
- * the cutoff_xid, since a snapshot conflict horizon sufficient to make
- * everything safe for REDO was logged when the page's tuples were frozen.
- */
- Assert(!presult.set_all_frozen ||
- !TransactionIdIsValid(presult.vm_conflict_horizon));
-
- visibilitymap_set(vacrel->rel, blkno, buf,
- InvalidXLogRecPtr,
- vmbuffer, presult.vm_conflict_horizon,
- new_vmbits);
-
- /*
- * If the page wasn't already set all-visible and/or all-frozen in the VM,
- * count it as newly set for logging.
- */
- if ((old_vmbits & VISIBILITYMAP_ALL_VISIBLE) == 0)
- {
- vacrel->new_all_visible_pages++;
- if (presult.set_all_frozen)
- {
- vacrel->new_all_visible_all_frozen_pages++;
- *vm_page_frozen = true;
- }
- }
- else if ((old_vmbits & VISIBILITYMAP_ALL_FROZEN) == 0 &&
- presult.set_all_frozen)
- {
- vacrel->new_all_frozen_pages++;
- *vm_page_frozen = true;
- }
-
return presult.ndeleted;
}
@@ -3613,7 +3524,7 @@ dead_items_cleanup(LVRelState *vacrel)
* that expect no LP_DEAD on the page. Currently assert-only, but there is no
* reason not to use it outside of asserts.
*/
-static bool
+bool
heap_page_is_all_visible(Relation rel, Buffer buf,
GlobalVisState *vistest,
bool *all_frozen,
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index ca5e8d1794f..0ab322bf58b 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -265,7 +265,8 @@ typedef struct PruneFreezeParams
/*
* Callers should provide a pinned vmbuffer corresponding to the heap
- * block in buffer. We will check for and repair any corruption in the VM.
+ * block in buffer. We will check for and repair any corruption in the VM
+ * and set the VM after pruning if the page is all-visible/all-frozen.
*/
Buffer vmbuffer;
@@ -281,8 +282,7 @@ typedef struct PruneFreezeParams
* HEAP_PAGE_PRUNE_MARK_UNUSED_NOW indicates that dead items can be set
* LP_UNUSED during pruning.
*
- * HEAP_PAGE_PRUNE_FREEZE indicates that we will also freeze tuples, and
- * will return 'all_visible', 'all_frozen' flags to the caller.
+ * HEAP_PAGE_PRUNE_FREEZE indicates that we will also freeze tuples.
*/
int options;
@@ -316,26 +316,12 @@ typedef struct PruneFreezeResult
int recently_dead_tuples;
/*
- * set_all_visible and set_all_frozen indicate if the all-visible and
- * all-frozen bits in the visibility map should be set for this page after
- * pruning.
- *
- * vm_conflict_horizon is the newest xmin of live tuples on the page. The
- * caller can use it as the conflict horizon when setting the VM bits. It
- * is only valid if we froze some tuples (nfrozen > 0), and set_all_frozen
- * is true.
- *
- * These are only set if the HEAP_PAGE_PRUNE_FREEZE option is set.
- */
- bool set_all_visible;
- bool set_all_frozen;
- TransactionId vm_conflict_horizon;
-
- /*
- * The value of the vmbuffer's vmbits at the beginning of pruning. It is
- * cleared if VM corruption is found and corrected.
+ * Whether or not the page was newly set all-visible and all-frozen during
+ * phase I of vacuuming.
*/
- uint8 old_vmbits;
+ bool newly_all_visible;
+ bool newly_all_visible_frozen;
+ bool newly_all_frozen;
/*
* Whether or not the page makes rel truncation unsafe. This is set to
@@ -472,6 +458,13 @@ extern void log_heap_prune_and_freeze(Relation relation, Buffer buffer,
/* in heap/vacuumlazy.c */
extern void heap_vacuum_rel(Relation rel,
const VacuumParams params, BufferAccessStrategy bstrategy);
+#ifdef USE_ASSERT_CHECKING
+extern bool heap_page_is_all_visible(Relation rel, Buffer buf,
+ GlobalVisState *vistest,
+ bool *all_frozen,
+ TransactionId *newest_live_xid,
+ OffsetNumber *logging_offnum);
+#endif
/* in heap/heapam_visibility.c */
extern bool HeapTupleSatisfiesVisibility(HeapTuple htup, Snapshot snapshot,
--
2.43.0
[text/x-patch] v44-0004-WAL-log-VM-setting-for-empty-pages-in-XLOG_HEAP2.patch (5.7K, 5-v44-0004-WAL-log-VM-setting-for-empty-pages-in-XLOG_HEAP2.patch)
download | inline diff:
From 134e19504883dc3b07c506332dd23533e381b699 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Sat, 27 Sep 2025 11:55:21 -0400
Subject: [PATCH v44 04/10] WAL log VM setting for empty pages in
XLOG_HEAP2_PRUNE_VACUUM_SCAN
As part of removing XLOG_HEAP2_VISIBLE records, phase I of VACUUM now
marks empty pages all-visible in a XLOG_HEAP2_PRUNE_VACUUM_SCAN record.
This has no independent benefit, but empty pages were the last user of
XLOG_HEAP2_VISIBLE, so by making this change we can next remove all of
the XLOG_HEAP2_VISIBLE code.
Author: Melanie Plageman <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Chao Li <[email protected]>
Earlier version Reviewed-by: Robert Haas <[email protected]>
---
src/backend/access/heap/pruneheap.c | 29 +++++++++++-------
src/backend/access/heap/vacuumlazy.c | 44 +++++++++++++++++-----------
2 files changed, 45 insertions(+), 28 deletions(-)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index 4d6d5e92773..fe9564b26c7 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -2541,6 +2541,8 @@ log_heap_prune_and_freeze(Relation relation, Buffer buffer,
uint8 info;
uint8 regbuf_flags_heap;
+ Page heap_page = BufferGetPage(buffer);
+
/* The following local variables hold data registered in the WAL record: */
xlhp_freeze_plan plans[MaxHeapTuplesPerPage];
xlhp_freeze_plans freeze_plans;
@@ -2559,14 +2561,18 @@ log_heap_prune_and_freeze(Relation relation, Buffer buffer,
/*
* We can avoid an FPI of the heap page if the only modification we are
* making to it is to set PD_ALL_VISIBLE and checksums/wal_log_hints are
- * disabled. Note that if we explicitly skip an FPI, we must not stamp the
- * heap page with this record's LSN. Recovery skips records <= the stamped
- * LSN, so this could lead to skipping an earlier FPI needed to repair a
- * torn page.
+ * disabled.
+ *
+ * However, if the page has never been WAL-logged (LSN is invalid), we
+ * must force an FPI regardless. This can happen when another backend
+ * extends the heap, initializes the page, and then fails before WAL-
+ * logging it. Since heap extension is not WAL-logged, recovery might try
+ * to replay our record and find that the page isn't initialized, which
+ * would cause a PANIC.
*/
- if (!do_prune &&
- nfrozen == 0 &&
- (!do_set_vm || !XLogHintBitIsNeeded()))
+ if (!XLogRecPtrIsValid(PageGetLSN(heap_page)))
+ regbuf_flags_heap |= REGBUF_FORCE_IMAGE;
+ else if (!do_prune && nfrozen == 0 && (!do_set_vm || !XLogHintBitIsNeeded()))
regbuf_flags_heap |= REGBUF_NO_IMAGE;
/*
@@ -2681,12 +2687,13 @@ log_heap_prune_and_freeze(Relation relation, Buffer buffer,
}
/*
- * See comment at the top of the function about regbuf_flags_heap for
- * details on when we can advance the page LSN.
+ * If we explicitly skip an FPI, we must not stamp the heap page with this
+ * record's LSN. Recovery skips records <= the stamped LSN, so this could
+ * lead to skipping an earlier FPI needed to repair a torn page.
*/
- if (do_prune || nfrozen > 0 || (do_set_vm && XLogHintBitIsNeeded()))
+ if (!(regbuf_flags_heap & REGBUF_NO_IMAGE))
{
Assert(BufferIsDirty(buffer));
- PageSetLSN(BufferGetPage(buffer), recptr);
+ PageSetLSN(heap_page, recptr);
}
}
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 23deabd8c01..63e6199241c 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -1929,33 +1929,43 @@ lazy_scan_new_or_empty(LVRelState *vacrel, Buffer buf, BlockNumber blkno,
*/
if (!PageIsAllVisible(page))
{
+ /* Lock vmbuffer before entering critical section */
+ LockBuffer(vmbuffer, BUFFER_LOCK_EXCLUSIVE);
+
START_CRIT_SECTION();
/* mark buffer dirty before writing a WAL record */
MarkBufferDirty(buf);
+ PageSetAllVisible(page);
+ PageClearPrunable(page);
+ visibilitymap_set_vmbits(blkno,
+ vmbuffer,
+ VISIBILITYMAP_ALL_VISIBLE |
+ VISIBILITYMAP_ALL_FROZEN,
+ vacrel->rel->rd_locator);
+
/*
- * It's possible that another backend has extended the heap,
- * initialized the page, and then failed to WAL-log the page due
- * to an ERROR. Since heap extension is not WAL-logged, recovery
- * might try to replay our record setting the page all-visible and
- * find that the page isn't initialized, which will cause a PANIC.
- * To prevent that, check whether the page has been previously
- * WAL-logged, and if not, do that now.
+ * Emit WAL for setting PD_ALL_VISIBLE on the heap page and
+ * setting the VM.
*/
- if (RelationNeedsWAL(vacrel->rel) &&
- !XLogRecPtrIsValid(PageGetLSN(page)))
- log_newpage_buffer(buf, true);
+ if (RelationNeedsWAL(vacrel->rel))
+ log_heap_prune_and_freeze(vacrel->rel, buf,
+ vmbuffer,
+ VISIBILITYMAP_ALL_VISIBLE |
+ VISIBILITYMAP_ALL_FROZEN,
+ InvalidTransactionId, /* conflict xid */
+ false, /* cleanup lock */
+ PRUNE_VACUUM_SCAN, /* reason */
+ NULL, 0,
+ NULL, 0,
+ NULL, 0,
+ NULL, 0);
- PageSetAllVisible(page);
- PageClearPrunable(page);
- visibilitymap_set(vacrel->rel, blkno, buf,
- InvalidXLogRecPtr,
- vmbuffer, InvalidTransactionId,
- VISIBILITYMAP_ALL_VISIBLE |
- VISIBILITYMAP_ALL_FROZEN);
END_CRIT_SECTION();
+ LockBuffer(vmbuffer, BUFFER_LOCK_UNLOCK);
+
/* Count the newly all-frozen pages for logging */
vacrel->new_all_visible_pages++;
vacrel->new_all_visible_all_frozen_pages++;
--
2.43.0
[text/x-patch] v44-0005-Remove-XLOG_HEAP2_VISIBLE-entirely.patch (27.6K, 6-v44-0005-Remove-XLOG_HEAP2_VISIBLE-entirely.patch)
download | inline diff:
From 42b15a654c87c36aadd1768f3f2fc915019ee44c Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Sat, 27 Sep 2025 11:55:36 -0400
Subject: [PATCH v44 05/10] Remove XLOG_HEAP2_VISIBLE entirely
There are no remaining users that emit XLOG_HEAP2_VISIBLE records, so it
can be removed. This includes deleting the xl_heap_visible struct and
all functions responsible for emitting or replaying XLOG_HEAP2_VISIBLE
records.
Author: Melanie Plageman <[email protected]>
Reviewed-by: Andrey Borodin <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Chao Li <[email protected]>
---
src/backend/access/common/bufmask.c | 5 +-
src/backend/access/heap/heapam.c | 54 +-------
src/backend/access/heap/heapam_xlog.c | 156 ++---------------------
src/backend/access/heap/pruneheap.c | 4 +-
src/backend/access/heap/vacuumlazy.c | 16 +--
src/backend/access/heap/visibilitymap.c | 151 +++++-----------------
src/backend/access/rmgrdesc/heapdesc.c | 10 --
src/backend/replication/logical/decode.c | 1 -
src/backend/storage/ipc/standby.c | 9 +-
src/include/access/heapam_xlog.h | 21 +--
src/include/access/visibilitymap.h | 13 +-
src/include/access/visibilitymapdefs.h | 9 --
src/tools/pgindent/typedefs.list | 1 -
13 files changed, 64 insertions(+), 386 deletions(-)
diff --git a/src/backend/access/common/bufmask.c b/src/backend/access/common/bufmask.c
index 8a67bfa1aff..f32e3911a57 100644
--- a/src/backend/access/common/bufmask.c
+++ b/src/backend/access/common/bufmask.c
@@ -55,9 +55,8 @@ mask_page_hint_bits(Page page)
PageClearHasFreeLinePointers(page);
/*
- * During replay, if the page LSN has advanced past our XLOG record's LSN,
- * we don't mark the page all-visible. See heap_xlog_visible() for
- * details.
+ * PD_ALL_VISIBLE is masked during WAL consistency checking. It is worth
+ * investigating if we could stop doing this.
*/
PageClearAllVisible(page);
}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index e5bd062de77..044f385e477 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -2589,11 +2589,11 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
{
PageSetAllVisible(page);
PageClearPrunable(page);
- visibilitymap_set_vmbits(BufferGetBlockNumber(buffer),
- vmbuffer,
- VISIBILITYMAP_ALL_VISIBLE |
- VISIBILITYMAP_ALL_FROZEN,
- relation->rd_locator);
+ visibilitymap_set(BufferGetBlockNumber(buffer),
+ vmbuffer,
+ VISIBILITYMAP_ALL_VISIBLE |
+ VISIBILITYMAP_ALL_FROZEN,
+ relation->rd_locator);
}
/*
@@ -8886,50 +8886,6 @@ bottomup_sort_and_shrink(TM_IndexDeleteOp *delstate)
return nblocksfavorable;
}
-/*
- * Perform XLogInsert for a heap-visible operation. 'block' is the block
- * being marked all-visible, and vm_buffer is the buffer containing the
- * corresponding visibility map block. Both should have already been modified
- * and dirtied.
- *
- * snapshotConflictHorizon comes from the largest xmin on the page being
- * marked all-visible. REDO routine uses it to generate recovery conflicts.
- *
- * If checksums or wal_log_hints are enabled, we may also generate a full-page
- * image of heap_buffer. Otherwise, we optimize away the FPI (by specifying
- * REGBUF_NO_IMAGE for the heap buffer), in which case the caller should *not*
- * update the heap page's LSN.
- */
-XLogRecPtr
-log_heap_visible(Relation rel, Buffer heap_buffer, Buffer vm_buffer,
- TransactionId snapshotConflictHorizon, uint8 vmflags)
-{
- xl_heap_visible xlrec;
- XLogRecPtr recptr;
- uint8 flags;
-
- Assert(BufferIsValid(heap_buffer));
- Assert(BufferIsValid(vm_buffer));
-
- xlrec.snapshotConflictHorizon = snapshotConflictHorizon;
- xlrec.flags = vmflags;
- if (RelationIsAccessibleInLogicalDecoding(rel))
- xlrec.flags |= VISIBILITYMAP_XLOG_CATALOG_REL;
- XLogBeginInsert();
- XLogRegisterData(&xlrec, SizeOfHeapVisible);
-
- XLogRegisterBuffer(0, vm_buffer, 0);
-
- flags = REGBUF_STANDARD;
- if (!XLogHintBitIsNeeded())
- flags |= REGBUF_NO_IMAGE;
- XLogRegisterBuffer(1, heap_buffer, flags);
-
- recptr = XLogInsert(RM_HEAP2_ID, XLOG_HEAP2_VISIBLE);
-
- return recptr;
-}
-
/*
* Perform XLogInsert for a heap-update operation. Caller must already
* have modified the buffer(s) and marked them dirty.
diff --git a/src/backend/access/heap/heapam_xlog.c b/src/backend/access/heap/heapam_xlog.c
index 1da774c1536..1302bb13e18 100644
--- a/src/backend/access/heap/heapam_xlog.c
+++ b/src/backend/access/heap/heapam_xlog.c
@@ -239,7 +239,7 @@ heap_xlog_prune_freeze(XLogReaderState *record)
if (PageIsNew(vmpage))
PageInit(vmpage, BLCKSZ, 0);
- visibilitymap_set_vmbits(blkno, vmbuffer, vmflags, rlocator);
+ visibilitymap_set(blkno, vmbuffer, vmflags, rlocator);
Assert(BufferIsDirty(vmbuffer));
PageSetLSN(vmpage, lsn);
@@ -252,143 +252,6 @@ heap_xlog_prune_freeze(XLogReaderState *record)
XLogRecordPageWithFreeSpace(rlocator, blkno, freespace);
}
-/*
- * Replay XLOG_HEAP2_VISIBLE records.
- *
- * The critical integrity requirement here is that we must never end up with
- * a situation where the visibility map bit is set, and the page-level
- * PD_ALL_VISIBLE bit is clear. If that were to occur, then a subsequent
- * page modification would fail to clear the visibility map bit.
- */
-static void
-heap_xlog_visible(XLogReaderState *record)
-{
- XLogRecPtr lsn = record->EndRecPtr;
- xl_heap_visible *xlrec = (xl_heap_visible *) XLogRecGetData(record);
- Buffer vmbuffer = InvalidBuffer;
- Buffer buffer;
- Page page;
- RelFileLocator rlocator;
- BlockNumber blkno;
- XLogRedoAction action;
-
- Assert((xlrec->flags & VISIBILITYMAP_XLOG_VALID_BITS) == xlrec->flags);
-
- XLogRecGetBlockTag(record, 1, &rlocator, NULL, &blkno);
-
- /*
- * If there are any Hot Standby transactions running that have an xmin
- * horizon old enough that this page isn't all-visible for them, they
- * might incorrectly decide that an index-only scan can skip a heap fetch.
- *
- * NB: It might be better to throw some kind of "soft" conflict here that
- * forces any index-only scan that is in flight to perform heap fetches,
- * rather than killing the transaction outright.
- */
- if (InHotStandby)
- ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon,
- xlrec->flags & VISIBILITYMAP_XLOG_CATALOG_REL,
- rlocator);
-
- /*
- * Read the heap page, if it still exists. If the heap file has dropped or
- * truncated later in recovery, we don't need to update the page, but we'd
- * better still update the visibility map.
- */
- action = XLogReadBufferForRedo(record, 1, &buffer);
- if (action == BLK_NEEDS_REDO)
- {
- /*
- * We don't bump the LSN of the heap page when setting the visibility
- * map bit (unless checksums or wal_hint_bits is enabled, in which
- * case we must). This exposes us to torn page hazards, but since
- * we're not inspecting the existing page contents in any way, we
- * don't care.
- */
- page = BufferGetPage(buffer);
-
- PageSetAllVisible(page);
- PageClearPrunable(page);
-
- if (XLogHintBitIsNeeded())
- PageSetLSN(page, lsn);
-
- MarkBufferDirty(buffer);
- }
- else if (action == BLK_RESTORED)
- {
- /*
- * If heap block was backed up, we already restored it and there's
- * nothing more to do. (This can only happen with checksums or
- * wal_log_hints enabled.)
- */
- }
-
- if (BufferIsValid(buffer))
- {
- Size space = PageGetFreeSpace(BufferGetPage(buffer));
-
- UnlockReleaseBuffer(buffer);
-
- /*
- * Since FSM is not WAL-logged and only updated heuristically, it
- * easily becomes stale in standbys. If the standby is later promoted
- * and runs VACUUM, it will skip updating individual free space
- * figures for pages that became all-visible (or all-frozen, depending
- * on the vacuum mode,) which is troublesome when FreeSpaceMapVacuum
- * propagates too optimistic free space values to upper FSM layers;
- * later inserters try to use such pages only to find out that they
- * are unusable. This can cause long stalls when there are many such
- * pages.
- *
- * Forestall those problems by updating FSM's idea about a page that
- * is becoming all-visible or all-frozen.
- *
- * Do this regardless of a full-page image being applied, since the
- * FSM data is not in the page anyway.
- */
- if (xlrec->flags & VISIBILITYMAP_VALID_BITS)
- XLogRecordPageWithFreeSpace(rlocator, blkno, space);
- }
-
- /*
- * Even if we skipped the heap page update due to the LSN interlock, it's
- * still safe to update the visibility map. Any WAL record that clears
- * the visibility map bit does so before checking the page LSN, so any
- * bits that need to be cleared will still be cleared.
- */
- if (XLogReadBufferForRedoExtended(record, 0, RBM_ZERO_ON_ERROR, false,
- &vmbuffer) == BLK_NEEDS_REDO)
- {
- Page vmpage = BufferGetPage(vmbuffer);
- Relation reln;
- uint8 vmbits;
-
- /* initialize the page if it was read as zeros */
- if (PageIsNew(vmpage))
- PageInit(vmpage, BLCKSZ, 0);
-
- /* remove VISIBILITYMAP_XLOG_* */
- vmbits = xlrec->flags & VISIBILITYMAP_VALID_BITS;
-
- /*
- * XLogReadBufferForRedoExtended locked the buffer. But
- * visibilitymap_set will handle locking itself.
- */
- LockBuffer(vmbuffer, BUFFER_LOCK_UNLOCK);
-
- reln = CreateFakeRelcacheEntry(rlocator);
-
- visibilitymap_set(reln, blkno, InvalidBuffer, lsn, vmbuffer,
- xlrec->snapshotConflictHorizon, vmbits);
-
- ReleaseBuffer(vmbuffer);
- FreeFakeRelcacheEntry(reln);
- }
- else if (BufferIsValid(vmbuffer))
- UnlockReleaseBuffer(vmbuffer);
-}
-
/*
* Given an "infobits" field from an XLog record, set the correct bits in the
* given infomask and infomask2 for the tuple touched by the record.
@@ -769,8 +632,8 @@ heap_xlog_multi_insert(XLogReaderState *record)
*
* During recovery, however, no concurrent writers exist. Therefore,
* updating the VM without holding the heap page lock is safe enough. This
- * same approach is taken when replaying xl_heap_visible records (see
- * heap_xlog_visible()).
+ * same approach is taken when replaying XLOG_HEAP2_PRUNE* records (see
+ * heap_xlog_prune_freeze()).
*/
if ((xlrec->flags & XLH_INSERT_ALL_FROZEN_SET) &&
XLogReadBufferForRedoExtended(record, 1, RBM_ZERO_ON_ERROR, false,
@@ -782,11 +645,11 @@ heap_xlog_multi_insert(XLogReaderState *record)
if (PageIsNew(vmpage))
PageInit(vmpage, BLCKSZ, 0);
- visibilitymap_set_vmbits(blkno,
- vmbuffer,
- VISIBILITYMAP_ALL_VISIBLE |
- VISIBILITYMAP_ALL_FROZEN,
- rlocator);
+ visibilitymap_set(blkno,
+ vmbuffer,
+ VISIBILITYMAP_ALL_VISIBLE |
+ VISIBILITYMAP_ALL_FROZEN,
+ rlocator);
Assert(BufferIsDirty(vmbuffer));
PageSetLSN(vmpage, lsn);
@@ -1369,9 +1232,6 @@ heap2_redo(XLogReaderState *record)
case XLOG_HEAP2_PRUNE_VACUUM_CLEANUP:
heap_xlog_prune_freeze(record);
break;
- case XLOG_HEAP2_VISIBLE:
- heap_xlog_visible(record);
- break;
case XLOG_HEAP2_MULTI_INSERT:
heap_xlog_multi_insert(record);
break;
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index fe9564b26c7..fc5345e1dff 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -1252,8 +1252,8 @@ heap_page_prune_and_freeze(PruneFreezeParams *params,
*/
PageSetAllVisible(prstate.page);
PageClearPrunable(prstate.page);
- visibilitymap_set_vmbits(prstate.block, prstate.vmbuffer, prstate.new_vmbits,
- prstate.relation->rd_locator);
+ visibilitymap_set(prstate.block, prstate.vmbuffer, prstate.new_vmbits,
+ prstate.relation->rd_locator);
}
MarkBufferDirty(prstate.buffer);
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 63e6199241c..f698c2d899b 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -1939,11 +1939,11 @@ lazy_scan_new_or_empty(LVRelState *vacrel, Buffer buf, BlockNumber blkno,
PageSetAllVisible(page);
PageClearPrunable(page);
- visibilitymap_set_vmbits(blkno,
- vmbuffer,
- VISIBILITYMAP_ALL_VISIBLE |
- VISIBILITYMAP_ALL_FROZEN,
- vacrel->rel->rd_locator);
+ visibilitymap_set(blkno,
+ vmbuffer,
+ VISIBILITYMAP_ALL_VISIBLE |
+ VISIBILITYMAP_ALL_FROZEN,
+ vacrel->rel->rd_locator);
/*
* Emit WAL for setting PD_ALL_VISIBLE on the heap page and
@@ -2821,9 +2821,9 @@ lazy_vacuum_heap_page(LVRelState *vacrel, BlockNumber blkno, Buffer buffer,
*/
PageSetAllVisible(page);
PageClearPrunable(page);
- visibilitymap_set_vmbits(blkno,
- vmbuffer, vmflags,
- vacrel->rel->rd_locator);
+ visibilitymap_set(blkno,
+ vmbuffer, vmflags,
+ vacrel->rel->rd_locator);
conflict_xid = newest_live_xid;
}
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index e21b96281a6..4fd470702aa 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -14,8 +14,7 @@
* visibilitymap_clear - clear bits for one page in the visibility map
* visibilitymap_pin - pin a map page for setting a bit
* visibilitymap_pin_ok - check whether correct map page is already pinned
- * visibilitymap_set - set bit(s) in a previously pinned page and log
- * visibilitymap_set_vmbits - set bit(s) in a pinned page
+ * visibilitymap_set - set bit(s) in a previously pinned page
* visibilitymap_get_status - get status of bits
* visibilitymap_count - count number of bits set in visibility map
* visibilitymap_prepare_truncate -
@@ -35,21 +34,32 @@
* is set, we know the condition is true, but if a bit is not set, it might or
* might not be true.
*
- * Clearing visibility map bits is not separately WAL-logged. The callers
- * must make sure that whenever a bit is cleared, the bit is cleared on WAL
- * replay of the updating operation as well.
- *
- * When we *set* a visibility map during VACUUM, we must write WAL. This may
- * seem counterintuitive, since the bit is basically a hint: if it is clear,
- * it may still be the case that every tuple on the page is visible to all
- * transactions; we just don't know that for certain. The difficulty is that
- * there are two bits which are typically set together: the PD_ALL_VISIBLE bit
- * on the page itself, and the visibility map bit. If a crash occurs after the
- * visibility map page makes it to disk and before the updated heap page makes
- * it to disk, redo must set the bit on the heap page. Otherwise, the next
- * insert, update, or delete on the heap page will fail to realize that the
- * visibility map bit must be cleared, possibly causing index-only scans to
- * return wrong answers.
+ * Changes to the visibility map bits are not separately WAL-logged. Callers
+ * must make sure that whenever a visibility map bit is cleared, the bit is
+ * cleared on WAL replay of the updating operation. And whenever a visibility
+ * map bit is set, the bit is set on WAL replay of the operation that rendered
+ * the page all-visible/all-frozen.
+ *
+ * The visibility map bits operate as a hint in one direction: if they are
+ * clear, it may still be the case that every tuple on the page is visible to
+ * all transactions (we just don't know that for certain). However, if they
+ * are set, we may skip vacuuming pages and advance relfrozenxid or skip
+ * reading heap pages for an index-only scan. If they are incorrectly set,
+ * this can lead to data corruption and wrong results.
+ *
+ * Additionally, it is critical that the heap-page level PD_ALL_VISIBLE bit be
+ * correctly set and cleared along with the VM bits.
+ *
+ * When clearing the VM, if a crash occurs after the heap page makes it to
+ * disk but before the VM page makes it to disk, replay must clear the VM or
+ * the next index-only scan can return wrong results or vacuum may incorrectly
+ * advance relfrozenxid.
+ *
+ * When setting the VM, if a crash occurs after the visibility map page makes
+ * it to disk and before the updated heap page makes it to disk, redo must set
+ * the bit on the heap page. Otherwise, the next insert, update, or delete on
+ * the heap page will fail to realize that the visibility map bit must be
+ * cleared, possibly causing index-only scans to return wrong answers.
*
* VACUUM will normally skip pages for which the visibility map bit is set;
* such pages can't contain any dead tuples and therefore don't need vacuuming.
@@ -222,112 +232,11 @@ visibilitymap_pin_ok(BlockNumber heapBlk, Buffer vmbuf)
return BufferIsValid(vmbuf) && BufferGetBlockNumber(vmbuf) == mapBlock;
}
-/*
- * visibilitymap_set - set bit(s) on a previously pinned page
- *
- * recptr is the LSN of the XLOG record we're replaying, if we're in recovery,
- * or InvalidXLogRecPtr in normal running. The VM page LSN is advanced to the
- * one provided; in normal running, we generate a new XLOG record and set the
- * page LSN to that value (though the heap page's LSN may *not* be updated;
- * see below). cutoff_xid is the largest xmin on the page being marked
- * all-visible; it is needed for Hot Standby, and can be InvalidTransactionId
- * if the page contains no tuples. It can also be set to InvalidTransactionId
- * when a page that is already all-visible is being marked all-frozen.
- *
- * Caller is expected to set the heap page's PD_ALL_VISIBLE bit before calling
- * this function. Except in recovery, caller should also pass the heap
- * buffer. When checksums are enabled and we're not in recovery, we must add
- * the heap buffer to the WAL chain to protect it from being torn.
- *
- * You must pass a buffer containing the correct map page to this function.
- * Call visibilitymap_pin first to pin the right one. This function doesn't do
- * any I/O.
- */
-void
-visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
- XLogRecPtr recptr, Buffer vmBuf, TransactionId cutoff_xid,
- uint8 flags)
-{
- BlockNumber mapBlock = HEAPBLK_TO_MAPBLOCK(heapBlk);
- uint32 mapByte = HEAPBLK_TO_MAPBYTE(heapBlk);
- uint8 mapOffset = HEAPBLK_TO_OFFSET(heapBlk);
- Page page;
- uint8 *map;
- uint8 status;
-
-#ifdef TRACE_VISIBILITYMAP
- elog(DEBUG1, "vm_set flags 0x%02X for %s %d",
- flags, RelationGetRelationName(rel), heapBlk);
-#endif
-
- Assert(InRecovery || !XLogRecPtrIsValid(recptr));
- Assert(InRecovery || PageIsAllVisible(BufferGetPage(heapBuf)));
- Assert((flags & VISIBILITYMAP_VALID_BITS) == flags);
-
- /* Must never set all_frozen bit without also setting all_visible bit */
- Assert(flags != VISIBILITYMAP_ALL_FROZEN);
-
- /* Check that we have the right heap page pinned, if present */
- if (BufferIsValid(heapBuf) && BufferGetBlockNumber(heapBuf) != heapBlk)
- elog(ERROR, "wrong heap buffer passed to visibilitymap_set");
-
- Assert(!BufferIsValid(heapBuf) ||
- BufferIsLockedByMeInMode(heapBuf, BUFFER_LOCK_EXCLUSIVE));
-
- /* Check that we have the right VM page pinned */
- if (!BufferIsValid(vmBuf) || BufferGetBlockNumber(vmBuf) != mapBlock)
- elog(ERROR, "wrong VM buffer passed to visibilitymap_set");
-
- page = BufferGetPage(vmBuf);
- map = (uint8 *) PageGetContents(page);
- LockBuffer(vmBuf, BUFFER_LOCK_EXCLUSIVE);
-
- status = (map[mapByte] >> mapOffset) & VISIBILITYMAP_VALID_BITS;
- if (flags != status)
- {
- START_CRIT_SECTION();
-
- map[mapByte] |= (flags << mapOffset);
- MarkBufferDirty(vmBuf);
-
- if (RelationNeedsWAL(rel))
- {
- if (!XLogRecPtrIsValid(recptr))
- {
- Assert(!InRecovery);
- recptr = log_heap_visible(rel, heapBuf, vmBuf, cutoff_xid, flags);
-
- /*
- * If data checksums are enabled (or wal_log_hints=on), we
- * need to protect the heap page from being torn.
- *
- * If not, then we must *not* update the heap page's LSN. In
- * this case, the FPI for the heap page was omitted from the
- * WAL record inserted above, so it would be incorrect to
- * update the heap page's LSN.
- */
- if (XLogHintBitIsNeeded())
- {
- Page heapPage = BufferGetPage(heapBuf);
-
- PageSetLSN(heapPage, recptr);
- }
- }
- PageSetLSN(page, recptr);
- }
-
- END_CRIT_SECTION();
- }
-
- LockBuffer(vmBuf, BUFFER_LOCK_UNLOCK);
-}
-
/*
* Set VM (visibility map) flags in the VM block in vmBuf.
*
* This function is intended for callers that log VM changes together
* with the heap page modifications that rendered the page all-visible.
- * Callers that log VM changes separately should use visibilitymap_set().
*
* vmBuf must be pinned and exclusively locked, and it must cover the VM bits
* corresponding to heapBlk.
@@ -343,9 +252,9 @@ visibilitymap_set(Relation rel, BlockNumber heapBlk, Buffer heapBuf,
* rlocator is used only for debugging messages.
*/
void
-visibilitymap_set_vmbits(BlockNumber heapBlk,
- Buffer vmBuf, uint8 flags,
- const RelFileLocator rlocator)
+visibilitymap_set(BlockNumber heapBlk,
+ Buffer vmBuf, uint8 flags,
+ const RelFileLocator rlocator)
{
BlockNumber mapBlock = HEAPBLK_TO_MAPBLOCK(heapBlk);
uint32 mapByte = HEAPBLK_TO_MAPBYTE(heapBlk);
diff --git a/src/backend/access/rmgrdesc/heapdesc.c b/src/backend/access/rmgrdesc/heapdesc.c
index 02ae91653c1..75ae6f9d375 100644
--- a/src/backend/access/rmgrdesc/heapdesc.c
+++ b/src/backend/access/rmgrdesc/heapdesc.c
@@ -349,13 +349,6 @@ heap2_desc(StringInfo buf, XLogReaderState *record)
}
}
}
- else if (info == XLOG_HEAP2_VISIBLE)
- {
- xl_heap_visible *xlrec = (xl_heap_visible *) rec;
-
- appendStringInfo(buf, "snapshotConflictHorizon: %u, flags: 0x%02X",
- xlrec->snapshotConflictHorizon, xlrec->flags);
- }
else if (info == XLOG_HEAP2_MULTI_INSERT)
{
xl_heap_multi_insert *xlrec = (xl_heap_multi_insert *) rec;
@@ -461,9 +454,6 @@ heap2_identify(uint8 info)
case XLOG_HEAP2_PRUNE_VACUUM_CLEANUP:
id = "PRUNE_VACUUM_CLEANUP";
break;
- case XLOG_HEAP2_VISIBLE:
- id = "VISIBLE";
- break;
case XLOG_HEAP2_MULTI_INSERT:
id = "MULTI_INSERT";
break;
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index 21f03864a66..3c027bcb2f7 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -448,7 +448,6 @@ heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
case XLOG_HEAP2_PRUNE_ON_ACCESS:
case XLOG_HEAP2_PRUNE_VACUUM_SCAN:
case XLOG_HEAP2_PRUNE_VACUUM_CLEANUP:
- case XLOG_HEAP2_VISIBLE:
case XLOG_HEAP2_LOCK_UPDATED:
break;
default:
diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c
index f3ad90c7c7a..de9092fdf5b 100644
--- a/src/backend/storage/ipc/standby.c
+++ b/src/backend/storage/ipc/standby.c
@@ -476,10 +476,11 @@ ResolveRecoveryConflictWithSnapshot(TransactionId snapshotConflictHorizon,
/*
* If we get passed InvalidTransactionId then we do nothing (no conflict).
*
- * This can happen when replaying already-applied WAL records after a
- * standby crash or restart, or when replaying an XLOG_HEAP2_VISIBLE
- * record that marks as frozen a page which was already all-visible. It's
- * also quite common with records generated during index deletion
+ * This can happen whenever the changes in the WAL record do not affect
+ * visibility on a standby. For example: a record that only freezes an
+ * xmax from a locker.
+ *
+ * It's also quite common with records generated during index deletion
* (original execution of the deletion can reason that a recovery conflict
* which is sufficient for the deletion operation must take place before
* replay of the deletion record itself).
diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h
index ce3566ba949..516806fcca2 100644
--- a/src/include/access/heapam_xlog.h
+++ b/src/include/access/heapam_xlog.h
@@ -60,7 +60,7 @@
#define XLOG_HEAP2_PRUNE_ON_ACCESS 0x10
#define XLOG_HEAP2_PRUNE_VACUUM_SCAN 0x20
#define XLOG_HEAP2_PRUNE_VACUUM_CLEANUP 0x30
-#define XLOG_HEAP2_VISIBLE 0x40
+/* 0x40 was XLOG_HEAP2_VISIBLE */
#define XLOG_HEAP2_MULTI_INSERT 0x50
#define XLOG_HEAP2_LOCK_UPDATED 0x60
#define XLOG_HEAP2_NEW_CID 0x70
@@ -443,20 +443,6 @@ typedef struct xl_heap_inplace
#define MinSizeOfHeapInplace (offsetof(xl_heap_inplace, nmsgs) + sizeof(int))
-/*
- * This is what we need to know about setting a visibility map bit
- *
- * Backup blk 0: visibility map buffer
- * Backup blk 1: heap buffer
- */
-typedef struct xl_heap_visible
-{
- TransactionId snapshotConflictHorizon;
- uint8 flags;
-} xl_heap_visible;
-
-#define SizeOfHeapVisible (offsetof(xl_heap_visible, flags) + sizeof(uint8))
-
typedef struct xl_heap_new_cid
{
/*
@@ -500,11 +486,6 @@ extern void heap2_desc(StringInfo buf, XLogReaderState *record);
extern const char *heap2_identify(uint8 info);
extern void heap_xlog_logical_rewrite(XLogReaderState *r);
-extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer,
- Buffer vm_buffer,
- TransactionId snapshotConflictHorizon,
- uint8 vmflags);
-
/* in heapdesc.c, so it can be shared between frontend/backend code */
extern void heap_xlog_deserialize_prune_and_freeze(char *cursor, uint16 flags,
int *nplans, xlhp_freeze_plan **plans,
diff --git a/src/include/access/visibilitymap.h b/src/include/access/visibilitymap.h
index 52cde56be86..e4e0cfa989e 100644
--- a/src/include/access/visibilitymap.h
+++ b/src/include/access/visibilitymap.h
@@ -15,7 +15,6 @@
#define VISIBILITYMAP_H
#include "access/visibilitymapdefs.h"
-#include "access/xlogdefs.h"
#include "storage/block.h"
#include "storage/buf.h"
#include "storage/relfilelocator.h"
@@ -32,15 +31,9 @@ extern bool visibilitymap_clear(Relation rel, BlockNumber heapBlk,
extern void visibilitymap_pin(Relation rel, BlockNumber heapBlk,
Buffer *vmbuf);
extern bool visibilitymap_pin_ok(BlockNumber heapBlk, Buffer vmbuf);
-extern void visibilitymap_set(Relation rel,
- BlockNumber heapBlk, Buffer heapBuf,
- XLogRecPtr recptr,
- Buffer vmBuf,
- TransactionId cutoff_xid,
- uint8 flags);
-extern void visibilitymap_set_vmbits(BlockNumber heapBlk,
- Buffer vmBuf, uint8 flags,
- const RelFileLocator rlocator);
+extern void visibilitymap_set(BlockNumber heapBlk,
+ Buffer vmBuf, uint8 flags,
+ const RelFileLocator rlocator);
extern uint8 visibilitymap_get_status(Relation rel, BlockNumber heapBlk, Buffer *vmbuf);
extern void visibilitymap_count(Relation rel, BlockNumber *all_visible, BlockNumber *all_frozen);
extern BlockNumber visibilitymap_prepare_truncate(Relation rel,
diff --git a/src/include/access/visibilitymapdefs.h b/src/include/access/visibilitymapdefs.h
index 89153b3cd9a..e5794c8559e 100644
--- a/src/include/access/visibilitymapdefs.h
+++ b/src/include/access/visibilitymapdefs.h
@@ -21,14 +21,5 @@
#define VISIBILITYMAP_ALL_FROZEN 0x02
#define VISIBILITYMAP_VALID_BITS 0x03 /* OR of all valid visibilitymap
* flags bits */
-/*
- * To detect recovery conflicts during logical decoding on a standby, we need
- * to know if a table is a user catalog table. For that we add an additional
- * bit into xl_heap_visible.flags, in addition to the above.
- *
- * NB: VISIBILITYMAP_XLOG_* may not be passed to visibilitymap_set().
- */
-#define VISIBILITYMAP_XLOG_CATALOG_REL 0x04
-#define VISIBILITYMAP_XLOG_VALID_BITS (VISIBILITYMAP_VALID_BITS | VISIBILITYMAP_XLOG_CATALOG_REL)
#endif /* VISIBILITYMAPDEFS_H */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 0c07c945f05..adc858c2a97 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -4421,7 +4421,6 @@ xl_heap_prune
xl_heap_rewrite_mapping
xl_heap_truncate
xl_heap_update
-xl_heap_visible
xl_invalid_page
xl_invalid_page_key
xl_invalidations
--
2.43.0
[text/x-patch] v44-0006-Track-which-relations-are-modified-by-a-query.patch (6.4K, 7-v44-0006-Track-which-relations-are-modified-by-a-query.patch)
download | inline diff:
From 8f637aeb39efe65e629f616fbf4362ce9476ea1a Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Wed, 3 Dec 2025 15:07:24 -0500
Subject: [PATCH v44 06/10] Track which relations are modified by a query
Save the OIDs of modified relations in a list in the PlannedStmt. A
later commit will use this information during scans to control whether
or not on-access pruning is allowed to set the visibility map. Setting
the visibility map during a scan is counterproductive if the query is
going to modify the page immediately after.
Relations are considered modified if they are the target of INSERT,
UPDATE, DELETE, or MERGE, or if they have any row mark (including SELECT
FOR UPDATE/SHARE). All row mark types are included, even those which
don't actually modify tuples, because this list is only used as a hint
to avoid unnecessary work.
Author: Melanie Plageman <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Chao Li <[email protected]>
Discussion: https://postgr.es/m/F5CDD1B5-628C-44A1-9F85-3958C626F6A9%40gmail.com
---
src/backend/executor/execParallel.c | 1 +
src/backend/executor/nodeLockRows.c | 3 ++
src/backend/executor/nodeModifyTable.c | 9 ++++++
src/backend/optimizer/plan/planner.c | 44 +++++++++++++++++++++++++-
src/include/nodes/plannodes.h | 6 ++++
5 files changed, 62 insertions(+), 1 deletion(-)
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index ac84af294c9..5c1cf51d71c 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -188,6 +188,7 @@ ExecSerializePlan(Plan *plan, EState *estate)
pstmt->partPruneInfos = estate->es_part_prune_infos;
pstmt->rtable = estate->es_range_table;
pstmt->unprunableRelids = estate->es_unpruned_relids;
+ pstmt->modifiedRelOids = estate->es_plannedstmt->modifiedRelOids;
pstmt->permInfos = estate->es_rteperminfos;
pstmt->resultRelations = NIL;
pstmt->appendRelations = NIL;
diff --git a/src/backend/executor/nodeLockRows.c b/src/backend/executor/nodeLockRows.c
index 8d865470780..49b55d15e3e 100644
--- a/src/backend/executor/nodeLockRows.c
+++ b/src/backend/executor/nodeLockRows.c
@@ -113,6 +113,9 @@ lnext:
}
erm->ermActive = true;
+ Assert(list_member_oid(estate->es_plannedstmt->modifiedRelOids,
+ RelationGetRelid(erm->relation)));
+
/* fetch the tuple's ctid */
datum = ExecGetJunkAttribute(slot,
aerm->ctidAttNo,
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 4cd5e262e0f..12ecdd383cc 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -896,6 +896,9 @@ ExecInsert(ModifyTableContext *context,
resultRelationDesc = resultRelInfo->ri_RelationDesc;
+ Assert(list_member_oid(estate->es_plannedstmt->modifiedRelOids,
+ RelationGetRelid(resultRelationDesc)));
+
/*
* Open the table's indexes, if we have not done so already, so that we
* can add new index entries for the inserted tuple.
@@ -1523,6 +1526,9 @@ ExecDeleteAct(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
{
EState *estate = context->estate;
+ Assert(list_member_oid(estate->es_plannedstmt->modifiedRelOids,
+ RelationGetRelid(resultRelInfo->ri_RelationDesc)));
+
return table_tuple_delete(resultRelInfo->ri_RelationDesc, tupleid,
estate->es_output_cid,
estate->es_snapshot,
@@ -2205,6 +2211,9 @@ ExecUpdateAct(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
bool partition_constraint_failed;
TM_Result result;
+ Assert(list_member_oid(estate->es_plannedstmt->modifiedRelOids,
+ RelationGetRelid(resultRelInfo->ri_RelationDesc)));
+
updateCxt->crossPartUpdate = false;
/*
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 42604a0f75c..039796773a9 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -340,8 +340,10 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
RelOptInfo *final_rel;
Path *best_path;
Plan *top_plan;
+ List *modifiedRelOids = NIL;
ListCell *lp,
- *lr;
+ *lr,
+ *lc;
/*
* Set up global state for this planner invocation. This data is needed
@@ -661,6 +663,46 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
result->subplans = glob->subplans;
result->rewindPlanIDs = glob->rewindPlanIDs;
result->rowMarks = glob->finalrowmarks;
+
+ /*
+ * Compute modifiedRelOids from result relations and row marks.
+ *
+ * This is a superset of what the executor will actually modify/lock at
+ * runtime, because runtime partition pruning may eliminate some result
+ * relations, and parent row marks are included here but skipped by the
+ * executor.
+ *
+ * For partitioned tables, modifiedRelOids is expanded to include all
+ * descendant partition OIDs. This is necessary because tuple routing
+ * lazily expands leaf partitions at execution time.
+ */
+ foreach(lc, glob->resultRelations)
+ {
+ Index rti = lfirst_int(lc);
+ RangeTblEntry *rte = rt_fetch(rti, glob->finalrtable);
+
+ modifiedRelOids = list_append_unique_oid(modifiedRelOids, rte->relid);
+
+ if (rte->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ List *children = find_all_inheritors(rte->relid,
+ NoLock, NULL);
+ ListCell *lc2;
+
+ foreach(lc2, children)
+ modifiedRelOids = list_append_unique_oid(modifiedRelOids,
+ lfirst_oid(lc2));
+ }
+ }
+ foreach(lc, glob->finalrowmarks)
+ {
+ PlanRowMark *rc = (PlanRowMark *) lfirst(lc);
+ RangeTblEntry *rte = rt_fetch(rc->rti, glob->finalrtable);
+
+ modifiedRelOids = list_append_unique_oid(modifiedRelOids, rte->relid);
+ }
+ result->modifiedRelOids = modifiedRelOids;
+
result->relationOids = glob->relationOids;
result->invalItems = glob->invalItems;
result->paramExecTypes = glob->paramExecTypes;
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index b6185825fcb..6a7008cd50a 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -112,6 +112,12 @@ typedef struct PlannedStmt
*/
Bitmapset *unprunableRelids;
+ /*
+ * OIDs of relations modified by the query through
+ * UPDATE/DELETE/INSERT/MERGE or targeted by SELECT FOR UPDATE/SHARE.
+ */
+ List *modifiedRelOids;
+
/*
* list of RTEPermissionInfo nodes for rtable entries needing one
*/
--
2.43.0
[text/x-patch] v44-0007-Thread-flags-through-begin-scan-APIs.patch (32.9K, 8-v44-0007-Thread-flags-through-begin-scan-APIs.patch)
download | inline diff:
From 134bdfa0fa25bb74c954c179c88d7cc38ca14c56 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Thu, 19 Mar 2026 17:05:55 -0400
Subject: [PATCH v44 07/10] Thread flags through begin-scan APIs
Add a user-settable flags parameter to the table_beginscan_* wrappers,
index_beginscan(), table_index_fetch_begin(), and the table
AM callback index_fetch_begin(). This allows users to pass additional
context to be used when building the scan descriptors.
For index scans, a new uint32 flags field is added to
IndexFetchTableData, and the heap AM stores the caller-provided flags
there in heapam_index_fetch_begin().
This introduces an extension point for follow-up work to pass
per-scan information (such as whether the relation is read-only for the
current query) from the executor to the AM layer.
Author: Melanie Plageman <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Chao Li <[email protected]>
Discussion: https://postgr.es/m/F5CDD1B5-628C-44A1-9F85-3958C626F6A9%40gmail.com
---
contrib/pgrowlocks/pgrowlocks.c | 2 +-
src/backend/access/brin/brin.c | 3 +-
src/backend/access/gin/gininsert.c | 3 +-
src/backend/access/heap/heapam_handler.c | 9 ++-
src/backend/access/index/genam.c | 2 +
src/backend/access/index/indexam.c | 7 ++-
src/backend/access/nbtree/nbtsort.c | 2 +-
src/backend/access/table/tableam.c | 21 +++----
src/backend/commands/constraint.c | 2 +-
src/backend/commands/copyto.c | 2 +-
src/backend/commands/tablecmds.c | 8 +--
src/backend/commands/typecmds.c | 4 +-
src/backend/executor/execIndexing.c | 3 +-
src/backend/executor/execReplication.c | 12 ++--
src/backend/executor/nodeBitmapHeapscan.c | 2 +-
src/backend/executor/nodeIndexonlyscan.c | 5 +-
src/backend/executor/nodeIndexscan.c | 6 +-
src/backend/executor/nodeSamplescan.c | 2 +-
src/backend/executor/nodeSeqscan.c | 6 +-
src/backend/executor/nodeTidrangescan.c | 6 +-
src/backend/partitioning/partbounds.c | 2 +-
src/backend/utils/adt/selfuncs.c | 1 +
src/include/access/genam.h | 4 +-
src/include/access/heapam.h | 5 +-
src/include/access/relscan.h | 1 +
src/include/access/tableam.h | 72 +++++++++++++++--------
26 files changed, 117 insertions(+), 75 deletions(-)
diff --git a/contrib/pgrowlocks/pgrowlocks.c b/contrib/pgrowlocks/pgrowlocks.c
index ff3692c87c4..0556e9f7b88 100644
--- a/contrib/pgrowlocks/pgrowlocks.c
+++ b/contrib/pgrowlocks/pgrowlocks.c
@@ -115,7 +115,7 @@ pgrowlocks(PG_FUNCTION_ARGS)
RelationGetRelationName(rel));
/* Scan the relation */
- scan = table_beginscan(rel, GetActiveSnapshot(), 0, NULL);
+ scan = table_beginscan(rel, GetActiveSnapshot(), 0, NULL, 0);
hscan = (HeapScanDesc) scan;
attinmeta = TupleDescGetAttInMetadata(rsinfo->setDesc);
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 2a0f8c8e3b8..b25e814a996 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2844,7 +2844,8 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
indexInfo->ii_Concurrent = brinshared->isconcurrent;
scan = table_beginscan_parallel(heap,
- ParallelTableScanFromBrinShared(brinshared));
+ ParallelTableScanFromBrinShared(brinshared),
+ 0);
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
diff --git a/src/backend/access/gin/gininsert.c b/src/backend/access/gin/gininsert.c
index e54782d9dd8..555b16771e9 100644
--- a/src/backend/access/gin/gininsert.c
+++ b/src/backend/access/gin/gininsert.c
@@ -2068,7 +2068,8 @@ _gin_parallel_scan_and_build(GinBuildState *state,
indexInfo->ii_Concurrent = ginshared->isconcurrent;
scan = table_beginscan_parallel(heap,
- ParallelTableScanFromGinBuildShared(ginshared));
+ ParallelTableScanFromGinBuildShared(ginshared),
+ 0);
reltuples = table_index_build_scan(heap, index, indexInfo, true, progress,
ginBuildCallbackParallel, state, scan);
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 253a735b6c1..66726b22de6 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -80,11 +80,12 @@ heapam_slot_callbacks(Relation relation)
*/
static IndexFetchTableData *
-heapam_index_fetch_begin(Relation rel)
+heapam_index_fetch_begin(Relation rel, uint32 flags)
{
IndexFetchHeapData *hscan = palloc0_object(IndexFetchHeapData);
hscan->xs_base.rel = rel;
+ hscan->xs_base.flags = flags;
hscan->xs_cbuf = InvalidBuffer;
hscan->xs_vmbuffer = InvalidBuffer;
@@ -762,7 +763,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
tableScan = NULL;
heapScan = NULL;
- indexScan = index_beginscan(OldHeap, OldIndex, SnapshotAny, NULL, 0, 0);
+ indexScan = index_beginscan(OldHeap, OldIndex,
+ 0, /* flags */
+ SnapshotAny, NULL, 0, 0);
index_rescan(indexScan, NULL, 0, NULL, 0);
}
else
@@ -771,7 +774,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_SEQ_SCAN_HEAP);
- tableScan = table_beginscan(OldHeap, SnapshotAny, 0, (ScanKey) NULL);
+ tableScan = table_beginscan(OldHeap, SnapshotAny, 0, (ScanKey) NULL, 0);
heapScan = (HeapScanDesc) tableScan;
indexScan = NULL;
diff --git a/src/backend/access/index/genam.c b/src/backend/access/index/genam.c
index 5e89b86a62c..b099d956e41 100644
--- a/src/backend/access/index/genam.c
+++ b/src/backend/access/index/genam.c
@@ -455,6 +455,7 @@ systable_beginscan(Relation heapRelation,
}
sysscan->iscan = index_beginscan(heapRelation, irel,
+ 0, /* flags */
snapshot, NULL, nkeys, 0);
index_rescan(sysscan->iscan, idxkey, nkeys, NULL, 0);
sysscan->scan = NULL;
@@ -716,6 +717,7 @@ systable_beginscan_ordered(Relation heapRelation,
bsysscan = true;
sysscan->iscan = index_beginscan(heapRelation, indexRelation,
+ 0, /* flags */
snapshot, NULL, nkeys, 0);
index_rescan(sysscan->iscan, idxkey, nkeys, NULL, 0);
sysscan->scan = NULL;
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 5eb7e99ad3e..63d5daadca6 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -255,6 +255,7 @@ index_insert_cleanup(Relation indexRelation,
IndexScanDesc
index_beginscan(Relation heapRelation,
Relation indexRelation,
+ uint32 flags,
Snapshot snapshot,
IndexScanInstrumentation *instrument,
int nkeys, int norderbys)
@@ -284,7 +285,7 @@ index_beginscan(Relation heapRelation,
scan->instrument = instrument;
/* prepare to fetch index matches from table */
- scan->xs_heapfetch = table_index_fetch_begin(heapRelation);
+ scan->xs_heapfetch = table_index_fetch_begin(heapRelation, flags);
return scan;
}
@@ -593,7 +594,7 @@ IndexScanDesc
index_beginscan_parallel(Relation heaprel, Relation indexrel,
IndexScanInstrumentation *instrument,
int nkeys, int norderbys,
- ParallelIndexScanDesc pscan)
+ ParallelIndexScanDesc pscan, uint32 flags)
{
Snapshot snapshot;
IndexScanDesc scan;
@@ -615,7 +616,7 @@ index_beginscan_parallel(Relation heaprel, Relation indexrel,
scan->instrument = instrument;
/* prepare to fetch index matches from table */
- scan->xs_heapfetch = table_index_fetch_begin(heaprel);
+ scan->xs_heapfetch = table_index_fetch_begin(heaprel, flags);
return scan;
}
diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c
index 47a9bda30c9..016a5e546dd 100644
--- a/src/backend/access/nbtree/nbtsort.c
+++ b/src/backend/access/nbtree/nbtsort.c
@@ -1928,7 +1928,7 @@ _bt_parallel_scan_and_sort(BTSpool *btspool, BTSpool *btspool2,
indexInfo = BuildIndexInfo(btspool->index);
indexInfo->ii_Concurrent = btshared->isconcurrent;
scan = table_beginscan_parallel(btspool->heap,
- ParallelTableScanFromBTShared(btshared));
+ ParallelTableScanFromBTShared(btshared), 0);
reltuples = table_index_build_scan(btspool->heap, btspool->index, indexInfo,
true, progress, _bt_build_callback,
&buildstate, scan);
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index dfda1af412e..7a12e808b07 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -118,7 +118,7 @@ table_beginscan_catalog(Relation relation, int nkeys, ScanKeyData *key)
Snapshot snapshot = RegisterSnapshot(GetCatalogSnapshot(relid));
return table_beginscan_common(relation, snapshot, nkeys, key,
- NULL, flags);
+ NULL, flags, 0);
}
@@ -163,10 +163,10 @@ table_parallelscan_initialize(Relation rel, ParallelTableScanDesc pscan,
}
TableScanDesc
-table_beginscan_parallel(Relation relation, ParallelTableScanDesc pscan)
+table_beginscan_parallel(Relation relation, ParallelTableScanDesc pscan, uint32 flags)
{
Snapshot snapshot;
- uint32 flags = SO_TYPE_SEQSCAN |
+ uint32 internal_flags = SO_TYPE_SEQSCAN |
SO_ALLOW_STRAT | SO_ALLOW_SYNC | SO_ALLOW_PAGEMODE;
Assert(RelFileLocatorEquals(relation->rd_locator, pscan->phs_locator));
@@ -176,7 +176,7 @@ table_beginscan_parallel(Relation relation, ParallelTableScanDesc pscan)
/* Snapshot was serialized -- restore it */
snapshot = RestoreSnapshot((char *) pscan + pscan->phs_snapshot_off);
RegisterSnapshot(snapshot);
- flags |= SO_TEMP_SNAPSHOT;
+ internal_flags |= SO_TEMP_SNAPSHOT;
}
else
{
@@ -185,16 +185,17 @@ table_beginscan_parallel(Relation relation, ParallelTableScanDesc pscan)
}
return table_beginscan_common(relation, snapshot, 0, NULL,
- pscan, flags);
+ pscan, internal_flags, flags);
}
TableScanDesc
table_beginscan_parallel_tidrange(Relation relation,
- ParallelTableScanDesc pscan)
+ ParallelTableScanDesc pscan,
+ uint32 flags)
{
Snapshot snapshot;
- uint32 flags = SO_TYPE_TIDRANGESCAN | SO_ALLOW_PAGEMODE;
TableScanDesc sscan;
+ uint32 internal_flags = SO_TYPE_TIDRANGESCAN | SO_ALLOW_PAGEMODE;
Assert(RelFileLocatorEquals(relation->rd_locator, pscan->phs_locator));
@@ -206,7 +207,7 @@ table_beginscan_parallel_tidrange(Relation relation,
/* Snapshot was serialized -- restore it */
snapshot = RestoreSnapshot((char *) pscan + pscan->phs_snapshot_off);
RegisterSnapshot(snapshot);
- flags |= SO_TEMP_SNAPSHOT;
+ internal_flags |= SO_TEMP_SNAPSHOT;
}
else
{
@@ -215,7 +216,7 @@ table_beginscan_parallel_tidrange(Relation relation,
}
sscan = table_beginscan_common(relation, snapshot, 0, NULL,
- pscan, flags);
+ pscan, internal_flags, flags);
return sscan;
}
@@ -248,7 +249,7 @@ table_index_fetch_tuple_check(Relation rel,
bool found;
slot = table_slot_create(rel, NULL);
- scan = table_index_fetch_begin(rel);
+ scan = table_index_fetch_begin(rel, 0);
found = table_index_fetch_tuple(scan, tid, snapshot, slot, &call_again,
all_dead);
table_index_fetch_end(scan);
diff --git a/src/backend/commands/constraint.c b/src/backend/commands/constraint.c
index cc11c47b6f2..37cfbd63938 100644
--- a/src/backend/commands/constraint.c
+++ b/src/backend/commands/constraint.c
@@ -106,7 +106,7 @@ unique_key_recheck(PG_FUNCTION_ARGS)
*/
tmptid = checktid;
{
- IndexFetchTableData *scan = table_index_fetch_begin(trigdata->tg_relation);
+ IndexFetchTableData *scan = table_index_fetch_begin(trigdata->tg_relation, 0);
bool call_again = false;
if (!table_index_fetch_tuple(scan, &tmptid, SnapshotSelf, slot,
diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c
index faf62d959b4..29d7c3514b6 100644
--- a/src/backend/commands/copyto.c
+++ b/src/backend/commands/copyto.c
@@ -1336,7 +1336,7 @@ CopyRelationTo(CopyToState cstate, Relation rel, Relation root_rel, uint64 *proc
AttrMap *map = NULL;
TupleTableSlot *root_slot = NULL;
- scandesc = table_beginscan(rel, GetActiveSnapshot(), 0, NULL);
+ scandesc = table_beginscan(rel, GetActiveSnapshot(), 0, NULL, 0);
slot = table_slot_create(rel, NULL);
/*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 219f604df7b..ec9bbfe554a 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -6411,7 +6411,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap)
* checking all the constraints.
*/
snapshot = RegisterSnapshot(GetLatestSnapshot());
- scan = table_beginscan(oldrel, snapshot, 0, NULL);
+ scan = table_beginscan(oldrel, snapshot, 0, NULL, 0);
/*
* Switch to per-tuple memory context and reset it for each tuple
@@ -13980,7 +13980,7 @@ validateForeignKeyConstraint(char *conname,
*/
snapshot = RegisterSnapshot(GetLatestSnapshot());
slot = table_slot_create(rel, NULL);
- scan = table_beginscan(rel, snapshot, 0, NULL);
+ scan = table_beginscan(rel, snapshot, 0, NULL, 0);
perTupCxt = AllocSetContextCreate(CurrentMemoryContext,
"validateForeignKeyConstraint",
@@ -22882,7 +22882,7 @@ MergePartitionsMoveRows(List **wqueue, List *mergingPartitions, Relation newPart
/* Scan through the rows. */
snapshot = RegisterSnapshot(GetLatestSnapshot());
- scan = table_beginscan(mergingPartition, snapshot, 0, NULL);
+ scan = table_beginscan(mergingPartition, snapshot, 0, NULL, 0);
/*
* Switch to per-tuple memory context and reset it for each tuple
@@ -23346,7 +23346,7 @@ SplitPartitionMoveRows(List **wqueue, Relation rel, Relation splitRel,
/* Scan through the rows. */
snapshot = RegisterSnapshot(GetLatestSnapshot());
- scan = table_beginscan(splitRel, snapshot, 0, NULL);
+ scan = table_beginscan(splitRel, snapshot, 0, NULL, 0);
/*
* Switch to per-tuple memory context and reset it for each tuple
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index 3dab6bb5a79..5316cea7cec 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -3185,7 +3185,7 @@ validateDomainNotNullConstraint(Oid domainoid)
/* Scan all tuples in this relation */
snapshot = RegisterSnapshot(GetLatestSnapshot());
- scan = table_beginscan(testrel, snapshot, 0, NULL);
+ scan = table_beginscan(testrel, snapshot, 0, NULL, 0);
slot = table_slot_create(testrel, NULL);
while (table_scan_getnextslot(scan, ForwardScanDirection, slot))
{
@@ -3266,7 +3266,7 @@ validateDomainCheckConstraint(Oid domainoid, const char *ccbin, LOCKMODE lockmod
/* Scan all tuples in this relation */
snapshot = RegisterSnapshot(GetLatestSnapshot());
- scan = table_beginscan(testrel, snapshot, 0, NULL);
+ scan = table_beginscan(testrel, snapshot, 0, NULL, 0);
slot = table_slot_create(testrel, NULL);
while (table_scan_getnextslot(scan, ForwardScanDirection, slot))
{
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index 9d071e495c6..c46beedeb71 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -815,7 +815,8 @@ check_exclusion_or_unique_constraint(Relation heap, Relation index,
retry:
conflict = false;
found_self = false;
- index_scan = index_beginscan(heap, index, &DirtySnapshot, NULL, indnkeyatts, 0);
+ index_scan = index_beginscan(heap, index, 0, /* flags */
+ &DirtySnapshot, NULL, indnkeyatts, 0);
index_rescan(index_scan, scankeys, indnkeyatts, NULL, 0);
while (index_getnext_slot(index_scan, ForwardScanDirection, existing_slot))
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index 2497ee7edc5..23509771557 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -205,7 +205,9 @@ RelationFindReplTupleByIndex(Relation rel, Oid idxoid,
skey_attoff = build_replindex_scan_key(skey, rel, idxrel, searchslot);
/* Start an index scan. */
- scan = index_beginscan(rel, idxrel, &snap, NULL, skey_attoff, 0);
+ scan = index_beginscan(rel, idxrel,
+ 0, /* flags */
+ &snap, NULL, skey_attoff, 0);
retry:
found = false;
@@ -383,7 +385,7 @@ RelationFindReplTupleSeq(Relation rel, LockTupleMode lockmode,
/* Start a heap scan. */
InitDirtySnapshot(snap);
- scan = table_beginscan(rel, &snap, 0, NULL);
+ scan = table_beginscan(rel, &snap, 0, NULL, 0);
scanslot = table_slot_create(rel, NULL);
retry:
@@ -602,7 +604,7 @@ RelationFindDeletedTupleInfoSeq(Relation rel, TupleTableSlot *searchslot,
* not yet committed or those just committed prior to the scan are
* excluded in update_most_recent_deletion_info().
*/
- scan = table_beginscan(rel, SnapshotAny, 0, NULL);
+ scan = table_beginscan(rel, SnapshotAny, 0, NULL, 0);
scanslot = table_slot_create(rel, NULL);
table_rescan(scan, NULL);
@@ -666,7 +668,9 @@ RelationFindDeletedTupleInfoByIndex(Relation rel, Oid idxoid,
* not yet committed or those just committed prior to the scan are
* excluded in update_most_recent_deletion_info().
*/
- scan = index_beginscan(rel, idxrel, SnapshotAny, NULL, skey_attoff, 0);
+ scan = index_beginscan(rel, idxrel,
+ 0, /* flags */
+ SnapshotAny, NULL, skey_attoff, 0);
index_rescan(scan, skey, skey_attoff, NULL, 0);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 7cf8d23c742..324e2bed22c 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -148,7 +148,7 @@ BitmapTableScanSetup(BitmapHeapScanState *node)
table_beginscan_bm(node->ss.ss_currentRelation,
node->ss.ps.state->es_snapshot,
0,
- NULL);
+ NULL, 0);
}
node->ss.ss_currentScanDesc->st.rs_tbmiterator = tbmiterator;
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index 9eab81fd1c8..f733be0220c 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -92,6 +92,7 @@ IndexOnlyNext(IndexOnlyScanState *node)
*/
scandesc = index_beginscan(node->ss.ss_currentRelation,
node->ioss_RelationDesc,
+ 0, /* flags */
estate->es_snapshot,
node->ioss_Instrument,
node->ioss_NumScanKeys,
@@ -794,7 +795,7 @@ ExecIndexOnlyScanInitializeDSM(IndexOnlyScanState *node,
node->ioss_Instrument,
node->ioss_NumScanKeys,
node->ioss_NumOrderByKeys,
- piscan);
+ piscan, 0);
node->ioss_ScanDesc->xs_want_itup = true;
node->ioss_VMBuffer = InvalidBuffer;
@@ -860,7 +861,7 @@ ExecIndexOnlyScanInitializeWorker(IndexOnlyScanState *node,
node->ioss_Instrument,
node->ioss_NumScanKeys,
node->ioss_NumOrderByKeys,
- piscan);
+ piscan, 0);
node->ioss_ScanDesc->xs_want_itup = true;
/*
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 06143e94c5a..1a101df492b 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -110,6 +110,7 @@ IndexNext(IndexScanState *node)
*/
scandesc = index_beginscan(node->ss.ss_currentRelation,
node->iss_RelationDesc,
+ 0, /* flags */
estate->es_snapshot,
node->iss_Instrument,
node->iss_NumScanKeys,
@@ -206,6 +207,7 @@ IndexNextWithReorder(IndexScanState *node)
*/
scandesc = index_beginscan(node->ss.ss_currentRelation,
node->iss_RelationDesc,
+ 0, /* flags */
estate->es_snapshot,
node->iss_Instrument,
node->iss_NumScanKeys,
@@ -1730,7 +1732,7 @@ ExecIndexScanInitializeDSM(IndexScanState *node,
node->iss_Instrument,
node->iss_NumScanKeys,
node->iss_NumOrderByKeys,
- piscan);
+ piscan, 0);
/*
* If no run-time keys to calculate or they are ready, go ahead and pass
@@ -1794,7 +1796,7 @@ ExecIndexScanInitializeWorker(IndexScanState *node,
node->iss_Instrument,
node->iss_NumScanKeys,
node->iss_NumOrderByKeys,
- piscan);
+ piscan, 0);
/*
* If no run-time keys to calculate or they are ready, go ahead and pass
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index 6b0d65f752f..cc6b23abee0 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -298,7 +298,7 @@ tablesample_init(SampleScanState *scanstate)
0, NULL,
scanstate->use_bulkread,
allow_sync,
- scanstate->use_pagemode);
+ scanstate->use_pagemode, 0);
}
else
{
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 8f219f60a93..c2d9b7293de 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -71,7 +71,7 @@ SeqNext(SeqScanState *node)
*/
scandesc = table_beginscan(node->ss.ss_currentRelation,
estate->es_snapshot,
- 0, NULL);
+ 0, NULL, 0);
node->ss.ss_currentScanDesc = scandesc;
}
@@ -375,7 +375,7 @@ ExecSeqScanInitializeDSM(SeqScanState *node,
estate->es_snapshot);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
node->ss.ss_currentScanDesc =
- table_beginscan_parallel(node->ss.ss_currentRelation, pscan);
+ table_beginscan_parallel(node->ss.ss_currentRelation, pscan, 0);
}
/* ----------------------------------------------------------------
@@ -408,5 +408,5 @@ ExecSeqScanInitializeWorker(SeqScanState *node,
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
- table_beginscan_parallel(node->ss.ss_currentRelation, pscan);
+ table_beginscan_parallel(node->ss.ss_currentRelation, pscan, 0);
}
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 617713bde04..994f70989bc 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -245,7 +245,7 @@ TidRangeNext(TidRangeScanState *node)
scandesc = table_beginscan_tidrange(node->ss.ss_currentRelation,
estate->es_snapshot,
&node->trss_mintid,
- &node->trss_maxtid);
+ &node->trss_maxtid, 0);
node->ss.ss_currentScanDesc = scandesc;
}
else
@@ -460,7 +460,7 @@ ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt)
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
- pscan);
+ pscan, 0);
}
/* ----------------------------------------------------------------
@@ -494,5 +494,5 @@ ExecTidRangeScanInitializeWorker(TidRangeScanState *node,
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
- pscan);
+ pscan, 0);
}
diff --git a/src/backend/partitioning/partbounds.c b/src/backend/partitioning/partbounds.c
index 0ca312ac27d..b7c4e6d1071 100644
--- a/src/backend/partitioning/partbounds.c
+++ b/src/backend/partitioning/partbounds.c
@@ -3362,7 +3362,7 @@ check_default_partition_contents(Relation parent, Relation default_rel,
econtext = GetPerTupleExprContext(estate);
snapshot = RegisterSnapshot(GetLatestSnapshot());
tupslot = table_slot_create(part_rel, &estate->es_tupleTable);
- scan = table_beginscan(part_rel, snapshot, 0, NULL);
+ scan = table_beginscan(part_rel, snapshot, 0, NULL, 0);
/*
* Switch to per-tuple memory context and reset it for each tuple
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
index 53f85ccde01..0528f8166d8 100644
--- a/src/backend/utils/adt/selfuncs.c
+++ b/src/backend/utils/adt/selfuncs.c
@@ -7177,6 +7177,7 @@ get_actual_variable_endpoint(Relation heapRel,
GlobalVisTestFor(heapRel));
index_scan = index_beginscan(heapRel, indexRel,
+ 0, /* flags */
&SnapshotNonVacuumable, NULL,
1, 0);
/* Set it up for index-only scan */
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 1a27bf060b3..b98c20a0edc 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -156,6 +156,7 @@ extern void index_insert_cleanup(Relation indexRelation,
extern IndexScanDesc index_beginscan(Relation heapRelation,
Relation indexRelation,
+ uint32 flags,
Snapshot snapshot,
IndexScanInstrumentation *instrument,
int nkeys, int norderbys);
@@ -184,7 +185,8 @@ extern IndexScanDesc index_beginscan_parallel(Relation heaprel,
Relation indexrel,
IndexScanInstrumentation *instrument,
int nkeys, int norderbys,
- ParallelIndexScanDesc pscan);
+ ParallelIndexScanDesc pscan,
+ uint32 flags);
extern ItemPointer index_getnext_tid(IndexScanDesc scan,
ScanDirection direction);
extern bool index_fetch_heap(IndexScanDesc scan, TupleTableSlot *slot);
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 0ab322bf58b..47cbf2a20cf 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -95,10 +95,7 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
- /*
- * For sequential scans and bitmap heap scans. The current heap block's
- * corresponding page in the visibility map.
- */
+ /* Current heap block's corresponding page in the visibility map */
Buffer rs_vmbuffer;
/* these fields only used in page-at-a-time mode and for bitmap scans */
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index ce340c076f8..80ea0b437d1 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -122,6 +122,7 @@ typedef struct ParallelBlockTableScanWorkerData *ParallelBlockTableScanWorker;
typedef struct IndexFetchTableData
{
Relation rel;
+ uint32 flags;
} IndexFetchTableData;
struct IndexScanInstrumentation;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 06084752245..8357d05d83b 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -65,6 +65,16 @@ typedef enum ScanOptions
SO_TEMP_SNAPSHOT = 1 << 9,
} ScanOptions;
+/*
+ * Mask of flags that are set internally by the table_beginscan_* functions
+ * and must not be passed by callers.
+ */
+#define SO_INTERNAL_FLAGS \
+ (SO_TYPE_SEQSCAN | SO_TYPE_BITMAPSCAN | SO_TYPE_SAMPLESCAN | \
+ SO_TYPE_TIDSCAN | SO_TYPE_TIDRANGESCAN | SO_TYPE_ANALYZE | \
+ SO_ALLOW_STRAT | SO_ALLOW_SYNC | SO_ALLOW_PAGEMODE | \
+ SO_TEMP_SNAPSHOT)
+
/*
* Result codes for table_{update,delete,lock_tuple}, and for visibility
* routines inside table AMs.
@@ -420,7 +430,7 @@ typedef struct TableAmRoutine
*
* Tuples for an index scan can then be fetched via index_fetch_tuple.
*/
- struct IndexFetchTableData *(*index_fetch_begin) (Relation rel);
+ struct IndexFetchTableData *(*index_fetch_begin) (Relation rel, uint32 flags);
/*
* Reset index fetch. Typically this will release cross index fetch
@@ -871,12 +881,18 @@ extern TupleTableSlot *table_slot_create(Relation relation, List **reglist);
* A wrapper around the Table Access Method scan_begin callback, to centralize
* error checking. All calls to ->scan_begin() should go through this
* function.
+ *
+ * The caller-provided user_flags are validated against SO_INTERNAL_FLAGS to
+ * catch callers that accidentally pass scan-type or other internal flags.
*/
static TableScanDesc
table_beginscan_common(Relation rel, Snapshot snapshot, int nkeys,
ScanKeyData *key, ParallelTableScanDesc pscan,
- uint32 flags)
+ uint32 flags, uint32 user_flags)
{
+ Assert((user_flags & SO_INTERNAL_FLAGS) == 0);
+ flags |= user_flags;
+
/*
* We don't allow scans to be started while CheckXidAlive is set, except
* via systable_beginscan() et al. See detailed comments in xact.c where
@@ -894,12 +910,13 @@ table_beginscan_common(Relation rel, Snapshot snapshot, int nkeys,
*/
static inline TableScanDesc
table_beginscan(Relation rel, Snapshot snapshot,
- int nkeys, ScanKeyData *key)
+ int nkeys, ScanKeyData *key, uint32 flags)
{
- uint32 flags = SO_TYPE_SEQSCAN |
+ uint32 internal_flags = SO_TYPE_SEQSCAN |
SO_ALLOW_STRAT | SO_ALLOW_SYNC | SO_ALLOW_PAGEMODE;
- return table_beginscan_common(rel, snapshot, nkeys, key, NULL, flags);
+ return table_beginscan_common(rel, snapshot, nkeys, key, NULL,
+ internal_flags, flags);
}
/*
@@ -928,7 +945,7 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
if (allow_sync)
flags |= SO_ALLOW_SYNC;
- return table_beginscan_common(rel, snapshot, nkeys, key, NULL, flags);
+ return table_beginscan_common(rel, snapshot, nkeys, key, NULL, flags, 0);
}
/*
@@ -939,11 +956,12 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
*/
static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
- int nkeys, ScanKeyData *key)
+ int nkeys, ScanKeyData *key, uint32 flags)
{
- uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE;
+ uint32 internal_flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE;
- return table_beginscan_common(rel, snapshot, nkeys, key, NULL, flags);
+ return table_beginscan_common(rel, snapshot, nkeys, key, NULL,
+ internal_flags, flags);
}
/*
@@ -957,18 +975,19 @@ static inline TableScanDesc
table_beginscan_sampling(Relation rel, Snapshot snapshot,
int nkeys, ScanKeyData *key,
bool allow_strat, bool allow_sync,
- bool allow_pagemode)
+ bool allow_pagemode, uint32 flags)
{
- uint32 flags = SO_TYPE_SAMPLESCAN;
+ uint32 internal_flags = SO_TYPE_SAMPLESCAN;
if (allow_strat)
- flags |= SO_ALLOW_STRAT;
+ internal_flags |= SO_ALLOW_STRAT;
if (allow_sync)
- flags |= SO_ALLOW_SYNC;
+ internal_flags |= SO_ALLOW_SYNC;
if (allow_pagemode)
- flags |= SO_ALLOW_PAGEMODE;
+ internal_flags |= SO_ALLOW_PAGEMODE;
- return table_beginscan_common(rel, snapshot, nkeys, key, NULL, flags);
+ return table_beginscan_common(rel, snapshot, nkeys, key, NULL,
+ internal_flags, flags);
}
/*
@@ -981,7 +1000,7 @@ table_beginscan_tid(Relation rel, Snapshot snapshot)
{
uint32 flags = SO_TYPE_TIDSCAN;
- return table_beginscan_common(rel, snapshot, 0, NULL, NULL, flags);
+ return table_beginscan_common(rel, snapshot, 0, NULL, NULL, flags, 0);
}
/*
@@ -994,7 +1013,7 @@ table_beginscan_analyze(Relation rel)
{
uint32 flags = SO_TYPE_ANALYZE;
- return table_beginscan_common(rel, NULL, 0, NULL, NULL, flags);
+ return table_beginscan_common(rel, NULL, 0, NULL, NULL, flags, 0);
}
/*
@@ -1059,12 +1078,13 @@ table_scan_getnextslot(TableScanDesc sscan, ScanDirection direction, TupleTableS
static inline TableScanDesc
table_beginscan_tidrange(Relation rel, Snapshot snapshot,
ItemPointer mintid,
- ItemPointer maxtid)
+ ItemPointer maxtid, uint32 flags)
{
TableScanDesc sscan;
- uint32 flags = SO_TYPE_TIDRANGESCAN | SO_ALLOW_PAGEMODE;
+ uint32 internal_flags = SO_TYPE_TIDRANGESCAN | SO_ALLOW_PAGEMODE;
- sscan = table_beginscan_common(rel, snapshot, 0, NULL, NULL, flags);
+ sscan = table_beginscan_common(rel, snapshot, 0, NULL, NULL,
+ internal_flags, flags);
/* Set the range of TIDs to scan */
sscan->rs_rd->rd_tableam->scan_set_tidrange(sscan, mintid, maxtid);
@@ -1139,7 +1159,8 @@ extern void table_parallelscan_initialize(Relation rel,
* Caller must hold a suitable lock on the relation.
*/
extern TableScanDesc table_beginscan_parallel(Relation relation,
- ParallelTableScanDesc pscan);
+ ParallelTableScanDesc pscan,
+ uint32 flags);
/*
* Begin a parallel tid range scan. `pscan` needs to have been initialized
@@ -1149,7 +1170,8 @@ extern TableScanDesc table_beginscan_parallel(Relation relation,
* Caller must hold a suitable lock on the relation.
*/
extern TableScanDesc table_beginscan_parallel_tidrange(Relation relation,
- ParallelTableScanDesc pscan);
+ ParallelTableScanDesc pscan,
+ uint32 flags);
/*
* Restart a parallel scan. Call this in the leader process. Caller is
@@ -1175,8 +1197,10 @@ table_parallelscan_reinitialize(Relation rel, ParallelTableScanDesc pscan)
* Tuples for an index scan can then be fetched via table_index_fetch_tuple().
*/
static inline IndexFetchTableData *
-table_index_fetch_begin(Relation rel)
+table_index_fetch_begin(Relation rel, uint32 flags)
{
+ Assert((flags & SO_INTERNAL_FLAGS) == 0);
+
/*
* We don't allow scans to be started while CheckXidAlive is set, except
* via systable_beginscan() et al. See detailed comments in xact.c where
@@ -1185,7 +1209,7 @@ table_index_fetch_begin(Relation rel)
if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
elog(ERROR, "scan started during logical decoding");
- return rel->rd_tableam->index_fetch_begin(rel);
+ return rel->rd_tableam->index_fetch_begin(rel, flags);
}
/*
--
2.43.0
[text/x-patch] v44-0008-Pass-down-information-on-table-modification-to-s.patch (11.8K, 9-v44-0008-Pass-down-information-on-table-modification-to-s.patch)
download | inline diff:
From 8bdd2f0a153abc23cc7c87eab00255b95a05446c Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 2 Mar 2026 16:31:33 -0500
Subject: [PATCH v44 08/10] Pass down information on table modification to scan
node
Pass down information to sequential scan, index [only] scan, bitmap
table scan, sample scan, and TID range scan nodes on whether or not the
query modifies the relation being scanned. A later commit will use this
information to update the VM during on-access pruning only if the
relation is not modified by the query.
Author: Melanie Plageman <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Andrey Borodin <[email protected]>
Reviewed-by: Chao Li <[email protected]>
Discussion: https://postgr.es/m/4379FDA3-9446-4E2C-9C15-32EFE8D4F31B%40yandex-team.ru
---
src/backend/executor/execUtils.c | 8 ++++++++
src/backend/executor/nodeBitmapHeapscan.c | 6 +++++-
src/backend/executor/nodeIndexonlyscan.c | 15 ++++++++++++---
src/backend/executor/nodeIndexscan.c | 18 ++++++++++++++----
src/backend/executor/nodeSamplescan.c | 5 ++++-
src/backend/executor/nodeSeqscan.c | 18 +++++++++++++++---
src/backend/executor/nodeTidrangescan.c | 15 ++++++++++++---
src/include/access/tableam.h | 3 +++
src/include/executor/executor.h | 2 ++
9 files changed, 75 insertions(+), 15 deletions(-)
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index 9886ab06b69..ff05eca3a61 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -736,6 +736,14 @@ ExecRelationIsTargetRelation(EState *estate, Index scanrelid)
return list_member_int(estate->es_plannedstmt->resultRelations, scanrelid);
}
+/* Return true if the scan node's relation is not modified by the query */
+bool
+ScanRelIsReadOnly(ScanState *ss)
+{
+ return !list_member_oid(ss->ps.state->es_plannedstmt->modifiedRelOids,
+ RelationGetRelid(ss->ss_currentRelation));
+}
+
/* ----------------------------------------------------------------
* ExecOpenScanRelation
*
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 324e2bed22c..aec92c868ac 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -144,11 +144,15 @@ BitmapTableScanSetup(BitmapHeapScanState *node)
*/
if (!node->ss.ss_currentScanDesc)
{
+ uint32 flags = ScanRelIsReadOnly(&node->ss) ?
+ SO_HINT_REL_READ_ONLY : 0;
+
node->ss.ss_currentScanDesc =
table_beginscan_bm(node->ss.ss_currentRelation,
node->ss.ps.state->es_snapshot,
0,
- NULL, 0);
+ NULL,
+ flags);
}
node->ss.ss_currentScanDesc->st.rs_tbmiterator = tbmiterator;
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index f733be0220c..de9db45322c 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -85,6 +85,9 @@ IndexOnlyNext(IndexOnlyScanState *node)
if (scandesc == NULL)
{
+ uint32 flags = ScanRelIsReadOnly(&node->ss) ?
+ SO_HINT_REL_READ_ONLY : 0;
+
/*
* We reach here if the index only scan is not parallel, or if we're
* serially executing an index only scan that was planned to be
@@ -92,7 +95,7 @@ IndexOnlyNext(IndexOnlyScanState *node)
*/
scandesc = index_beginscan(node->ss.ss_currentRelation,
node->ioss_RelationDesc,
- 0, /* flags */
+ flags,
estate->es_snapshot,
node->ioss_Instrument,
node->ioss_NumScanKeys,
@@ -795,7 +798,10 @@ ExecIndexOnlyScanInitializeDSM(IndexOnlyScanState *node,
node->ioss_Instrument,
node->ioss_NumScanKeys,
node->ioss_NumOrderByKeys,
- piscan, 0);
+ piscan,
+ ScanRelIsReadOnly(&node->ss) ?
+ SO_HINT_REL_READ_ONLY : 0);
+
node->ioss_ScanDesc->xs_want_itup = true;
node->ioss_VMBuffer = InvalidBuffer;
@@ -861,7 +867,10 @@ ExecIndexOnlyScanInitializeWorker(IndexOnlyScanState *node,
node->ioss_Instrument,
node->ioss_NumScanKeys,
node->ioss_NumOrderByKeys,
- piscan, 0);
+ piscan,
+ ScanRelIsReadOnly(&node->ss) ?
+ SO_HINT_REL_READ_ONLY : 0);
+
node->ioss_ScanDesc->xs_want_itup = true;
/*
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 1a101df492b..9df4a699504 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -104,13 +104,16 @@ IndexNext(IndexScanState *node)
if (scandesc == NULL)
{
+ uint32 flags = ScanRelIsReadOnly(&node->ss) ?
+ SO_HINT_REL_READ_ONLY : 0;
+
/*
* We reach here if the index scan is not parallel, or if we're
* serially executing an index scan that was planned to be parallel.
*/
scandesc = index_beginscan(node->ss.ss_currentRelation,
node->iss_RelationDesc,
- 0, /* flags */
+ flags,
estate->es_snapshot,
node->iss_Instrument,
node->iss_NumScanKeys,
@@ -201,13 +204,16 @@ IndexNextWithReorder(IndexScanState *node)
if (scandesc == NULL)
{
+ uint32 flags = ScanRelIsReadOnly(&node->ss) ?
+ SO_HINT_REL_READ_ONLY : 0;
+
/*
* We reach here if the index scan is not parallel, or if we're
* serially executing an index scan that was planned to be parallel.
*/
scandesc = index_beginscan(node->ss.ss_currentRelation,
node->iss_RelationDesc,
- 0, /* flags */
+ flags,
estate->es_snapshot,
node->iss_Instrument,
node->iss_NumScanKeys,
@@ -1732,7 +1738,9 @@ ExecIndexScanInitializeDSM(IndexScanState *node,
node->iss_Instrument,
node->iss_NumScanKeys,
node->iss_NumOrderByKeys,
- piscan, 0);
+ piscan,
+ ScanRelIsReadOnly(&node->ss) ?
+ SO_HINT_REL_READ_ONLY : 0);
/*
* If no run-time keys to calculate or they are ready, go ahead and pass
@@ -1796,7 +1804,9 @@ ExecIndexScanInitializeWorker(IndexScanState *node,
node->iss_Instrument,
node->iss_NumScanKeys,
node->iss_NumOrderByKeys,
- piscan, 0);
+ piscan,
+ ScanRelIsReadOnly(&node->ss) ?
+ SO_HINT_REL_READ_ONLY : 0);
/*
* If no run-time keys to calculate or they are ready, go ahead and pass
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index cc6b23abee0..71c70e5e5c7 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -292,13 +292,16 @@ tablesample_init(SampleScanState *scanstate)
/* Now we can create or reset the HeapScanDesc */
if (scanstate->ss.ss_currentScanDesc == NULL)
{
+ uint32 flags = ScanRelIsReadOnly(&scanstate->ss) ?
+ SO_HINT_REL_READ_ONLY : 0;
+
scanstate->ss.ss_currentScanDesc =
table_beginscan_sampling(scanstate->ss.ss_currentRelation,
scanstate->ss.ps.state->es_snapshot,
0, NULL,
scanstate->use_bulkread,
allow_sync,
- scanstate->use_pagemode, 0);
+ scanstate->use_pagemode, flags);
}
else
{
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index c2d9b7293de..79470e6b9b5 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -65,13 +65,17 @@ SeqNext(SeqScanState *node)
if (scandesc == NULL)
{
+ uint32 flags = ScanRelIsReadOnly(&node->ss) ?
+ SO_HINT_REL_READ_ONLY : 0;
+
/*
* We reach here if the scan is not parallel, or if we're serially
* executing a scan that was planned to be parallel.
*/
scandesc = table_beginscan(node->ss.ss_currentRelation,
estate->es_snapshot,
- 0, NULL, 0);
+ 0, NULL, flags);
+
node->ss.ss_currentScanDesc = scandesc;
}
@@ -368,14 +372,18 @@ ExecSeqScanInitializeDSM(SeqScanState *node,
{
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ uint32 flags = ScanRelIsReadOnly(&node->ss) ?
+ SO_HINT_REL_READ_ONLY : 0;
pscan = shm_toc_allocate(pcxt->toc, node->pscan_len);
table_parallelscan_initialize(node->ss.ss_currentRelation,
pscan,
estate->es_snapshot);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
+
node->ss.ss_currentScanDesc =
- table_beginscan_parallel(node->ss.ss_currentRelation, pscan, 0);
+ table_beginscan_parallel(node->ss.ss_currentRelation, pscan,
+ flags);
}
/* ----------------------------------------------------------------
@@ -405,8 +413,12 @@ ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt)
{
ParallelTableScanDesc pscan;
+ uint32 flags = ScanRelIsReadOnly(&node->ss) ?
+ SO_HINT_REL_READ_ONLY : 0;
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
- table_beginscan_parallel(node->ss.ss_currentRelation, pscan, 0);
+ table_beginscan_parallel(node->ss.ss_currentRelation,
+ pscan,
+ flags);
}
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 994f70989bc..4257afd96ed 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -242,10 +242,13 @@ TidRangeNext(TidRangeScanState *node)
if (scandesc == NULL)
{
+ uint32 flags = ScanRelIsReadOnly(&node->ss) ?
+ SO_HINT_REL_READ_ONLY : 0;
+
scandesc = table_beginscan_tidrange(node->ss.ss_currentRelation,
estate->es_snapshot,
&node->trss_mintid,
- &node->trss_maxtid, 0);
+ &node->trss_maxtid, flags);
node->ss.ss_currentScanDesc = scandesc;
}
else
@@ -452,15 +455,18 @@ ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ uint32 flags = ScanRelIsReadOnly(&node->ss) ?
+ SO_HINT_REL_READ_ONLY : 0;
pscan = shm_toc_allocate(pcxt->toc, node->trss_pscanlen);
table_parallelscan_initialize(node->ss.ss_currentRelation,
pscan,
estate->es_snapshot);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
+
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
- pscan, 0);
+ pscan, flags);
}
/* ----------------------------------------------------------------
@@ -490,9 +496,12 @@ ExecTidRangeScanInitializeWorker(TidRangeScanState *node,
ParallelWorkerContext *pwcxt)
{
ParallelTableScanDesc pscan;
+ uint32 flags = ScanRelIsReadOnly(&node->ss) ?
+ SO_HINT_REL_READ_ONLY : 0;
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
+
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
- pscan, 0);
+ pscan, flags);
}
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 8357d05d83b..487e38292fa 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -63,6 +63,9 @@ typedef enum ScanOptions
/* unregister snapshot at scan end? */
SO_TEMP_SNAPSHOT = 1 << 9,
+
+ /* set if the query doesn't modify the relation */
+ SO_HINT_REL_READ_ONLY = 1 << 10,
} ScanOptions;
/*
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 07f4b1f7490..7979a17e4ec 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -690,6 +690,8 @@ extern void ExecCreateScanSlotFromOuterPlan(EState *estate,
extern bool ExecRelationIsTargetRelation(EState *estate, Index scanrelid);
+extern bool ScanRelIsReadOnly(ScanState *ss);
+
extern Relation ExecOpenScanRelation(EState *estate, Index scanrelid, int eflags);
extern void ExecInitRangeTable(EState *estate, List *rangeTable, List *permInfos,
--
2.43.0
[text/x-patch] v44-0009-Allow-on-access-pruning-to-set-pages-all-visible.patch (10.1K, 10-v44-0009-Allow-on-access-pruning-to-set-pages-all-visible.patch)
download | inline diff:
From 273a8d32ec46ac37286b1952f164b6290cee6c66 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 27 Feb 2026 16:33:40 -0500
Subject: [PATCH v44 09/10] Allow on-access pruning to set pages all-visible
Many queries do not modify the underlying relation. For such queries, if
on-access pruning occurs during the scan, we can check whether the page
has become all-visible and update the visibility map accordingly.
Previously, only vacuum and COPY FREEZE marked pages as all-visible or
all-frozen.
This commit implements on-access VM setting for sequential scans as well
as for the underlying heap relation in index scans and bitmap heap
scans.
Setting the visibility map on-access can avoid write amplification
caused by vacuum later needing to set the page all-visible, trigger a
write and potentially FPI. It also allows more frequent index-only
scans, since they require pages to be marked all-visible in the VM.
Author: Melanie Plageman <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Kirill Reshke <[email protected]>
Reviewed-by: Chao Li <[email protected]>
Discussion: https://postgr.es/m/flat/CAAKRu_ZMw6Npd_qm2KM%2BFwQ3cMOMx1Dh3VMhp8-V7SOLxdK9-g%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 3 +-
src/backend/access/heap/heapam_handler.c | 6 ++--
src/backend/access/heap/pruneheap.c | 46 +++++++++++++++++-------
src/backend/access/heap/vacuumlazy.c | 2 +-
src/include/access/heapam.h | 16 +++++++--
5 files changed, 54 insertions(+), 19 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 044f385e477..dbdf6521c42 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -633,7 +633,8 @@ heap_prepare_pagescan(TableScanDesc sscan)
/*
* Prune and repair fragmentation for the whole page, if possible.
*/
- heap_page_prune_opt(scan->rs_base.rs_rd, buffer, &scan->rs_vmbuffer);
+ heap_page_prune_opt(scan->rs_base.rs_rd, buffer, &scan->rs_vmbuffer,
+ (sscan->rs_flags & SO_HINT_REL_READ_ONLY));
/*
* We must hold share lock on the buffer content while examining tuple
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 66726b22de6..651efa0127a 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -148,7 +148,8 @@ heapam_index_fetch_tuple(struct IndexFetchTableData *scan,
*/
if (prev_buf != hscan->xs_cbuf)
heap_page_prune_opt(hscan->xs_base.rel, hscan->xs_cbuf,
- &hscan->xs_vmbuffer);
+ &hscan->xs_vmbuffer,
+ (hscan->xs_base.flags & SO_HINT_REL_READ_ONLY));
}
/* Obtain share-lock on the buffer so we can examine visibility */
@@ -2545,7 +2546,8 @@ BitmapHeapScanNextBlock(TableScanDesc scan,
/*
* Prune and repair fragmentation for the whole page, if possible.
*/
- heap_page_prune_opt(scan->rs_rd, buffer, &hscan->rs_vmbuffer);
+ heap_page_prune_opt(scan->rs_rd, buffer, &hscan->rs_vmbuffer,
+ scan->rs_flags & SO_HINT_REL_READ_ONLY);
/*
* We must hold share lock on the buffer content while examining tuple
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index fc5345e1dff..36260897503 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -44,6 +44,8 @@ typedef struct
bool mark_unused_now;
/* whether to attempt freezing tuples */
bool attempt_freeze;
+ /* whether to attempt setting the VM */
+ bool attempt_set_vm;
struct VacuumCutoffs *cutoffs;
Relation relation;
@@ -232,7 +234,8 @@ static void page_verify_redirects(Page page);
static bool heap_page_will_freeze(bool did_tuple_hint_fpi, bool do_prune, bool do_hint_prune,
PruneState *prstate);
-static bool heap_page_will_set_vm(PruneState *prstate, PruneReason reason);
+static bool heap_page_will_set_vm(PruneState *prstate, PruneReason reason,
+ bool do_prune, bool do_freeze);
/*
@@ -253,7 +256,8 @@ static bool heap_page_will_set_vm(PruneState *prstate, PruneReason reason);
* unpinning *vmbuffer.
*/
void
-heap_page_prune_opt(Relation relation, Buffer buffer, Buffer *vmbuffer)
+heap_page_prune_opt(Relation relation, Buffer buffer, Buffer *vmbuffer,
+ bool rel_read_only)
{
Page page = BufferGetPage(buffer);
TransactionId prune_xid;
@@ -336,6 +340,8 @@ heap_page_prune_opt(Relation relation, Buffer buffer, Buffer *vmbuffer)
* current implementation.
*/
params.options = HEAP_PAGE_PRUNE_ALLOW_FAST_PATH;
+ if (rel_read_only)
+ params.options |= HEAP_PAGE_PRUNE_SET_VM;
heap_page_prune_and_freeze(¶ms, &presult, &dummy_off_loc,
NULL, NULL);
@@ -392,6 +398,7 @@ prune_freeze_setup(PruneFreezeParams *params,
/* cutoffs must be provided if we will attempt freezing */
Assert(!(params->options & HEAP_PAGE_PRUNE_FREEZE) || params->cutoffs);
prstate->attempt_freeze = (params->options & HEAP_PAGE_PRUNE_FREEZE) != 0;
+ prstate->attempt_set_vm = (params->options & HEAP_PAGE_PRUNE_SET_VM) != 0;
prstate->cutoffs = params->cutoffs;
prstate->relation = params->relation;
prstate->block = BufferGetBlockNumber(params->buffer);
@@ -461,9 +468,8 @@ prune_freeze_setup(PruneFreezeParams *params,
* We track whether the page will be all-visible/all-frozen at the end of
* pruning and freezing. While examining tuple visibility, we'll set
* set_all_visible to false if there are tuples on the page not visible to
- * all running and future transactions. set_all_visible is always
- * maintained but only VACUUM will set the VM if the page ends up being
- * all-visible.
+ * all running and future transactions. If enabled for this scan, we will
+ * set the VM if the page ends up being all-visible.
*
* We also keep track of the newest live XID, which is used to calculate
* the snapshot conflict horizon for a WAL record setting the VM.
@@ -920,21 +926,37 @@ heap_page_fix_vm_corruption(PruneState *prstate, OffsetNumber offnum,
* This function does not actually set the VM bits or page-level visibility
* hint, PD_ALL_VISIBLE.
*
+ * This should be called only after do_freeze has been decided (and do_prune
+ * has been set), as these factor into our heuristic-based decision.
+ *
* Returns true if one or both VM bits should be set and false otherwise.
*/
static bool
-heap_page_will_set_vm(PruneState *prstate, PruneReason reason)
+heap_page_will_set_vm(PruneState *prstate, PruneReason reason,
+ bool do_prune, bool do_freeze)
{
- /*
- * Though on-access pruning maintains prstate->set_all_visible, we don't
- * set the VM on-access for now.
- */
- if (reason == PRUNE_ON_ACCESS)
+ if (!prstate->attempt_set_vm)
return false;
if (!prstate->set_all_visible)
return false;
+ /*
+ * If this is an on-access call and we're not actually pruning, avoid
+ * setting the visibility map if it would newly dirty the heap page or, if
+ * the page is already dirty, if doing so would require including a
+ * full-page image (FPI) of the heap page in the WAL. This situation
+ * should be rare, as on-access pruning is only attempted when
+ * pd_prune_xid is valid.
+ */
+ if (reason == PRUNE_ON_ACCESS && !do_prune && !do_freeze &&
+ (!BufferIsDirty(prstate->buffer) || XLogCheckBufferNeedsBackup(prstate->buffer)))
+ {
+ prstate->set_all_visible = false;
+ prstate->set_all_frozen = false;
+ return false;
+ }
+
prstate->new_vmbits = VISIBILITYMAP_ALL_VISIBLE;
if (prstate->set_all_frozen)
@@ -1167,7 +1189,7 @@ heap_page_prune_and_freeze(PruneFreezeParams *params,
Assert(!prstate.set_all_frozen || prstate.set_all_visible);
Assert(!prstate.set_all_visible || (prstate.lpdead_items == 0));
- do_set_vm = heap_page_will_set_vm(&prstate, params->reason);
+ do_set_vm = heap_page_will_set_vm(&prstate, params->reason, do_prune, do_freeze);
/*
* new_vmbits should be 0 regardless of whether or not the page is
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index f698c2d899b..24001b27387 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -2021,7 +2021,7 @@ lazy_scan_prune(LVRelState *vacrel,
.buffer = buf,
.vmbuffer = vmbuffer,
.reason = PRUNE_VACUUM_SCAN,
- .options = HEAP_PAGE_PRUNE_FREEZE,
+ .options = HEAP_PAGE_PRUNE_FREEZE | HEAP_PAGE_PRUNE_SET_VM,
.vistest = vacrel->vistest,
.cutoffs = &vacrel->cutoffs,
};
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 47cbf2a20cf..bfc7d482827 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -43,6 +43,7 @@
#define HEAP_PAGE_PRUNE_MARK_UNUSED_NOW (1 << 0)
#define HEAP_PAGE_PRUNE_FREEZE (1 << 1)
#define HEAP_PAGE_PRUNE_ALLOW_FAST_PATH (1 << 2)
+#define HEAP_PAGE_PRUNE_SET_VM (1 << 3)
typedef struct BulkInsertStateData *BulkInsertState;
typedef struct GlobalVisState GlobalVisState;
@@ -95,7 +96,12 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
- /* Current heap block's corresponding page in the visibility map */
+ /*
+ * For sequential scans, bitmap heap scans, TID range scans, and sample
+ * scans. The current heap block's corresponding page in the visibility
+ * map. If the relation is not modified by the query, on-access pruning
+ * may set the VM.
+ */
Buffer rs_vmbuffer;
/* these fields only used in page-at-a-time mode and for bitmap scans */
@@ -126,7 +132,11 @@ typedef struct IndexFetchHeapData
*/
Buffer xs_cbuf;
- /* Current heap block's corresponding page in the visibility map */
+ /*
+ * Current heap block's corresponding page in the visibility map. For
+ * index scans that do not modify the underlying heap table, on-access
+ * pruning may set the VM on-access.
+ */
Buffer xs_vmbuffer;
} IndexFetchHeapData;
@@ -431,7 +441,7 @@ extern TransactionId heap_index_delete_tuples(Relation rel,
/* in heap/pruneheap.c */
extern void heap_page_prune_opt(Relation relation, Buffer buffer,
- Buffer *vmbuffer);
+ Buffer *vmbuffer, bool rel_read_only);
extern void heap_page_prune_and_freeze(PruneFreezeParams *params,
PruneFreezeResult *presult,
OffsetNumber *off_loc,
--
2.43.0
[text/x-patch] v44-0010-Set-pd_prune_xid-on-insert.patch (8.8K, 11-v44-0010-Set-pd_prune_xid-on-insert.patch)
download | inline diff:
From 05279abac3bf623887c1e4883d360116ad5538b0 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Tue, 29 Jul 2025 16:12:56 -0400
Subject: [PATCH v44 10/10] Set pd_prune_xid on insert
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Now that visibility map (VM) updates can occur during read-only queries,
it makes sense to also set the page’s pd_prune_xid hint during inserts
and on the new page during updates.
This enables heap_page_prune_and_freeze() to set the VM all-visible
after a page is filled with newly inserted tuples the first time it is
read. This means the page will get set all-visible when it is still in
shared buffers and avoid potential I/O amplification when vacuum later
has to scan the page and set it all-visible. It also enables index-only
scans of newly inserted data much sooner.
This change also addresses a long-standing note in heap_insert() and
heap_multi_insert(), which observed that setting pd_prune_xid would
help clean up aborted insertions sooner. Without it, such tuples might
linger until VACUUM, whereas now they can be pruned earlier.
Author: Melanie Plageman <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Chao Li <[email protected]>
Discussion: https://postgr.es/m/flat/CAAKRu_ZMw6Npd_qm2KM%2BFwQ3cMOMx1Dh3VMhp8-V7SOLxdK9-g%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 40 +++++++++++++++++----------
src/backend/access/heap/heapam_xlog.c | 19 ++++++++++++-
src/backend/access/heap/pruneheap.c | 18 ++++++------
3 files changed, 52 insertions(+), 25 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index dbdf6521c42..ba11bbc03a5 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -2156,6 +2156,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
TransactionId xid = GetCurrentTransactionId();
HeapTuple heaptup;
Buffer buffer;
+ Page page;
Buffer vmbuffer = InvalidBuffer;
bool all_visible_cleared = false;
@@ -2182,6 +2183,8 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
&vmbuffer, NULL,
0);
+ page = BufferGetPage(buffer);
+
/*
* We're about to do the actual insert -- but check for conflict first, to
* avoid possibly having to roll back work we've just done.
@@ -2205,25 +2208,30 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
RelationPutHeapTuple(relation, buffer, heaptup,
(options & HEAP_INSERT_SPECULATIVE) != 0);
- if (PageIsAllVisible(BufferGetPage(buffer)))
+ if (PageIsAllVisible(page))
{
all_visible_cleared = true;
- PageClearAllVisible(BufferGetPage(buffer));
+ PageClearAllVisible(page);
visibilitymap_clear(relation,
ItemPointerGetBlockNumber(&(heaptup->t_self)),
vmbuffer, VISIBILITYMAP_VALID_BITS);
}
/*
- * XXX Should we set PageSetPrunable on this page ?
+ * Set pd_prune_xid to trigger heap_page_prune_and_freeze() once the page
+ * is full so that we can set the page all-visible in the VM on the next
+ * page access.
*
- * The inserting transaction may eventually abort thus making this tuple
- * DEAD and hence available for pruning. Though we don't want to optimize
- * for aborts, if no other tuple in this page is UPDATEd/DELETEd, the
- * aborted tuple will never be pruned until next vacuum is triggered.
+ * Setting pd_prune_xid is also handy if the inserting transaction
+ * eventually aborts making this tuple DEAD and hence available for
+ * pruning. If no other tuple in this page is UPDATEd/DELETEd, the aborted
+ * tuple would never otherwise be pruned until next vacuum is triggered.
*
- * If you do add PageSetPrunable here, add it in heap_xlog_insert too.
+ * Don't set it if we are in bootstrap mode or we are inserting a frozen
+ * tuple, as there is no further pruning/freezing needed in those cases.
*/
+ if (TransactionIdIsNormal(xid) && !(options & HEAP_INSERT_FROZEN))
+ PageSetPrunable(page, xid);
MarkBufferDirty(buffer);
@@ -2233,7 +2241,6 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
xl_heap_insert xlrec;
xl_heap_header xlhdr;
XLogRecPtr recptr;
- Page page = BufferGetPage(buffer);
uint8 info = XLOG_HEAP_INSERT;
int bufflags = 0;
@@ -2598,8 +2605,13 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
}
/*
- * XXX Should we set PageSetPrunable on this page ? See heap_insert()
+ * Set pd_prune_xid. See heap_insert() for more on why we do this when
+ * inserting tuples. This only makes sense if we aren't already
+ * setting the page frozen in the VM. We also don't set it in
+ * bootstrap mode.
*/
+ if (!all_frozen_set && TransactionIdIsNormal(xid))
+ PageSetPrunable(page, xid);
MarkBufferDirty(buffer);
@@ -4141,12 +4153,12 @@ l2:
* the subsequent page pruning will be a no-op and the hint will be
* cleared.
*
- * XXX Should we set hint on newbuf as well? If the transaction aborts,
- * there would be a prunable tuple in the newbuf; but for now we choose
- * not to optimize for aborts. Note that heap_xlog_update must be kept in
- * sync if this decision changes.
+ * We set the new page prunable as well. See heap_insert() for more on why
+ * we do this when inserting tuples.
*/
PageSetPrunable(page, xid);
+ if (newbuf != buffer)
+ PageSetPrunable(newpage, xid);
if (use_hot_update)
{
diff --git a/src/backend/access/heap/heapam_xlog.c b/src/backend/access/heap/heapam_xlog.c
index 1302bb13e18..f3f419d3dc1 100644
--- a/src/backend/access/heap/heapam_xlog.c
+++ b/src/backend/access/heap/heapam_xlog.c
@@ -450,6 +450,14 @@ heap_xlog_insert(XLogReaderState *record)
freespace = PageGetHeapFreeSpace(page); /* needed to update FSM below */
+ /*
+ * Set the page prunable to trigger on-access pruning later, which may
+ * set the page all-visible in the VM. See comments in heap_insert().
+ */
+ if (TransactionIdIsNormal(XLogRecGetXid(record)) &&
+ !HeapTupleHeaderXminFrozen(htup))
+ PageSetPrunable(page, XLogRecGetXid(record));
+
PageSetLSN(page, lsn);
if (xlrec->flags & XLH_INSERT_ALL_VISIBLE_CLEARED)
@@ -599,12 +607,19 @@ heap_xlog_multi_insert(XLogReaderState *record)
if (xlrec->flags & XLH_INSERT_ALL_VISIBLE_CLEARED)
PageClearAllVisible(page);
- /* XLH_INSERT_ALL_FROZEN_SET implies that all tuples are visible */
+ /*
+ * XLH_INSERT_ALL_FROZEN_SET implies that all tuples are visible. If
+ * we are not setting the page frozen, then set the page's prunable
+ * hint so that we trigger on-access pruning later which may set the
+ * page all-visible in the VM.
+ */
if (xlrec->flags & XLH_INSERT_ALL_FROZEN_SET)
{
PageSetAllVisible(page);
PageClearPrunable(page);
}
+ else
+ PageSetPrunable(page, XLogRecGetXid(record));
MarkBufferDirty(buffer);
}
@@ -921,6 +936,8 @@ heap_xlog_update(XLogReaderState *record, bool hot_update)
freespace = PageGetHeapFreeSpace(npage);
PageSetLSN(npage, lsn);
+ /* See heap_insert() for why we set pd_prune_xid on insert */
+ PageSetPrunable(npage, XLogRecGetXid(record));
MarkBufferDirty(nbuffer);
}
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index 36260897503..4e21c6f94ea 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -275,7 +275,8 @@ heap_page_prune_opt(Relation relation, Buffer buffer, Buffer *vmbuffer,
/*
* First check whether there's any chance there's something to prune,
* determining the appropriate horizon is a waste if there's no prune_xid
- * (i.e. no updates/deletes left potentially dead tuples around).
+ * (i.e. no updates/deletes left potentially dead tuples around and no
+ * inserts inserted new tuples that may be visible to all).
*/
prune_xid = PageGetPruneXid(page);
if (!TransactionIdIsValid(prune_xid))
@@ -1918,17 +1919,14 @@ heap_prune_record_unchanged_lp_normal(PruneState *prstate, OffsetNumber offnum)
prstate->set_all_visible = false;
prstate->set_all_frozen = false;
- /* The page should not be marked all-visible */
- if (PageIsAllVisible(page))
- heap_page_fix_vm_corruption(prstate, offnum,
- VM_CORRUPT_TUPLE_VISIBILITY);
-
/*
- * If we wanted to optimize for aborts, we might consider marking
- * the page prunable when we see INSERT_IN_PROGRESS. But we
- * don't. See related decisions about when to mark the page
- * prunable in heapam.c.
+ * Though there is nothing "prunable" on the page, we maintain
+ * pd_prune_xid for inserts so that we have the opportunity to
+ * mark them all-visible during the next round of pruning.
*/
+ heap_prune_record_prunable(prstate,
+ HeapTupleHeaderGetXmin(htup),
+ offnum);
break;
case HEAPTUPLE_DELETE_IN_PROGRESS:
--
2.43.0
view thread (144+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: eliminate xl_heap_visible to reduce WAL (and eventually set VM on-access)
In-Reply-To: <CAAKRu_bgP-DMZs=D2j2N0+U9+uWU5cVagw-yZLOuhYbWj_KwnA@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox