public inbox for [email protected]
help / color / mirror / Atom feedFrom: Melanie Plageman <[email protected]>
To: Heikki Linnakangas <[email protected]>
Cc: Pg Hackers <[email protected]>
Cc: Andres Freund <[email protected]>
Cc: Robert Haas <[email protected]>
Cc: Peter Geoghegan <[email protected]>
Subject: Re: Combine Prune and Freeze records emitted by vacuum
Date: Thu, 14 Mar 2024 20:56:58 -0400
Message-ID: <20240315005658.y7bvbzvnlciqmhd6@liskov> (raw)
In-Reply-To: <20240313232556.ohv3z7qzc5wtwqh5@liskov>
References: <CAAKRu_azf-zH=DgVbquZ3tFWjMY1w5pO8m-TXJaMdri8z3933g@mail.gmail.com>
<[email protected]>
<CAAKRu_bEo54H+sKXiCb4zsG32vXU7QoO+kGYTAoFyvYZzXxGiA@mail.gmail.com>
<[email protected]>
<20240313232556.ohv3z7qzc5wtwqh5@liskov>
On Wed, Mar 13, 2024 at 07:25:56PM -0400, Melanie Plageman wrote:
> On Mon, Mar 11, 2024 at 6:38 PM Heikki Linnakangas <[email protected]> wrote:
> >
> > On 09/03/2024 22:41, Melanie Plageman wrote:
> > > On Wed, Mar 6, 2024 at 7:59 AM Heikki Linnakangas <[email protected]> wrote:
> > >> Does GlobalVisTestIsRemovableXid() handle FrozenTransactionId correctly?
> > >
> > > Okay, so I thought a lot about this, and I don't understand how
> > > GlobalVisTestIsRemovableXid() would not handle FrozenTransactionId
> > > correctly.
> > >
> > > vacrel->cutoffs.OldestXmin is computed initially from
> > > GetOldestNonRemovableTransactionId() which uses ComputeXidHorizons().
> > > GlobalVisState is updated by ComputeXidHorizons() (when it is
> > > updated).
> > >
> > > I do see that the comment above GlobalVisTestIsRemovableXid() says
> > >
> > > * It is crucial that this only gets called for xids from a source that
> > > * protects against xid wraparounds (e.g. from a table and thus protected by
> > > * relfrozenxid).
> > >
> > > and then in
> > >
> > > * Convert 32 bit argument to FullTransactionId. We can do so safely
> > > * because we know the xid has to, at the very least, be between
> > > * [oldestXid, nextXid), i.e. within 2 billion of xid.
> > >
> > > I'm not sure what oldestXid is here.
> > > It's true that I don't see GlobalVisTestIsRemovableXid() being called
> > > anywhere else with an xmin as an input. I think that hints that it is
> > > not safe with FrozenTransactionId. But I don't see what could go
> > > wrong.
> > >
> > > Maybe it has something to do with converting it to a FullTransactionId?
> > >
> > > FullTransactionIdFromU64(U64FromFullTransactionId(rel) + (int32)
> > > (xid - rel_xid));
> > >
> > > Sorry, I couldn't quite figure it out :(
> >
> > I just tested it. No, GlobalVisTestIsRemovableXid does not work for
> > FrozenTransactionId. I just tested it with state->definitely_needed ==
> > {0, 4000000000} and xid == FrozenTransactionid, and it incorrectly
> > returned 'false'. It treats FrozenTransactionId as if was a regular xid '2'.
>
> I see. Looking at the original code:
> if (!TransactionIdPrecedes(xmin,
> vacrel->cutoffs.OldestXmin))
>
> And the source code for TransactionIdPrecedes:
>
> if (!TransactionIdIsNormal(id1) || !TransactionIdIsNormal(id2))
> return (id1 < id2);
>
> diff = (int32) (id1 - id2);
> return (diff < 0);
>
> In your example, It seems like you mean GlobalVisState->maybe_needed is
> 0 and GlobalVisState->definitely_needed = 4000000000. In this example,
> if vacrel->cutoffs.OldestXmin was 0, we would get a wrong answer also.
>
> But, I do see that the comparison done by TransactionIdPrecedes() is is
> quite different than that done by FullTransactionIdPrecedes() because of
> the modulo 2**32 arithmetic.
>
> Solving the handling of FrozenTransactionId specifically by
> GlobalVisTestIsRemovableXid() seems like it would be done easily in our
> case by wrapping it in a function which checks if
> TransactionIdIsNormal() and returns true if it is not normal. But, I'm
> not sure if I am missing the larger problem.
I've made such a wrapper in attached v3.
> > >> The XLOG_HEAP2_FREEZE_PAGE record is a little smaller than
> > >> XLOG_HEAP2_PRUNE. But we could optimize the XLOG_HEAP2_PRUNE format for
> > >> the case that there's no pruning, just freezing. The record format
> > >> (xl_heap_prune) looks pretty complex as it is, I think it could be made
> > >> both more compact and more clear with some refactoring.
> > >
> > > I'm happy to change up xl_heap_prune format. In its current form,
> > > according to pahole, it has no holes and just 3 bytes of padding at
> > > the end.
> > >
> > > One way we could make it smaller is by moving the isCatalogRel member
> > > to directly after snapshotConflictHorizon and then adding a flags
> > > field and defining flags to indicate whether or not other members
> > > exist at all. We could set bits for HAS_FREEZE_PLANS, HAS_REDIRECTED,
> > > HAS_UNUSED, HAS_DEAD. Then I would remove the non-optional uint16
> > > nredirected, nunused, nplans, ndead and just put the number of
> > > redirected/unused/etc at the beginning of the arrays containing the
> > > offsets.
> >
> > Sounds good.
> >
> > > Then I could write various macros for accessing them. That
> > > would make it smaller, but it definitely wouldn't make it less complex
> > > (IMO).
> >
> > I don't know, it might turn out not that complex. If you define the
> > formats of each of those "sub-record types" as explicit structs, per
> > attached sketch, you won't need so many macros. Some care is still
> > needed with alignment though.
>
> In the attached v2, I've done as you suggested and made all members
> except flags and snapshotConflictHorizon optional in the xl_heap_prune
> struct and obsoleted the xl_heap_freeze struct. I've kept the actual
> xl_heap_freeze_page struct and heap_xlog_freeze_page() function so that
> we can replay previously made XLOG_HEAP2_FREEZE_PAGE records.
>
> Because we may set line pointers unused during vacuum's first pass, I
> couldn't use the presence of nowunused without redirected or dead items
> to indicate that this was a record emitted by vacuum's second pass. As
> such, I haven't obsoleted the xl_heap_vacuum struct. I was thinking I
> could add a flag that indicates the record was emitted by vacuum's
> second pass? We would want to distinguish this so that we could set the
> items unused without calling heap_page_prune_execute() -- because that
> calls PageRepairFragmentation() which requires a full cleanup lock.
Okay, so I was going to start using xl_heap_prune for vacuum here too,
but I realized it would be bigger because of the
snapshotConflictHorizon. Do you think there is a non-terrible way to
make the snapshotConflictHorizon optional? Like with a flag?
> I introduced a few sub-record types similar to what you suggested --
> they help a bit with alignment, so I think they are worth keeping. There
> are comments around them, but perhaps a larger diagram of the layout of
> a the new XLOG_HEAP2_PRUNE record would be helpful.
I started doing this, but I can't find a way of laying out the diagram
that pgindent doesn't destroy. I thought I remember other diagrams in
the source code showing the layout of something (something with pages
somewhere?) that don't get messed up by pgindent, but I couldn't find
them.
> There is a bit of duplicated code between heap_xlog_prune() and
> heap2_desc() since they both need to deserialize the record. Before the
> code to do this was small and it didn't matter, but it might be worth
> refactoring it that way now.
I have added a helper function to do the deserialization,
heap_xlog_deserialize_prune_and_freeze(). But I didn't start using it in
heap2_desc() because of the way the pg_waldump build file works. Do you
think the helper belongs in any of waldump's existing sources?
pg_waldump_sources = files(
'compat.c',
'pg_waldump.c',
'rmgrdesc.c',
)
pg_waldump_sources += rmgr_desc_sources
pg_waldump_sources += xlogreader_sources
pg_waldump_sources += files('../../backend/access/transam/xlogstats.c')
Otherwise, I assume I am suppose to avoid adding some big new include to
waldump.
> On Wed, Mar 6, 2024 at 7:59 AM Heikki Linnakangas <[email protected]> wrote:
> >
> > I don't think we need XLOG_HEAP2_FREEZE_PAGE as a separate record type
> > anymore. By removing that, you also get rid of the freeze-only codepath
> > near the end of heap_page_prune(), and the
> > heap_freeze_execute_prepared() function.
> >
> > The XLOG_HEAP2_FREEZE_PAGE record is a little smaller than
> > XLOG_HEAP2_PRUNE. But we could optimize the XLOG_HEAP2_PRUNE format for
> > the case that there's no pruning, just freezing. The record format
> > (xl_heap_prune) looks pretty complex as it is, I think it could be made
> > both more compact and more clear with some refactoring.
>
> On the point of removing the freeze-only code path from
> heap_page_prune() (now heap_page_prune_and_freeze()): while doing this,
> I realized that heap_pre_freeze_checks() was not being called in the
> case that we decide to freeze because we emitted an FPI while setting
> the hint bit. I've fixed that, however, I've done so by moving
> heap_pre_freeze_checks() into the critical section. I think that is not
> okay? I could move it earlier and not do call it when the hint bit FPI
> leads us to freeze tuples. But, I think that would lead to us doing a
> lot less validation of tuples being frozen when checksums are enabled.
> Or, I could make two critical sections?
I found another approach and just do the pre-freeze checks if we are
considering freezing except for the FPI criteria.
> > HeapPageFreeze has two "trackers", for the "freeze" and "no freeze"
> > cases. heap_page_prune() needs to track both, until it decides whether
> > to freeze or not. But it doesn't make much sense that the caller
> > (lazy_scan_prune()) has to initialize both, and has to choose which of
> > the values to use after the call depending on whether heap_page_prune()
> > froze or not. The two trackers should be just heap_page_prune()'s
> > internal business.
>
> I've added new_relminmxid and new_relfrozenxid to PruneFreezeResult and
> set them appropriately in heap_page_prune_and_freeze().
>
> It's a bit sad because if it wasn't for vacrel->skippedallvis,
> vacrel->NewRelfrozenXid and vacrel->NewRelminMxid would be
> vacrel->cutoffs.OldestXmin and vacrel->cutoffs.OldestMxact respectively
> and we could avoid having lazy_scan_prune() initializing the
> HeapPageFreeze altogether and just pass VacuumCutoffs (and
> heap_page_prune_opt() could pass NULL) to heap_page_prune_and_freeze().
>
> I think it is probably worse to add both of them as additional optional
> arguments, so I've just left lazy_scan_prune() with the job of
> initializing them.
>
> In heap_page_prune_and_freeze(), I initialize new_relminmxid and
> new_relfrozenxid to InvalidMultiXactId and InvalidTransactionId
> respectively because on-access pruning doesn't have a value to set them
> to. But I wasn't sure if this was okay -- since I don't see validation
> that TransactionIdIsValid() in vac_update_relstats(). It will work now
> -- just worried about future issues. I could add an assert there?
I looked more closely at vac_update_relstats() and it won't update
relfrozenxid unless the proposed value is smaller than the existing one.
And for MultiXactIds, it checks if it is valid. So, this is not an
issue.
I've also optimized the member ordering of PruneFreezeResult. pahole
identified some avoidable holes. I went through each commit and
optimized the PruneResult data structure as members are being added and
removed.
- Melanie
Attachments:
[text/x-diff] v3-0001-lazy_scan_prune-tests-tuple-vis-with-GlobalVisTes.patch (2.3K, ../20240315005658.y7bvbzvnlciqmhd6@liskov/2-v3-0001-lazy_scan_prune-tests-tuple-vis-with-GlobalVisTes.patch)
download | inline diff:
From 377172183a63e133d62996768e0f927d54aa7adf Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Sat, 6 Jan 2024 13:14:47 -0500
Subject: [PATCH v3 01/17] lazy_scan_prune tests tuple vis with GlobalVisTest
One requirement for eventually combining the prune and freeze records,
is that we must check during pruning if live tuples on the page are
visible to everyone and thus, whether or not the page is all visible. We
only consider opportunistically freezing tuples if the whole page is all
visible and could be set all frozen.
During pruning (in heap_page_prune()), we do not have access to
VacuumCutoffs -- as on access pruning also calls heap_page_prune(). We
do, however, have access to a GlobalVisState. This can be used to
determine whether or not the tuple is visible to everyone. It also has
the potential of being more up-to-date than VacuumCutoffs->OldestXmin.
This commit simply modifies lazy_scan_prune() to use GlobalVisState
instead of OldestXmin. Future commits will move the
all_visible/all_frozen calculation into heap_page_prune().
---
src/backend/access/heap/vacuumlazy.c | 17 +++++++++++++++--
1 file changed, 15 insertions(+), 2 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 18004907750..fe31c0125d6 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -1373,6 +1373,20 @@ lazy_scan_new_or_empty(LVRelState *vacrel, Buffer buf, BlockNumber blkno,
return false;
}
+/*
+ * Wrap GlobalVisTestIsRemovableXid() to handle FrozenTransactionIds when we
+ * are examining tuple xmins to determine if the page is all-visible during
+ * pruning. Old tuples may have FrozenTransactionId xmins.
+ */
+static inline bool
+prune_freeze_xmin_is_removable(GlobalVisState *visstate, TransactionId xmin)
+{
+ if (xmin == FrozenTransactionId)
+ return true;
+
+ return GlobalVisTestIsRemovableXid(visstate, xmin);
+}
+
/*
* lazy_scan_prune() -- lazy_scan_heap() pruning and freezing.
*
@@ -1582,8 +1596,7 @@ lazy_scan_prune(LVRelState *vacrel,
* that everyone sees it as committed?
*/
xmin = HeapTupleHeaderGetXmin(htup);
- if (!TransactionIdPrecedes(xmin,
- vacrel->cutoffs.OldestXmin))
+ if (!prune_freeze_xmin_is_removable(vacrel->vistest, xmin))
{
all_visible = false;
break;
--
2.40.1
[text/x-diff] v3-0002-Pass-heap_prune_chain-PruneResult-output-paramete.patch (3.3K, ../20240315005658.y7bvbzvnlciqmhd6@liskov/3-v3-0002-Pass-heap_prune_chain-PruneResult-output-paramete.patch)
download | inline diff:
From a0ae45fd2b1bec08d5040a08663df94c37ad7a9f Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Sat, 6 Jan 2024 13:39:59 -0500
Subject: [PATCH v3 02/17] Pass heap_prune_chain() PruneResult output parameter
Future commits will set other members of PruneResult in
heap_prune_chain(), so start passing it as an output parameter now. This
eliminates the output parameter htsv -- the array of HTSV_Results --
since that is a member of the PruneResult.
---
src/backend/access/heap/pruneheap.c | 15 +++++++--------
1 file changed, 7 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index 4f12413b8b1..4a2bf3dd780 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -61,8 +61,7 @@ static HTSV_Result heap_prune_satisfies_vacuum(PruneState *prstate,
Buffer buffer);
static int heap_prune_chain(Buffer buffer,
OffsetNumber rootoffnum,
- int8 *htsv,
- PruneState *prstate);
+ PruneState *prstate, PruneResult *presult);
static void heap_prune_record_prunable(PruneState *prstate, TransactionId xid);
static void heap_prune_record_redirect(PruneState *prstate,
OffsetNumber offnum, OffsetNumber rdoffnum);
@@ -325,7 +324,7 @@ heap_page_prune(Relation relation, Buffer buffer,
/* Process this item or chain of items */
presult->ndeleted += heap_prune_chain(buffer, offnum,
- presult->htsv, &prstate);
+ &prstate, presult);
}
/* Clear the offset information once we have processed the given page. */
@@ -454,7 +453,7 @@ heap_prune_satisfies_vacuum(PruneState *prstate, HeapTuple tup, Buffer buffer)
/*
* Prune specified line pointer or a HOT chain originating at line pointer.
*
- * Tuple visibility information is provided in htsv.
+ * Tuple visibility information is provided in presult->htsv.
*
* If the item is an index-referenced tuple (i.e. not a heap-only tuple),
* the HOT chain is pruned by removing all DEAD tuples at the start of the HOT
@@ -484,7 +483,7 @@ heap_prune_satisfies_vacuum(PruneState *prstate, HeapTuple tup, Buffer buffer)
*/
static int
heap_prune_chain(Buffer buffer, OffsetNumber rootoffnum,
- int8 *htsv, PruneState *prstate)
+ PruneState *prstate, PruneResult *presult)
{
int ndeleted = 0;
Page dp = (Page) BufferGetPage(buffer);
@@ -505,7 +504,7 @@ heap_prune_chain(Buffer buffer, OffsetNumber rootoffnum,
*/
if (ItemIdIsNormal(rootlp))
{
- Assert(htsv[rootoffnum] != -1);
+ Assert(presult->htsv[rootoffnum] != -1);
htup = (HeapTupleHeader) PageGetItem(dp, rootlp);
if (HeapTupleHeaderIsHeapOnly(htup))
@@ -528,7 +527,7 @@ heap_prune_chain(Buffer buffer, OffsetNumber rootoffnum,
* either here or while following a chain below. Whichever path
* gets there first will mark the tuple unused.
*/
- if (htsv[rootoffnum] == HEAPTUPLE_DEAD &&
+ if (presult->htsv[rootoffnum] == HEAPTUPLE_DEAD &&
!HeapTupleHeaderIsHotUpdated(htup))
{
heap_prune_record_unused(prstate, rootoffnum);
@@ -625,7 +624,7 @@ heap_prune_chain(Buffer buffer, OffsetNumber rootoffnum,
*/
tupdead = recent_dead = false;
- switch (htsv_get_valid_status(htsv[offnum]))
+ switch (htsv_get_valid_status(presult->htsv[offnum]))
{
case HEAPTUPLE_DEAD:
tupdead = true;
--
2.40.1
[text/x-diff] v3-0003-heap_page_prune-sets-all_visible-and-frz_conflict.patch (19.5K, ../20240315005658.y7bvbzvnlciqmhd6@liskov/4-v3-0003-heap_page_prune-sets-all_visible-and-frz_conflict.patch)
download | inline diff:
From 8759a7a28009cbafc75c1ef986454847ef90338f Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Sat, 6 Jan 2024 14:01:37 -0500
Subject: [PATCH v3 03/17] heap_page_prune sets all_visible and
frz_conflict_horizon
In order to combine the prune and freeze records, we must know if the
page is eligible to be opportunistically frozen before finishing
pruning. Save all_visible in the PruneResult and set it to false when we
see non-removable tuples which are not visible to everyone.
We will also need to ensure that the snapshotConflictHorizon for the combined
prune + freeze record is the more conservative of that calculated for each of
pruning and freezing. Calculate the visibility_cutoff_xid for the purposes of
freezing -- the newest xmin on the page -- in heap_page_prune() and save it in
PruneResult.frz_conflict_horizon.
---
src/backend/access/heap/pruneheap.c | 136 +++++++++++++++++++++++++--
src/backend/access/heap/vacuumlazy.c | 130 ++++++-------------------
src/include/access/heapam.h | 3 +
3 files changed, 160 insertions(+), 109 deletions(-)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index 4a2bf3dd780..42fd4a74845 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -65,8 +65,10 @@ static int heap_prune_chain(Buffer buffer,
static void heap_prune_record_prunable(PruneState *prstate, TransactionId xid);
static void heap_prune_record_redirect(PruneState *prstate,
OffsetNumber offnum, OffsetNumber rdoffnum);
-static void heap_prune_record_dead(PruneState *prstate, OffsetNumber offnum);
-static void heap_prune_record_dead_or_unused(PruneState *prstate, OffsetNumber offnum);
+static void heap_prune_record_dead(PruneState *prstate, OffsetNumber offnum,
+ PruneResult *presult);
+static void heap_prune_record_dead_or_unused(PruneState *prstate, OffsetNumber offnum,
+ PruneResult *presult);
static void heap_prune_record_unused(PruneState *prstate, OffsetNumber offnum);
static void page_verify_redirects(Page page);
@@ -187,6 +189,20 @@ heap_page_prune_opt(Relation relation, Buffer buffer)
}
+/*
+ * Wrap GlobalVisTestIsRemovableXid() to handle FrozenTransactionIds when we
+ * are examining tuple xmins to determine if the page is all-visible during
+ * pruning. Old tuples may have FrozenTransactionId xmins.
+ */
+static inline bool
+prune_freeze_xmin_is_removable(GlobalVisState *visstate, TransactionId xmin)
+{
+ if (xmin == FrozenTransactionId)
+ return true;
+
+ return GlobalVisTestIsRemovableXid(visstate, xmin);
+}
+
/*
* Prune and repair fragmentation in the specified page.
*
@@ -249,6 +265,14 @@ heap_page_prune(Relation relation, Buffer buffer,
presult->ndeleted = 0;
presult->nnewlpdead = 0;
+ /*
+ * Keep track of whether or not the page is all_visible in case the caller
+ * wants to use this information to update the VM.
+ */
+ presult->all_visible = true;
+ /* for recovery conflicts */
+ presult->frz_conflict_horizon = InvalidTransactionId;
+
maxoff = PageGetMaxOffsetNumber(page);
tup.t_tableOid = RelationGetRelid(prstate.rel);
@@ -300,8 +324,92 @@ heap_page_prune(Relation relation, Buffer buffer,
presult->htsv[offnum] = heap_prune_satisfies_vacuum(&prstate, &tup,
buffer);
+ switch (presult->htsv[offnum])
+ {
+ case HEAPTUPLE_DEAD:
+
+ /*
+ * Deliberately delay unsetting all_visible until later during
+ * pruning. Removable dead tuples shouldn't preclude freezing
+ * the page. After finishing this first pass of tuple
+ * visibility checks, initialize all_visible_except_removable
+ * with the current value of all_visible to indicate whether
+ * or not the page is all visible except for dead tuples. This
+ * will allow us to attempt to freeze the page after pruning.
+ * Later during pruning, if we encounter an LP_DEAD item or
+ * are setting an item LP_DEAD, we will unset all_visible. As
+ * long as we unset it before updating the visibility map,
+ * this will be correct.
+ */
+ break;
+ case HEAPTUPLE_LIVE:
+
+ /*
+ * Is the tuple definitely visible to all transactions?
+ *
+ * NB: Like with per-tuple hint bits, we can't set the
+ * PD_ALL_VISIBLE flag if the inserter committed
+ * asynchronously. See SetHintBits for more info. Check that
+ * the tuple is hinted xmin-committed because of that.
+ */
+ if (presult->all_visible)
+ {
+ TransactionId xmin;
+
+ if (!HeapTupleHeaderXminCommitted(htup))
+ {
+ presult->all_visible = false;
+ break;
+ }
+
+ /*
+ * The inserter definitely committed. But is it old enough
+ * that everyone sees it as committed?
+ */
+ xmin = HeapTupleHeaderGetXmin(htup);
+ if (!prune_freeze_xmin_is_removable(vistest, xmin))
+ {
+ presult->all_visible = false;
+ break;
+ }
+
+ /* Track newest xmin on page. */
+ if (TransactionIdFollows(xmin, presult->frz_conflict_horizon) &&
+ TransactionIdIsNormal(xmin))
+ presult->frz_conflict_horizon = xmin;
+ }
+ break;
+ case HEAPTUPLE_RECENTLY_DEAD:
+ presult->all_visible = false;
+ break;
+ case HEAPTUPLE_INSERT_IN_PROGRESS:
+ presult->all_visible = false;
+ break;
+ case HEAPTUPLE_DELETE_IN_PROGRESS:
+ /* This is an expected case during concurrent vacuum */
+ presult->all_visible = false;
+ break;
+ default:
+ elog(ERROR, "unexpected HeapTupleSatisfiesVacuum result");
+ break;
+ }
}
+ /*
+ * For vacuum, if the whole page will become frozen, we consider
+ * opportunistically freezing tuples. Dead tuples which will be removed by
+ * the end of vacuuming should not preclude us from opportunistically
+ * freezing. We will not be able to freeze the whole page if there are
+ * tuples present which are not visible to everyone or if there are dead
+ * tuples which are not yet removable. We need all_visible to be false if
+ * LP_DEAD tuples remain after pruning so that we do not incorrectly
+ * update the visibility map or page hint bit. So, we will update
+ * presult->all_visible to reflect the presence of LP_DEAD items while
+ * pruning and keep all_visible_except_removable to permit freezing if the
+ * whole page will eventually become all visible after removing tuples.
+ */
+ presult->all_visible_except_removable = presult->all_visible;
+
/* Scan the page */
for (offnum = FirstOffsetNumber;
offnum <= maxoff;
@@ -596,10 +704,14 @@ heap_prune_chain(Buffer buffer, OffsetNumber rootoffnum,
/*
* If the caller set mark_unused_now true, we can set dead line
* pointers LP_UNUSED now. We don't increment ndeleted here since
- * the LP was already marked dead.
+ * the LP was already marked dead. If it will not be marked
+ * LP_UNUSED, it will remain LP_DEAD, making the page not
+ * all_visible.
*/
if (unlikely(prstate->mark_unused_now))
heap_prune_record_unused(prstate, offnum);
+ else
+ presult->all_visible = false;
break;
}
@@ -736,7 +848,7 @@ heap_prune_chain(Buffer buffer, OffsetNumber rootoffnum,
* redirect the root to the correct chain member.
*/
if (i >= nchain)
- heap_prune_record_dead_or_unused(prstate, rootoffnum);
+ heap_prune_record_dead_or_unused(prstate, rootoffnum, presult);
else
heap_prune_record_redirect(prstate, rootoffnum, chainitems[i]);
}
@@ -749,7 +861,7 @@ heap_prune_chain(Buffer buffer, OffsetNumber rootoffnum,
* redirect item. We can clean up by setting the redirect item to
* DEAD state or LP_UNUSED if the caller indicated.
*/
- heap_prune_record_dead_or_unused(prstate, rootoffnum);
+ heap_prune_record_dead_or_unused(prstate, rootoffnum, presult);
}
return ndeleted;
@@ -786,13 +898,20 @@ heap_prune_record_redirect(PruneState *prstate,
/* Record line pointer to be marked dead */
static void
-heap_prune_record_dead(PruneState *prstate, OffsetNumber offnum)
+heap_prune_record_dead(PruneState *prstate, OffsetNumber offnum,
+ PruneResult *presult)
{
Assert(prstate->ndead < MaxHeapTuplesPerPage);
prstate->nowdead[prstate->ndead] = offnum;
prstate->ndead++;
Assert(!prstate->marked[offnum]);
prstate->marked[offnum] = true;
+
+ /*
+ * Setting the line pointer LP_DEAD means the page will definitely not be
+ * all_visible.
+ */
+ presult->all_visible = false;
}
/*
@@ -802,7 +921,8 @@ heap_prune_record_dead(PruneState *prstate, OffsetNumber offnum)
* pointers LP_DEAD if mark_unused_now is true.
*/
static void
-heap_prune_record_dead_or_unused(PruneState *prstate, OffsetNumber offnum)
+heap_prune_record_dead_or_unused(PruneState *prstate, OffsetNumber offnum,
+ PruneResult *presult)
{
/*
* If the caller set mark_unused_now to true, we can remove dead tuples
@@ -813,7 +933,7 @@ heap_prune_record_dead_or_unused(PruneState *prstate, OffsetNumber offnum)
if (unlikely(prstate->mark_unused_now))
heap_prune_record_unused(prstate, offnum);
else
- heap_prune_record_dead(prstate, offnum);
+ heap_prune_record_dead(prstate, offnum, presult);
}
/* Record line pointer to be marked unused */
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index fe31c0125d6..f9892f4cd08 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -1373,20 +1373,6 @@ lazy_scan_new_or_empty(LVRelState *vacrel, Buffer buf, BlockNumber blkno,
return false;
}
-/*
- * Wrap GlobalVisTestIsRemovableXid() to handle FrozenTransactionIds when we
- * are examining tuple xmins to determine if the page is all-visible during
- * pruning. Old tuples may have FrozenTransactionId xmins.
- */
-static inline bool
-prune_freeze_xmin_is_removable(GlobalVisState *visstate, TransactionId xmin)
-{
- if (xmin == FrozenTransactionId)
- return true;
-
- return GlobalVisTestIsRemovableXid(visstate, xmin);
-}
-
/*
* lazy_scan_prune() -- lazy_scan_heap() pruning and freezing.
*
@@ -1436,9 +1422,7 @@ lazy_scan_prune(LVRelState *vacrel,
recently_dead_tuples;
HeapPageFreeze pagefrz;
bool hastup = false;
- bool all_visible,
- all_frozen;
- TransactionId visibility_cutoff_xid;
+ bool all_frozen;
int64 fpi_before = pgWalUsage.wal_fpi;
OffsetNumber deadoffsets[MaxHeapTuplesPerPage];
HeapTupleFreeze frozen[MaxHeapTuplesPerPage];
@@ -1479,17 +1463,16 @@ lazy_scan_prune(LVRelState *vacrel,
&presult, &vacrel->offnum);
/*
- * We will update the VM after collecting LP_DEAD items and freezing
- * tuples. Keep track of whether or not the page is all_visible and
- * all_frozen and use this information to update the VM. all_visible
- * implies 0 lpdead_items, but don't trust all_frozen result unless
- * all_visible is also set to true.
+ * Now scan the page to collect LP_DEAD items and check for tuples
+ * requiring freezing among remaining tuples with storage. We will update
+ * the VM after collecting LP_DEAD items and freezing tuples. Pruning will
+ * have determined whether or not the page is all_visible. Keep track of
+ * whether or not the page is all_frozen and use this information to
+ * update the VM. all_visible implies lpdead_items == 0, but don't trust
+ * all_frozen result unless all_visible is also set to true.
*
- * Also keep track of the visibility cutoff xid for recovery conflicts.
*/
- all_visible = true;
all_frozen = true;
- visibility_cutoff_xid = InvalidTransactionId;
/*
* Now scan the page to collect LP_DEAD items and update the variables set
@@ -1530,11 +1513,6 @@ lazy_scan_prune(LVRelState *vacrel,
* will only happen every other VACUUM, at most. Besides, VACUUM
* must treat hastup/nonempty_pages as provisional no matter how
* LP_DEAD items are handled (handled here, or handled later on).
- *
- * Also deliberately delay unsetting all_visible until just before
- * we return to lazy_scan_heap caller, as explained in full below.
- * (This is another case where it's useful to anticipate that any
- * LP_DEAD items will become LP_UNUSED during the ongoing VACUUM.)
*/
deadoffsets[lpdead_items++] = offnum;
continue;
@@ -1572,41 +1550,6 @@ lazy_scan_prune(LVRelState *vacrel,
* what acquire_sample_rows() does.
*/
live_tuples++;
-
- /*
- * Is the tuple definitely visible to all transactions?
- *
- * NB: Like with per-tuple hint bits, we can't set the
- * PD_ALL_VISIBLE flag if the inserter committed
- * asynchronously. See SetHintBits for more info. Check that
- * the tuple is hinted xmin-committed because of that.
- */
- if (all_visible)
- {
- TransactionId xmin;
-
- if (!HeapTupleHeaderXminCommitted(htup))
- {
- all_visible = false;
- break;
- }
-
- /*
- * The inserter definitely committed. But is it old enough
- * that everyone sees it as committed?
- */
- xmin = HeapTupleHeaderGetXmin(htup);
- if (!prune_freeze_xmin_is_removable(vacrel->vistest, xmin))
- {
- all_visible = false;
- break;
- }
-
- /* Track newest xmin on page. */
- if (TransactionIdFollows(xmin, visibility_cutoff_xid) &&
- TransactionIdIsNormal(xmin))
- visibility_cutoff_xid = xmin;
- }
break;
case HEAPTUPLE_RECENTLY_DEAD:
@@ -1616,7 +1559,6 @@ lazy_scan_prune(LVRelState *vacrel,
* pruning.)
*/
recently_dead_tuples++;
- all_visible = false;
break;
case HEAPTUPLE_INSERT_IN_PROGRESS:
@@ -1627,16 +1569,13 @@ lazy_scan_prune(LVRelState *vacrel,
* results. This assumption is a bit shaky, but it is what
* acquire_sample_rows() does, so be consistent.
*/
- all_visible = false;
break;
case HEAPTUPLE_DELETE_IN_PROGRESS:
- /* This is an expected case during concurrent vacuum */
- all_visible = false;
/*
- * Count such rows as live. As above, we assume the deleting
- * transaction will commit and update the counters after we
- * report.
+ * This an expected case during concurrent vacuum. Count such
+ * rows as live. As above, we assume the deleting transaction
+ * will commit and update the counters after we report.
*/
live_tuples++;
break;
@@ -1679,7 +1618,7 @@ lazy_scan_prune(LVRelState *vacrel,
* page all-frozen afterwards (might not happen until final heap pass).
*/
if (pagefrz.freeze_required || tuples_frozen == 0 ||
- (all_visible && all_frozen &&
+ (presult.all_visible_except_removable && all_frozen &&
fpi_before != pgWalUsage.wal_fpi))
{
/*
@@ -1712,16 +1651,16 @@ lazy_scan_prune(LVRelState *vacrel,
vacrel->frozen_pages++;
/*
- * We can use visibility_cutoff_xid as our cutoff for conflicts
+ * We can use frz_conflict_horizon as our cutoff for conflicts
* when the whole page is eligible to become all-frozen in the VM
* once we're done with it. Otherwise we generate a conservative
* cutoff by stepping back from OldestXmin.
*/
- if (all_visible && all_frozen)
+ if (presult.all_visible_except_removable && all_frozen)
{
/* Using same cutoff when setting VM is now unnecessary */
- snapshotConflictHorizon = visibility_cutoff_xid;
- visibility_cutoff_xid = InvalidTransactionId;
+ snapshotConflictHorizon = presult.frz_conflict_horizon;
+ presult.frz_conflict_horizon = InvalidTransactionId;
}
else
{
@@ -1757,17 +1696,19 @@ lazy_scan_prune(LVRelState *vacrel,
*/
#ifdef USE_ASSERT_CHECKING
/* Note that all_frozen value does not matter when !all_visible */
- if (all_visible && lpdead_items == 0)
+ if (presult.all_visible)
{
TransactionId debug_cutoff;
bool debug_all_frozen;
+ Assert(lpdead_items == 0);
+
if (!heap_page_is_all_visible(vacrel, buf,
&debug_cutoff, &debug_all_frozen))
Assert(false);
Assert(!TransactionIdIsValid(debug_cutoff) ||
- debug_cutoff == visibility_cutoff_xid);
+ debug_cutoff == presult.frz_conflict_horizon);
}
#endif
@@ -1792,19 +1733,6 @@ lazy_scan_prune(LVRelState *vacrel,
Assert(dead_items->num_items <= dead_items->max_items);
pgstat_progress_update_param(PROGRESS_VACUUM_NUM_DEAD_TUPLES,
dead_items->num_items);
-
- /*
- * It was convenient to ignore LP_DEAD items in all_visible earlier on
- * to make the choice of whether or not to freeze the page unaffected
- * by the short-term presence of LP_DEAD items. These LP_DEAD items
- * were effectively assumed to be LP_UNUSED items in the making. It
- * doesn't matter which heap pass (initial pass or final pass) ends up
- * setting the page all-frozen, as long as the ongoing VACUUM does it.
- *
- * Now that freezing has been finalized, unset all_visible. It needs
- * to reflect the present state of things, as expected by our caller.
- */
- all_visible = false;
}
/* Finally, add page-local counts to whole-VACUUM counts */
@@ -1821,20 +1749,20 @@ lazy_scan_prune(LVRelState *vacrel,
/* Did we find LP_DEAD items? */
*has_lpdead_items = (lpdead_items > 0);
- Assert(!all_visible || !(*has_lpdead_items));
+ Assert(!presult.all_visible || !(*has_lpdead_items));
/*
* Handle setting visibility map bit based on information from the VM (as
* of last heap_vac_scan_next_block() call), and from all_visible and
* all_frozen variables
*/
- if (!all_visible_according_to_vm && all_visible)
+ if (!all_visible_according_to_vm && presult.all_visible)
{
uint8 flags = VISIBILITYMAP_ALL_VISIBLE;
if (all_frozen)
{
- Assert(!TransactionIdIsValid(visibility_cutoff_xid));
+ Assert(!TransactionIdIsValid(presult.frz_conflict_horizon));
flags |= VISIBILITYMAP_ALL_FROZEN;
}
@@ -1854,7 +1782,7 @@ lazy_scan_prune(LVRelState *vacrel,
PageSetAllVisible(page);
MarkBufferDirty(buf);
visibilitymap_set(vacrel->rel, blkno, buf, InvalidXLogRecPtr,
- vmbuffer, visibility_cutoff_xid,
+ vmbuffer, presult.frz_conflict_horizon,
flags);
}
@@ -1902,7 +1830,7 @@ lazy_scan_prune(LVRelState *vacrel,
* it as all-frozen. Note that all_frozen is only valid if all_visible is
* true, so we must check both all_visible and all_frozen.
*/
- else if (all_visible_according_to_vm && all_visible &&
+ else if (all_visible_according_to_vm && presult.all_visible &&
all_frozen && !VM_ALL_FROZEN(vacrel->rel, blkno, &vmbuffer))
{
/*
@@ -1919,11 +1847,11 @@ lazy_scan_prune(LVRelState *vacrel,
/*
* Set the page all-frozen (and all-visible) in the VM.
*
- * We can pass InvalidTransactionId as our visibility_cutoff_xid,
- * since a snapshotConflictHorizon sufficient to make everything safe
- * for REDO was logged when the page's tuples were frozen.
+ * We can pass InvalidTransactionId as our frz_conflict_horizon, since
+ * a snapshotConflictHorizon sufficient to make everything safe for
+ * REDO was logged when the page's tuples were frozen.
*/
- Assert(!TransactionIdIsValid(visibility_cutoff_xid));
+ Assert(!TransactionIdIsValid(presult.frz_conflict_horizon));
visibilitymap_set(vacrel->rel, blkno, buf, InvalidXLogRecPtr,
vmbuffer, InvalidTransactionId,
VISIBILITYMAP_ALL_VISIBLE |
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 4b133f68593..d8e65ae7e35 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -198,6 +198,9 @@ typedef struct PruneResult
{
int ndeleted; /* Number of tuples deleted from the page */
int nnewlpdead; /* Number of newly LP_DEAD items */
+ bool all_visible; /* Whether or not the page is all visible */
+ bool all_visible_except_removable;
+ TransactionId frz_conflict_horizon; /* Newest xmin on the page */
/*
* Tuple visibility is only computed once for each tuple, for correctness
--
2.40.1
[text/x-diff] v3-0004-Add-reference-to-VacuumCutoffs-in-HeapPageFreeze.patch (12.4K, ../20240315005658.y7bvbzvnlciqmhd6@liskov/5-v3-0004-Add-reference-to-VacuumCutoffs-in-HeapPageFreeze.patch)
download | inline diff:
From 317b479b009c13836b28e289a1782ed6f865b732 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Sat, 6 Jan 2024 16:22:17 -0500
Subject: [PATCH v3 04/17] Add reference to VacuumCutoffs in HeapPageFreeze
Future commits will move opportunistic freezing into the main path of
pruning in heap_page_prune(). Because on-access pruning will not do
opportunistic freezing, it is cleaner to keep the visibility information
required for calling heap_prepare_freeze_tuple() inside of the
HeapPageFreeze structure itself by saving a reference to VacuumCutoffs.
---
src/backend/access/heap/heapam.c | 67 ++++++++++++++--------------
src/backend/access/heap/vacuumlazy.c | 3 +-
src/include/access/heapam.h | 2 +-
3 files changed, 36 insertions(+), 36 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 34bc60f625f..7261c4988d7 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -6023,7 +6023,7 @@ heap_inplace_update(Relation relation, HeapTuple tuple)
*/
static TransactionId
FreezeMultiXactId(MultiXactId multi, uint16 t_infomask,
- const struct VacuumCutoffs *cutoffs, uint16 *flags,
+ uint16 *flags,
HeapPageFreeze *pagefrz)
{
TransactionId newxmax;
@@ -6049,12 +6049,12 @@ FreezeMultiXactId(MultiXactId multi, uint16 t_infomask,
pagefrz->freeze_required = true;
return InvalidTransactionId;
}
- else if (MultiXactIdPrecedes(multi, cutoffs->relminmxid))
+ else if (MultiXactIdPrecedes(multi, pagefrz->cutoffs->relminmxid))
ereport(ERROR,
(errcode(ERRCODE_DATA_CORRUPTED),
errmsg_internal("found multixact %u from before relminmxid %u",
- multi, cutoffs->relminmxid)));
- else if (MultiXactIdPrecedes(multi, cutoffs->OldestMxact))
+ multi, pagefrz->cutoffs->relminmxid)));
+ else if (MultiXactIdPrecedes(multi, pagefrz->cutoffs->OldestMxact))
{
TransactionId update_xact;
@@ -6069,7 +6069,7 @@ FreezeMultiXactId(MultiXactId multi, uint16 t_infomask,
ereport(ERROR,
(errcode(ERRCODE_DATA_CORRUPTED),
errmsg_internal("multixact %u from before multi freeze cutoff %u found to be still running",
- multi, cutoffs->OldestMxact)));
+ multi, pagefrz->cutoffs->OldestMxact)));
if (HEAP_XMAX_IS_LOCKED_ONLY(t_infomask))
{
@@ -6080,13 +6080,13 @@ FreezeMultiXactId(MultiXactId multi, uint16 t_infomask,
/* replace multi with single XID for its updater? */
update_xact = MultiXactIdGetUpdateXid(multi, t_infomask);
- if (TransactionIdPrecedes(update_xact, cutoffs->relfrozenxid))
+ if (TransactionIdPrecedes(update_xact, pagefrz->cutoffs->relfrozenxid))
ereport(ERROR,
(errcode(ERRCODE_DATA_CORRUPTED),
errmsg_internal("multixact %u contains update XID %u from before relfrozenxid %u",
multi, update_xact,
- cutoffs->relfrozenxid)));
- else if (TransactionIdPrecedes(update_xact, cutoffs->OldestXmin))
+ pagefrz->cutoffs->relfrozenxid)));
+ else if (TransactionIdPrecedes(update_xact, pagefrz->cutoffs->OldestXmin))
{
/*
* Updater XID has to have aborted (otherwise the tuple would have
@@ -6098,7 +6098,7 @@ FreezeMultiXactId(MultiXactId multi, uint16 t_infomask,
(errcode(ERRCODE_DATA_CORRUPTED),
errmsg_internal("multixact %u contains committed update XID %u from before removable cutoff %u",
multi, update_xact,
- cutoffs->OldestXmin)));
+ pagefrz->cutoffs->OldestXmin)));
*flags |= FRM_INVALIDATE_XMAX;
pagefrz->freeze_required = true;
return InvalidTransactionId;
@@ -6150,9 +6150,9 @@ FreezeMultiXactId(MultiXactId multi, uint16 t_infomask,
{
TransactionId xid = members[i].xid;
- Assert(!TransactionIdPrecedes(xid, cutoffs->relfrozenxid));
+ Assert(!TransactionIdPrecedes(xid, pagefrz->cutoffs->relfrozenxid));
- if (TransactionIdPrecedes(xid, cutoffs->FreezeLimit))
+ if (TransactionIdPrecedes(xid, pagefrz->cutoffs->FreezeLimit))
{
/* Can't violate the FreezeLimit postcondition */
need_replace = true;
@@ -6164,7 +6164,7 @@ FreezeMultiXactId(MultiXactId multi, uint16 t_infomask,
/* Can't violate the MultiXactCutoff postcondition, either */
if (!need_replace)
- need_replace = MultiXactIdPrecedes(multi, cutoffs->MultiXactCutoff);
+ need_replace = MultiXactIdPrecedes(multi, pagefrz->cutoffs->MultiXactCutoff);
if (!need_replace)
{
@@ -6203,7 +6203,7 @@ FreezeMultiXactId(MultiXactId multi, uint16 t_infomask,
TransactionId xid = members[i].xid;
MultiXactStatus mstatus = members[i].status;
- Assert(!TransactionIdPrecedes(xid, cutoffs->relfrozenxid));
+ Assert(!TransactionIdPrecedes(xid, pagefrz->cutoffs->relfrozenxid));
if (!ISUPDATE_from_mxstatus(mstatus))
{
@@ -6214,12 +6214,12 @@ FreezeMultiXactId(MultiXactId multi, uint16 t_infomask,
if (TransactionIdIsCurrentTransactionId(xid) ||
TransactionIdIsInProgress(xid))
{
- if (TransactionIdPrecedes(xid, cutoffs->OldestXmin))
+ if (TransactionIdPrecedes(xid, pagefrz->cutoffs->OldestXmin))
ereport(ERROR,
(errcode(ERRCODE_DATA_CORRUPTED),
errmsg_internal("multixact %u contains running locker XID %u from before removable cutoff %u",
multi, xid,
- cutoffs->OldestXmin)));
+ pagefrz->cutoffs->OldestXmin)));
newmembers[nnewmembers++] = members[i];
has_lockers = true;
}
@@ -6277,11 +6277,11 @@ FreezeMultiXactId(MultiXactId multi, uint16 t_infomask,
* We determined that updater must be kept -- add it to pending new
* members list
*/
- if (TransactionIdPrecedes(xid, cutoffs->OldestXmin))
+ if (TransactionIdPrecedes(xid, pagefrz->cutoffs->OldestXmin))
ereport(ERROR,
(errcode(ERRCODE_DATA_CORRUPTED),
errmsg_internal("multixact %u contains committed update XID %u from before removable cutoff %u",
- multi, xid, cutoffs->OldestXmin)));
+ multi, xid, pagefrz->cutoffs->OldestXmin)));
newmembers[nnewmembers++] = members[i];
}
@@ -6373,7 +6373,6 @@ FreezeMultiXactId(MultiXactId multi, uint16 t_infomask,
*/
bool
heap_prepare_freeze_tuple(HeapTupleHeader tuple,
- const struct VacuumCutoffs *cutoffs,
HeapPageFreeze *pagefrz,
HeapTupleFreeze *frz, bool *totally_frozen)
{
@@ -6401,14 +6400,14 @@ heap_prepare_freeze_tuple(HeapTupleHeader tuple,
xmin_already_frozen = true;
else
{
- if (TransactionIdPrecedes(xid, cutoffs->relfrozenxid))
+ if (TransactionIdPrecedes(xid, pagefrz->cutoffs->relfrozenxid))
ereport(ERROR,
(errcode(ERRCODE_DATA_CORRUPTED),
errmsg_internal("found xmin %u from before relfrozenxid %u",
- xid, cutoffs->relfrozenxid)));
+ xid, pagefrz->cutoffs->relfrozenxid)));
/* Will set freeze_xmin flags in freeze plan below */
- freeze_xmin = TransactionIdPrecedes(xid, cutoffs->OldestXmin);
+ freeze_xmin = TransactionIdPrecedes(xid, pagefrz->cutoffs->OldestXmin);
/* Verify that xmin committed if and when freeze plan is executed */
if (freeze_xmin)
@@ -6422,8 +6421,8 @@ heap_prepare_freeze_tuple(HeapTupleHeader tuple,
xid = HeapTupleHeaderGetXvac(tuple);
if (TransactionIdIsNormal(xid))
{
- Assert(TransactionIdPrecedesOrEquals(cutoffs->relfrozenxid, xid));
- Assert(TransactionIdPrecedes(xid, cutoffs->OldestXmin));
+ Assert(TransactionIdPrecedesOrEquals(pagefrz->cutoffs->relfrozenxid, xid));
+ Assert(TransactionIdPrecedes(xid, pagefrz->cutoffs->OldestXmin));
/*
* For Xvac, we always freeze proactively. This allows totally_frozen
@@ -6448,8 +6447,7 @@ heap_prepare_freeze_tuple(HeapTupleHeader tuple,
* perform no-op xmax processing. The only constraint is that the
* FreezeLimit/MultiXactCutoff postcondition must never be violated.
*/
- newxmax = FreezeMultiXactId(xid, tuple->t_infomask, cutoffs,
- &flags, pagefrz);
+ newxmax = FreezeMultiXactId(xid, tuple->t_infomask, &flags, pagefrz);
if (flags & FRM_NOOP)
{
@@ -6472,7 +6470,7 @@ heap_prepare_freeze_tuple(HeapTupleHeader tuple,
* (This repeats work from FreezeMultiXactId, but allows "no
* freeze" tracker maintenance to happen in only one place.)
*/
- Assert(!MultiXactIdPrecedes(newxmax, cutoffs->MultiXactCutoff));
+ Assert(!MultiXactIdPrecedes(newxmax, pagefrz->cutoffs->MultiXactCutoff));
Assert(MultiXactIdIsValid(newxmax) && xid == newxmax);
}
else if (flags & FRM_RETURN_IS_XID)
@@ -6481,7 +6479,7 @@ heap_prepare_freeze_tuple(HeapTupleHeader tuple,
* xmax will become an updater Xid (original MultiXact's updater
* member Xid will be carried forward as a simple Xid in Xmax).
*/
- Assert(!TransactionIdPrecedes(newxmax, cutoffs->OldestXmin));
+ Assert(!TransactionIdPrecedes(newxmax, pagefrz->cutoffs->OldestXmin));
/*
* NB -- some of these transformations are only valid because we
@@ -6505,7 +6503,7 @@ heap_prepare_freeze_tuple(HeapTupleHeader tuple,
* xmax is an old MultiXactId that we have to replace with a new
* MultiXactId, to carry forward two or more original member XIDs.
*/
- Assert(!MultiXactIdPrecedes(newxmax, cutoffs->OldestMxact));
+ Assert(!MultiXactIdPrecedes(newxmax, pagefrz->cutoffs->OldestMxact));
/*
* We can't use GetMultiXactIdHintBits directly on the new multi
@@ -6540,14 +6538,14 @@ heap_prepare_freeze_tuple(HeapTupleHeader tuple,
else if (TransactionIdIsNormal(xid))
{
/* Raw xmax is normal XID */
- if (TransactionIdPrecedes(xid, cutoffs->relfrozenxid))
+ if (TransactionIdPrecedes(xid, pagefrz->cutoffs->relfrozenxid))
ereport(ERROR,
(errcode(ERRCODE_DATA_CORRUPTED),
errmsg_internal("found xmax %u from before relfrozenxid %u",
- xid, cutoffs->relfrozenxid)));
+ xid, pagefrz->cutoffs->relfrozenxid)));
/* Will set freeze_xmax flags in freeze plan below */
- freeze_xmax = TransactionIdPrecedes(xid, cutoffs->OldestXmin);
+ freeze_xmax = TransactionIdPrecedes(xid, pagefrz->cutoffs->OldestXmin);
/*
* Verify that xmax aborted if and when freeze plan is executed,
@@ -6627,7 +6625,7 @@ heap_prepare_freeze_tuple(HeapTupleHeader tuple,
* Does this tuple force caller to freeze the entire page?
*/
pagefrz->freeze_required =
- heap_tuple_should_freeze(tuple, cutoffs,
+ heap_tuple_should_freeze(tuple, pagefrz->cutoffs,
&pagefrz->NoFreezePageRelfrozenXid,
&pagefrz->NoFreezePageRelminMxid);
}
@@ -6949,8 +6947,9 @@ heap_freeze_tuple(HeapTupleHeader tuple,
pagefrz.NoFreezePageRelfrozenXid = FreezeLimit;
pagefrz.NoFreezePageRelminMxid = MultiXactCutoff;
- do_freeze = heap_prepare_freeze_tuple(tuple, &cutoffs,
- &pagefrz, &frz, &totally_frozen);
+ pagefrz.cutoffs = &cutoffs;
+
+ do_freeze = heap_prepare_freeze_tuple(tuple, &pagefrz, &frz, &totally_frozen);
/*
* Note that because this is not a WAL-logged operation, we don't need to
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index f9892f4cd08..06e0e841582 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -1442,6 +1442,7 @@ lazy_scan_prune(LVRelState *vacrel,
pagefrz.FreezePageRelminMxid = vacrel->NewRelminMxid;
pagefrz.NoFreezePageRelfrozenXid = vacrel->NewRelfrozenXid;
pagefrz.NoFreezePageRelminMxid = vacrel->NewRelminMxid;
+ pagefrz.cutoffs = &vacrel->cutoffs;
tuples_frozen = 0;
lpdead_items = 0;
live_tuples = 0;
@@ -1587,7 +1588,7 @@ lazy_scan_prune(LVRelState *vacrel,
hastup = true; /* page makes rel truncation unsafe */
/* Tuple with storage -- consider need to freeze */
- if (heap_prepare_freeze_tuple(htup, &vacrel->cutoffs, &pagefrz,
+ if (heap_prepare_freeze_tuple(htup, &pagefrz,
&frozen[tuples_frozen], &totally_frozen))
{
/* Save prepared freeze plan for later */
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index d8e65ae7e35..297ba03bf09 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -189,6 +189,7 @@ typedef struct HeapPageFreeze
TransactionId NoFreezePageRelfrozenXid;
MultiXactId NoFreezePageRelminMxid;
+ struct VacuumCutoffs *cutoffs;
} HeapPageFreeze;
/*
@@ -295,7 +296,6 @@ extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
extern void heap_inplace_update(Relation relation, HeapTuple tuple);
extern bool heap_prepare_freeze_tuple(HeapTupleHeader tuple,
- const struct VacuumCutoffs *cutoffs,
HeapPageFreeze *pagefrz,
HeapTupleFreeze *frz, bool *totally_frozen);
extern void heap_freeze_execute_prepared(Relation rel, Buffer buffer,
--
2.40.1
[text/x-diff] v3-0005-Prepare-freeze-tuples-in-heap_page_prune.patch (13.6K, ../20240315005658.y7bvbzvnlciqmhd6@liskov/6-v3-0005-Prepare-freeze-tuples-in-heap_page_prune.patch)
download | inline diff:
From c9f8a6a8fa06c62a738a8597f7fa0186719e3e0b Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Sun, 7 Jan 2024 11:18:52 -0500
Subject: [PATCH v3 05/17] Prepare freeze tuples in heap_page_prune()
In order to combine the freeze and prune records, we must determine
which tuples are freezable before actually executing pruning. All of the
page modifications should be made in the same critical section along
with emitting the combined WAL. Determine whether or not tuples should
or must be frozen and whether or not the page will be all frozen as a
consequence during pruning.
---
src/backend/access/heap/pruneheap.c | 78 ++++++++++++++++++++++++++--
src/backend/access/heap/vacuumlazy.c | 68 ++++++------------------
src/include/access/heapam.h | 12 +++++
3 files changed, 101 insertions(+), 57 deletions(-)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index 42fd4a74845..6bd8400b33b 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -62,6 +62,9 @@ static HTSV_Result heap_prune_satisfies_vacuum(PruneState *prstate,
static int heap_prune_chain(Buffer buffer,
OffsetNumber rootoffnum,
PruneState *prstate, PruneResult *presult);
+
+static void prune_prepare_freeze_tuple(Page page, OffsetNumber offnum,
+ HeapPageFreeze *pagefrz, PruneResult *presult);
static void heap_prune_record_prunable(PruneState *prstate, TransactionId xid);
static void heap_prune_record_redirect(PruneState *prstate,
OffsetNumber offnum, OffsetNumber rdoffnum);
@@ -155,7 +158,7 @@ heap_page_prune_opt(Relation relation, Buffer buffer)
* not the relation has indexes, since we cannot safely determine
* that during on-access pruning with the current implementation.
*/
- heap_page_prune(relation, buffer, vistest, false,
+ heap_page_prune(relation, buffer, vistest, false, NULL,
&presult, NULL);
/*
@@ -218,6 +221,9 @@ prune_freeze_xmin_is_removable(GlobalVisState *visstate, TransactionId xmin)
* mark_unused_now indicates whether or not dead items can be set LP_UNUSED during
* pruning.
*
+ * pagefrz contains both input and output parameters used if the caller is
+ * interested in potentially freezing tuples on the page.
+ *
* off_loc is the offset location required by the caller to use in error
* callback.
*
@@ -229,6 +235,7 @@ void
heap_page_prune(Relation relation, Buffer buffer,
GlobalVisState *vistest,
bool mark_unused_now,
+ HeapPageFreeze *pagefrz,
PruneResult *presult,
OffsetNumber *off_loc)
{
@@ -264,6 +271,7 @@ heap_page_prune(Relation relation, Buffer buffer,
*/
presult->ndeleted = 0;
presult->nnewlpdead = 0;
+ presult->nfrozen = 0;
/*
* Keep track of whether or not the page is all_visible in case the caller
@@ -410,6 +418,15 @@ heap_page_prune(Relation relation, Buffer buffer,
*/
presult->all_visible_except_removable = presult->all_visible;
+ /*
+ * We will update the VM after pruning, collecting LP_DEAD items, and
+ * freezing tuples. Keep track of whether or not the page is all_visible
+ * and all_frozen and use this information to update the VM. all_visible
+ * implies lpdead_items == 0, but don't trust all_frozen result unless
+ * all_visible is also set to true.
+ */
+ presult->all_frozen = true;
+
/* Scan the page */
for (offnum = FirstOffsetNumber;
offnum <= maxoff;
@@ -417,14 +434,18 @@ heap_page_prune(Relation relation, Buffer buffer,
{
ItemId itemid;
- /* Ignore items already processed as part of an earlier chain */
- if (prstate.marked[offnum])
- continue;
-
/* see preceding loop */
if (off_loc)
*off_loc = offnum;
+ if (pagefrz)
+ prune_prepare_freeze_tuple(page, offnum,
+ pagefrz, presult);
+
+ /* Ignore items already processed as part of an earlier chain */
+ if (prstate.marked[offnum])
+ continue;
+
/* Nothing to do if slot is empty */
itemid = PageGetItemId(page, offnum);
if (!ItemIdIsUsed(itemid))
@@ -867,6 +888,53 @@ heap_prune_chain(Buffer buffer, OffsetNumber rootoffnum,
return ndeleted;
}
+/*
+ * While pruning, before actually executing pruning and updating the line
+ * pointers, we may consider freezing tuples referred to by LP_NORMAL line
+ * pointers whose visibility status is not HEAPTUPLE_DEAD. That is to say, we
+ * want to consider freezing normal tuples which will not be removed.
+*/
+static void
+prune_prepare_freeze_tuple(Page page, OffsetNumber offnum,
+ HeapPageFreeze *pagefrz,
+ PruneResult *presult)
+{
+ bool totally_frozen;
+ HeapTupleHeader htup;
+ ItemId itemid;
+
+ Assert(pagefrz);
+
+ itemid = PageGetItemId(page, offnum);
+
+ if (!ItemIdIsNormal(itemid))
+ return;
+
+ /* We do not consider freezing tuples which will be removed. */
+ if (presult->htsv[offnum] == HEAPTUPLE_DEAD ||
+ presult->htsv[offnum] == -1)
+ return;
+
+ htup = (HeapTupleHeader) PageGetItem(page, itemid);
+
+ /* Tuple with storage -- consider need to freeze */
+ if ((heap_prepare_freeze_tuple(htup, pagefrz,
+ &presult->frozen[presult->nfrozen],
+ &totally_frozen)))
+ {
+ /* Save prepared freeze plan for later */
+ presult->frozen[presult->nfrozen++].offset = offnum;
+ }
+
+ /*
+ * If any tuple isn't either totally frozen already or eligible to become
+ * totally frozen (according to its freeze plan), then the page definitely
+ * cannot be set all-frozen in the visibility map later on
+ */
+ if (!totally_frozen)
+ presult->all_frozen = false;
+}
+
/* Record lowest soon-prunable XID */
static void
heap_prune_record_prunable(PruneState *prstate, TransactionId xid)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 06e0e841582..4187c998d25 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -1416,16 +1416,13 @@ lazy_scan_prune(LVRelState *vacrel,
maxoff;
ItemId itemid;
PruneResult presult;
- int tuples_frozen,
- lpdead_items,
+ int lpdead_items,
live_tuples,
recently_dead_tuples;
HeapPageFreeze pagefrz;
bool hastup = false;
- bool all_frozen;
int64 fpi_before = pgWalUsage.wal_fpi;
OffsetNumber deadoffsets[MaxHeapTuplesPerPage];
- HeapTupleFreeze frozen[MaxHeapTuplesPerPage];
Assert(BufferGetBlockNumber(buf) == blkno);
@@ -1443,7 +1440,6 @@ lazy_scan_prune(LVRelState *vacrel,
pagefrz.NoFreezePageRelfrozenXid = vacrel->NewRelfrozenXid;
pagefrz.NoFreezePageRelminMxid = vacrel->NewRelminMxid;
pagefrz.cutoffs = &vacrel->cutoffs;
- tuples_frozen = 0;
lpdead_items = 0;
live_tuples = 0;
recently_dead_tuples = 0;
@@ -1461,31 +1457,20 @@ lazy_scan_prune(LVRelState *vacrel,
* false otherwise.
*/
heap_page_prune(rel, buf, vacrel->vistest, vacrel->nindexes == 0,
- &presult, &vacrel->offnum);
+ &pagefrz, &presult, &vacrel->offnum);
/*
* Now scan the page to collect LP_DEAD items and check for tuples
* requiring freezing among remaining tuples with storage. We will update
* the VM after collecting LP_DEAD items and freezing tuples. Pruning will
- * have determined whether or not the page is all_visible. Keep track of
- * whether or not the page is all_frozen and use this information to
- * update the VM. all_visible implies lpdead_items == 0, but don't trust
- * all_frozen result unless all_visible is also set to true.
+ * have determined whether or not the page is all_visible and able to
+ * become all_frozen.
*
*/
- all_frozen = true;
-
- /*
- * Now scan the page to collect LP_DEAD items and update the variables set
- * just above.
- */
for (offnum = FirstOffsetNumber;
offnum <= maxoff;
offnum = OffsetNumberNext(offnum))
{
- HeapTupleHeader htup;
- bool totally_frozen;
-
/*
* Set the offset number so that we can display it along with any
* error that occurred while processing this tuple.
@@ -1521,8 +1506,6 @@ lazy_scan_prune(LVRelState *vacrel,
Assert(ItemIdIsNormal(itemid));
- htup = (HeapTupleHeader) PageGetItem(page, itemid);
-
/*
* The criteria for counting a tuple as live in this block need to
* match what analyze.c's acquire_sample_rows() does, otherwise VACUUM
@@ -1587,29 +1570,8 @@ lazy_scan_prune(LVRelState *vacrel,
hastup = true; /* page makes rel truncation unsafe */
- /* Tuple with storage -- consider need to freeze */
- if (heap_prepare_freeze_tuple(htup, &pagefrz,
- &frozen[tuples_frozen], &totally_frozen))
- {
- /* Save prepared freeze plan for later */
- frozen[tuples_frozen++].offset = offnum;
- }
-
- /*
- * If any tuple isn't either totally frozen already or eligible to
- * become totally frozen (according to its freeze plan), then the page
- * definitely cannot be set all-frozen in the visibility map later on
- */
- if (!totally_frozen)
- all_frozen = false;
}
- /*
- * We have now divided every item on the page into either an LP_DEAD item
- * that will need to be vacuumed in indexes later, or a LP_NORMAL tuple
- * that remains and needs to be considered for freezing now (LP_UNUSED and
- * LP_REDIRECT items also remain, but are of no further interest to us).
- */
vacrel->offnum = InvalidOffsetNumber;
/*
@@ -1618,8 +1580,8 @@ lazy_scan_prune(LVRelState *vacrel,
* freeze when pruning generated an FPI, if doing so means that we set the
* page all-frozen afterwards (might not happen until final heap pass).
*/
- if (pagefrz.freeze_required || tuples_frozen == 0 ||
- (presult.all_visible_except_removable && all_frozen &&
+ if (pagefrz.freeze_required || presult.nfrozen == 0 ||
+ (presult.all_visible_except_removable && presult.all_frozen &&
fpi_before != pgWalUsage.wal_fpi))
{
/*
@@ -1629,7 +1591,7 @@ lazy_scan_prune(LVRelState *vacrel,
vacrel->NewRelfrozenXid = pagefrz.FreezePageRelfrozenXid;
vacrel->NewRelminMxid = pagefrz.FreezePageRelminMxid;
- if (tuples_frozen == 0)
+ if (presult.nfrozen == 0)
{
/*
* We have no freeze plans to execute, so there's no added cost
@@ -1657,7 +1619,7 @@ lazy_scan_prune(LVRelState *vacrel,
* once we're done with it. Otherwise we generate a conservative
* cutoff by stepping back from OldestXmin.
*/
- if (presult.all_visible_except_removable && all_frozen)
+ if (presult.all_visible_except_removable && presult.all_frozen)
{
/* Using same cutoff when setting VM is now unnecessary */
snapshotConflictHorizon = presult.frz_conflict_horizon;
@@ -1673,7 +1635,7 @@ lazy_scan_prune(LVRelState *vacrel,
/* Execute all freeze plans for page as a single atomic action */
heap_freeze_execute_prepared(vacrel->rel, buf,
snapshotConflictHorizon,
- frozen, tuples_frozen);
+ presult.frozen, presult.nfrozen);
}
}
else
@@ -1684,8 +1646,8 @@ lazy_scan_prune(LVRelState *vacrel,
*/
vacrel->NewRelfrozenXid = pagefrz.NoFreezePageRelfrozenXid;
vacrel->NewRelminMxid = pagefrz.NoFreezePageRelminMxid;
- all_frozen = false;
- tuples_frozen = 0; /* avoid miscounts in instrumentation */
+ presult.all_frozen = false;
+ presult.nfrozen = 0; /* avoid miscounts in instrumentation */
}
/*
@@ -1708,6 +1670,8 @@ lazy_scan_prune(LVRelState *vacrel,
&debug_cutoff, &debug_all_frozen))
Assert(false);
+ Assert(presult.all_frozen == debug_all_frozen);
+
Assert(!TransactionIdIsValid(debug_cutoff) ||
debug_cutoff == presult.frz_conflict_horizon);
}
@@ -1738,7 +1702,7 @@ lazy_scan_prune(LVRelState *vacrel,
/* Finally, add page-local counts to whole-VACUUM counts */
vacrel->tuples_deleted += presult.ndeleted;
- vacrel->tuples_frozen += tuples_frozen;
+ vacrel->tuples_frozen += presult.nfrozen;
vacrel->lpdead_items += lpdead_items;
vacrel->live_tuples += live_tuples;
vacrel->recently_dead_tuples += recently_dead_tuples;
@@ -1761,7 +1725,7 @@ lazy_scan_prune(LVRelState *vacrel,
{
uint8 flags = VISIBILITYMAP_ALL_VISIBLE;
- if (all_frozen)
+ if (presult.all_frozen)
{
Assert(!TransactionIdIsValid(presult.frz_conflict_horizon));
flags |= VISIBILITYMAP_ALL_FROZEN;
@@ -1832,7 +1796,7 @@ lazy_scan_prune(LVRelState *vacrel,
* true, so we must check both all_visible and all_frozen.
*/
else if (all_visible_according_to_vm && presult.all_visible &&
- all_frozen && !VM_ALL_FROZEN(vacrel->rel, blkno, &vmbuffer))
+ presult.all_frozen && !VM_ALL_FROZEN(vacrel->rel, blkno, &vmbuffer))
{
/*
* Avoid relying on all_visible_according_to_vm as a proxy for the
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 297ba03bf09..2339abfd28a 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -201,6 +201,11 @@ typedef struct PruneResult
int nnewlpdead; /* Number of newly LP_DEAD items */
bool all_visible; /* Whether or not the page is all visible */
bool all_visible_except_removable;
+ /* Whether or not the page can be set all frozen in the VM */
+ bool all_frozen;
+
+ /* Number of newly frozen tuples */
+ int nfrozen;
TransactionId frz_conflict_horizon; /* Newest xmin on the page */
/*
@@ -213,6 +218,12 @@ typedef struct PruneResult
* 1. Otherwise every access would need to subtract 1.
*/
int8 htsv[MaxHeapTuplesPerPage + 1];
+
+
+ /*
+ * One entry for every tuple that we may freeze.
+ */
+ HeapTupleFreeze frozen[MaxHeapTuplesPerPage];
} PruneResult;
/*
@@ -324,6 +335,7 @@ extern void heap_page_prune_opt(Relation relation, Buffer buffer);
extern void heap_page_prune(Relation relation, Buffer buffer,
struct GlobalVisState *vistest,
bool mark_unused_now,
+ HeapPageFreeze *pagefrz,
PruneResult *presult,
OffsetNumber *off_loc);
extern void heap_page_prune_execute(Buffer buffer,
--
2.40.1
[text/x-diff] v3-0006-lazy_scan_prune-reorder-freeze-execution-logic.patch (6.9K, ../20240315005658.y7bvbzvnlciqmhd6@liskov/7-v3-0006-lazy_scan_prune-reorder-freeze-execution-logic.patch)
download | inline diff:
From 1bb0bdd4e1337fe95b34bedaac255285144a3329 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Sun, 7 Jan 2024 14:50:12 -0500
Subject: [PATCH v3 06/17] lazy_scan_prune reorder freeze execution logic
To combine the prune and freeze records, freezing must be done before a
pruning WAL record is emitted. We will move the freeze execution into
heap_page_prune() in future commits. lazy_scan_prune() currently
executes freezing, updates vacrel->NewRelfrozenXid and
vacrel->NewRelminMxid, and resets the snapshotConflictHorizon that the
visibility map update record may use in the same block of if statements.
This commit starts reordering that logic so that the freeze execution
can be separated from the other updates which should not be done in
pruning. It also adds a helper calculating freeze snapshot conflict
horizon. This will be useful when the freeze execution is moved into
pruning because not all callers of heap_page_prune() have access to
VacuumCutoffs.
---
src/backend/access/heap/vacuumlazy.c | 112 ++++++++++++++++-----------
1 file changed, 67 insertions(+), 45 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 4187c998d25..abbb7ab3ada 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -269,6 +269,8 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+static TransactionId heap_frz_conflict_horizon(PruneResult *presult,
+ HeapPageFreeze *pagefrz);
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
@@ -1373,6 +1375,33 @@ lazy_scan_new_or_empty(LVRelState *vacrel, Buffer buf, BlockNumber blkno,
return false;
}
+/*
+ * Determine the snapshotConflictHorizon for freezing. Must only be called
+ * after pruning and determining if the page is freezable.
+ */
+static TransactionId
+heap_frz_conflict_horizon(PruneResult *presult, HeapPageFreeze *pagefrz)
+{
+ TransactionId result;
+
+ /*
+ * We can use frz_conflict_horizon as our cutoff for conflicts when the
+ * whole page is eligible to become all-frozen in the VM once we're done
+ * with it. Otherwise we generate a conservative cutoff by stepping back
+ * from OldestXmin.
+ */
+ if (presult->all_visible_except_removable && presult->all_frozen)
+ result = presult->frz_conflict_horizon;
+ else
+ {
+ /* Avoids false conflicts when hot_standby_feedback in use */
+ result = pagefrz->cutoffs->OldestXmin;
+ TransactionIdRetreat(result);
+ }
+
+ return result;
+}
+
/*
* lazy_scan_prune() -- lazy_scan_heap() pruning and freezing.
*
@@ -1421,6 +1450,7 @@ lazy_scan_prune(LVRelState *vacrel,
recently_dead_tuples;
HeapPageFreeze pagefrz;
bool hastup = false;
+ bool do_freeze;
int64 fpi_before = pgWalUsage.wal_fpi;
OffsetNumber deadoffsets[MaxHeapTuplesPerPage];
@@ -1580,10 +1610,15 @@ lazy_scan_prune(LVRelState *vacrel,
* freeze when pruning generated an FPI, if doing so means that we set the
* page all-frozen afterwards (might not happen until final heap pass).
*/
- if (pagefrz.freeze_required || presult.nfrozen == 0 ||
+ do_freeze = pagefrz.freeze_required ||
(presult.all_visible_except_removable && presult.all_frozen &&
- fpi_before != pgWalUsage.wal_fpi))
+ presult.nfrozen > 0 &&
+ fpi_before != pgWalUsage.wal_fpi);
+
+ if (do_freeze)
{
+ TransactionId snapshotConflictHorizon;
+
/*
* We're freezing the page. Our final NewRelfrozenXid doesn't need to
* be affected by the XIDs that are just about to be frozen anyway.
@@ -1591,52 +1626,39 @@ lazy_scan_prune(LVRelState *vacrel,
vacrel->NewRelfrozenXid = pagefrz.FreezePageRelfrozenXid;
vacrel->NewRelminMxid = pagefrz.FreezePageRelminMxid;
- if (presult.nfrozen == 0)
- {
- /*
- * We have no freeze plans to execute, so there's no added cost
- * from following the freeze path. That's why it was chosen. This
- * is important in the case where the page only contains totally
- * frozen tuples at this point (perhaps only following pruning).
- * Such pages can be marked all-frozen in the VM by our caller,
- * even though none of its tuples were newly frozen here (note
- * that the "no freeze" path never sets pages all-frozen).
- *
- * We never increment the frozen_pages instrumentation counter
- * here, since it only counts pages with newly frozen tuples
- * (don't confuse that with pages newly set all-frozen in VM).
- */
- }
- else
- {
- TransactionId snapshotConflictHorizon;
+ vacrel->frozen_pages++;
- vacrel->frozen_pages++;
+ snapshotConflictHorizon = heap_frz_conflict_horizon(&presult, &pagefrz);
- /*
- * We can use frz_conflict_horizon as our cutoff for conflicts
- * when the whole page is eligible to become all-frozen in the VM
- * once we're done with it. Otherwise we generate a conservative
- * cutoff by stepping back from OldestXmin.
- */
- if (presult.all_visible_except_removable && presult.all_frozen)
- {
- /* Using same cutoff when setting VM is now unnecessary */
- snapshotConflictHorizon = presult.frz_conflict_horizon;
- presult.frz_conflict_horizon = InvalidTransactionId;
- }
- else
- {
- /* Avoids false conflicts when hot_standby_feedback in use */
- snapshotConflictHorizon = vacrel->cutoffs.OldestXmin;
- TransactionIdRetreat(snapshotConflictHorizon);
- }
+ /* Using same cutoff when setting VM is now unnecessary */
+ if (presult.all_visible_except_removable && presult.all_frozen)
+ presult.frz_conflict_horizon = InvalidTransactionId;
- /* Execute all freeze plans for page as a single atomic action */
- heap_freeze_execute_prepared(vacrel->rel, buf,
- snapshotConflictHorizon,
- presult.frozen, presult.nfrozen);
- }
+ /* Execute all freeze plans for page as a single atomic action */
+ heap_freeze_execute_prepared(vacrel->rel, buf,
+ snapshotConflictHorizon,
+ presult.frozen, presult.nfrozen);
+ }
+ else if (presult.all_frozen && presult.nfrozen == 0)
+ {
+ /* Page should be all visible except to-be-removed tuples */
+ Assert(presult.all_visible_except_removable);
+
+ /*
+ * We have no freeze plans to execute, so there's no added cost from
+ * following the freeze path. That's why it was chosen. This is
+ * important in the case where the page only contains totally frozen
+ * tuples at this point (perhaps only following pruning). Such pages
+ * can be marked all-frozen in the VM by our caller, even though none
+ * of its tuples were newly frozen here (note that the "no freeze"
+ * path never sets pages all-frozen).
+ *
+ * We never increment the frozen_pages instrumentation counter here,
+ * since it only counts pages with newly frozen tuples (don't confuse
+ * that with pages newly set all-frozen in VM).
+ */
+ vacrel->NewRelfrozenXid = pagefrz.FreezePageRelfrozenXid;
+ vacrel->NewRelminMxid = pagefrz.FreezePageRelminMxid;
}
else
{
--
2.40.1
[text/x-diff] v3-0007-Execute-freezing-in-heap_page_prune.patch (27.5K, ../20240315005658.y7bvbzvnlciqmhd6@liskov/8-v3-0007-Execute-freezing-in-heap_page_prune.patch)
download | inline diff:
From ab80dd6a10d28e2483a074f1a9ea8b445e7d487c Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 8 Mar 2024 16:45:57 -0500
Subject: [PATCH v3 07/17] Execute freezing in heap_page_prune()
As a step toward combining the prune and freeze WAL records, execute
freezing in heap_page_prune(). The logic to determine whether or not to
execute freeze plans was moved from lazy_scan_prune() over to
heap_page_prune() with little modification.
---
src/backend/access/heap/heapam_handler.c | 2 +-
src/backend/access/heap/pruneheap.c | 151 +++++++++++++++++------
src/backend/access/heap/vacuumlazy.c | 129 ++++++-------------
src/backend/storage/ipc/procarray.c | 6 +-
src/include/access/heapam.h | 41 +++---
src/tools/pgindent/typedefs.list | 2 +-
6 files changed, 180 insertions(+), 151 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 680a50bf8b1..5e522f5b0ba 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -1046,7 +1046,7 @@ heapam_scan_analyze_next_tuple(TableScanDesc scan, TransactionId OldestXmin,
* We ignore unused and redirect line pointers. DEAD line pointers
* should be counted as dead, because we need vacuum to run to get rid
* of them. Note that this rule agrees with the way that
- * heap_page_prune() counts things.
+ * heap_page_prune_and_freeze() counts things.
*/
if (!ItemIdIsNormal(itemid))
{
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index 6bd8400b33b..abf6bdb2d99 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -17,16 +17,18 @@
#include "access/heapam.h"
#include "access/heapam_xlog.h"
#include "access/htup_details.h"
+#include "access/multixact.h"
#include "access/transam.h"
#include "access/xlog.h"
#include "access/xloginsert.h"
+#include "executor/instrument.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "storage/bufmgr.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
-/* Working data for heap_page_prune and subroutines */
+/* Working data for heap_page_prune_and_freeze() and subroutines */
typedef struct
{
Relation rel;
@@ -61,17 +63,18 @@ static HTSV_Result heap_prune_satisfies_vacuum(PruneState *prstate,
Buffer buffer);
static int heap_prune_chain(Buffer buffer,
OffsetNumber rootoffnum,
- PruneState *prstate, PruneResult *presult);
+ PruneState *prstate, PruneFreezeResult *presult);
static void prune_prepare_freeze_tuple(Page page, OffsetNumber offnum,
- HeapPageFreeze *pagefrz, PruneResult *presult);
+ HeapPageFreeze *pagefrz, HeapTupleFreeze *frozen,
+ PruneFreezeResult *presult);
static void heap_prune_record_prunable(PruneState *prstate, TransactionId xid);
static void heap_prune_record_redirect(PruneState *prstate,
OffsetNumber offnum, OffsetNumber rdoffnum);
static void heap_prune_record_dead(PruneState *prstate, OffsetNumber offnum,
- PruneResult *presult);
+ PruneFreezeResult *presult);
static void heap_prune_record_dead_or_unused(PruneState *prstate, OffsetNumber offnum,
- PruneResult *presult);
+ PruneFreezeResult *presult);
static void heap_prune_record_unused(PruneState *prstate, OffsetNumber offnum);
static void page_verify_redirects(Page page);
@@ -151,15 +154,15 @@ heap_page_prune_opt(Relation relation, Buffer buffer)
*/
if (PageIsFull(page) || PageGetHeapFreeSpace(page) < minfree)
{
- PruneResult presult;
+ PruneFreezeResult presult;
/*
* For now, pass mark_unused_now as false regardless of whether or
* not the relation has indexes, since we cannot safely determine
* that during on-access pruning with the current implementation.
*/
- heap_page_prune(relation, buffer, vistest, false, NULL,
- &presult, NULL);
+ heap_page_prune_and_freeze(relation, buffer, vistest, false, NULL,
+ &presult, NULL);
/*
* Report the number of tuples reclaimed to pgstats. This is
@@ -207,7 +210,12 @@ prune_freeze_xmin_is_removable(GlobalVisState *visstate, TransactionId xmin)
}
/*
- * Prune and repair fragmentation in the specified page.
+ * Prune and repair fragmentation and potentially freeze tuples on the
+ * specified page.
+ *
+ * If the page can be marked all-frozen in the visibility map, we may
+ * opportunistically freeze tuples on the page if either its tuples are old
+ * enough or freezing will be cheap enough.
*
* Caller must have pin and buffer cleanup lock on the page. Note that we
* don't update the FSM information for page on caller's behalf. Caller might
@@ -221,23 +229,24 @@ prune_freeze_xmin_is_removable(GlobalVisState *visstate, TransactionId xmin)
* mark_unused_now indicates whether or not dead items can be set LP_UNUSED during
* pruning.
*
- * pagefrz contains both input and output parameters used if the caller is
- * interested in potentially freezing tuples on the page.
+ * pagefrz is an input parameter containing visibility cutoff information and
+ * the current relfrozenxid and relminmxids used if the caller is interested in
+ * freezing tuples on the page.
*
* off_loc is the offset location required by the caller to use in error
* callback.
*
* presult contains output parameters needed by callers such as the number of
* tuples removed and the number of line pointers newly marked LP_DEAD.
- * heap_page_prune() is responsible for initializing it.
+ * heap_page_prune_and_freeze() is responsible for initializing it.
*/
void
-heap_page_prune(Relation relation, Buffer buffer,
- GlobalVisState *vistest,
- bool mark_unused_now,
- HeapPageFreeze *pagefrz,
- PruneResult *presult,
- OffsetNumber *off_loc)
+heap_page_prune_and_freeze(Relation relation, Buffer buffer,
+ GlobalVisState *vistest,
+ bool mark_unused_now,
+ HeapPageFreeze *pagefrz,
+ PruneFreezeResult *presult,
+ OffsetNumber *off_loc)
{
Page page = BufferGetPage(buffer);
BlockNumber blockno = BufferGetBlockNumber(buffer);
@@ -245,6 +254,14 @@ heap_page_prune(Relation relation, Buffer buffer,
maxoff;
PruneState prstate;
HeapTupleData tup;
+ bool do_freeze;
+ int64 fpi_before = pgWalUsage.wal_fpi;
+ TransactionId frz_conflict_horizon = InvalidTransactionId;
+
+ /*
+ * One entry for every tuple that we may freeze.
+ */
+ HeapTupleFreeze frozen[MaxHeapTuplesPerPage];
/*
* Our strategy is to scan the page and make lists of items to change,
@@ -281,6 +298,10 @@ heap_page_prune(Relation relation, Buffer buffer,
/* for recovery conflicts */
presult->frz_conflict_horizon = InvalidTransactionId;
+ /* For advancing relfrozenxid and relminmxid */
+ presult->new_relfrozenxid = InvalidTransactionId;
+ presult->new_relminmxid = InvalidMultiXactId;
+
maxoff = PageGetMaxOffsetNumber(page);
tup.t_tableOid = RelationGetRelid(prstate.rel);
@@ -440,7 +461,7 @@ heap_page_prune(Relation relation, Buffer buffer,
if (pagefrz)
prune_prepare_freeze_tuple(page, offnum,
- pagefrz, presult);
+ pagefrz, frozen, presult);
/* Ignore items already processed as part of an earlier chain */
if (prstate.marked[offnum])
@@ -555,6 +576,61 @@ heap_page_prune(Relation relation, Buffer buffer,
/* Record number of newly-set-LP_DEAD items for caller */
presult->nnewlpdead = prstate.ndead;
+
+ /*
+ * Freeze the page when heap_prepare_freeze_tuple indicates that at least
+ * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
+ * freeze when pruning generated an FPI, if doing so means that we set the
+ * page all-frozen afterwards (might not happen until final heap pass).
+ */
+ if (pagefrz)
+ do_freeze = pagefrz->freeze_required ||
+ (presult->all_visible_except_removable && presult->all_frozen &&
+ presult->nfrozen > 0 &&
+ fpi_before != pgWalUsage.wal_fpi);
+ else
+ do_freeze = false;
+
+ if (do_freeze)
+ {
+ frz_conflict_horizon = heap_frz_conflict_horizon(presult, pagefrz);
+
+ /* Execute all freeze plans for page as a single atomic action */
+ heap_freeze_execute_prepared(relation, buffer,
+ frz_conflict_horizon,
+ frozen, presult->nfrozen);
+ }
+ else if (!pagefrz || !presult->all_frozen || presult->nfrozen > 0)
+ {
+ /*
+ * If we will neither freeze tuples on the page nor set the page all
+ * frozen in the visibility map, the page is not all frozen and there
+ * will be no newly frozen tuples.
+ */
+ presult->all_frozen = false;
+ presult->nfrozen = 0; /* avoid miscounts in instrumentation */
+ }
+
+ /* Caller won't update new_relfrozenxid and new_relminmxid */
+ if (!pagefrz)
+ return;
+
+ /*
+ * If we will freeze tuples on the page or, even if we don't freeze tuples
+ * on the page, if we will set the page all-frozen in the visibility map,
+ * we can advance relfrozenxid and relminmxid to the values in
+ * pagefrz->FreezePageRelfrozenXid and pagefrz->FreezePageRelminMxid.
+ */
+ if (presult->all_frozen || presult->nfrozen > 0)
+ {
+ presult->new_relfrozenxid = pagefrz->FreezePageRelfrozenXid;
+ presult->new_relminmxid = pagefrz->FreezePageRelminMxid;
+ }
+ else
+ {
+ presult->new_relfrozenxid = pagefrz->NoFreezePageRelfrozenXid;
+ presult->new_relminmxid = pagefrz->NoFreezePageRelminMxid;
+ }
}
@@ -612,7 +688,7 @@ heap_prune_satisfies_vacuum(PruneState *prstate, HeapTuple tup, Buffer buffer)
*/
static int
heap_prune_chain(Buffer buffer, OffsetNumber rootoffnum,
- PruneState *prstate, PruneResult *presult)
+ PruneState *prstate, PruneFreezeResult *presult)
{
int ndeleted = 0;
Page dp = (Page) BufferGetPage(buffer);
@@ -877,10 +953,10 @@ heap_prune_chain(Buffer buffer, OffsetNumber rootoffnum,
{
/*
* We found a redirect item that doesn't point to a valid follow-on
- * item. This can happen if the loop in heap_page_prune caused us to
- * visit the dead successor of a redirect item before visiting the
- * redirect item. We can clean up by setting the redirect item to
- * DEAD state or LP_UNUSED if the caller indicated.
+ * item. This can happen if the loop in heap_page_prune_and_freeze()
+ * caused us to visit the dead successor of a redirect item before
+ * visiting the redirect item. We can clean up by setting the
+ * redirect item to DEAD state or LP_UNUSED if the caller indicated.
*/
heap_prune_record_dead_or_unused(prstate, rootoffnum, presult);
}
@@ -897,7 +973,8 @@ heap_prune_chain(Buffer buffer, OffsetNumber rootoffnum,
static void
prune_prepare_freeze_tuple(Page page, OffsetNumber offnum,
HeapPageFreeze *pagefrz,
- PruneResult *presult)
+ HeapTupleFreeze *frozen,
+ PruneFreezeResult *presult)
{
bool totally_frozen;
HeapTupleHeader htup;
@@ -919,11 +996,11 @@ prune_prepare_freeze_tuple(Page page, OffsetNumber offnum,
/* Tuple with storage -- consider need to freeze */
if ((heap_prepare_freeze_tuple(htup, pagefrz,
- &presult->frozen[presult->nfrozen],
+ &frozen[presult->nfrozen],
&totally_frozen)))
{
/* Save prepared freeze plan for later */
- presult->frozen[presult->nfrozen++].offset = offnum;
+ frozen[presult->nfrozen++].offset = offnum;
}
/*
@@ -967,7 +1044,7 @@ heap_prune_record_redirect(PruneState *prstate,
/* Record line pointer to be marked dead */
static void
heap_prune_record_dead(PruneState *prstate, OffsetNumber offnum,
- PruneResult *presult)
+ PruneFreezeResult *presult)
{
Assert(prstate->ndead < MaxHeapTuplesPerPage);
prstate->nowdead[prstate->ndead] = offnum;
@@ -990,7 +1067,7 @@ heap_prune_record_dead(PruneState *prstate, OffsetNumber offnum,
*/
static void
heap_prune_record_dead_or_unused(PruneState *prstate, OffsetNumber offnum,
- PruneResult *presult)
+ PruneFreezeResult *presult)
{
/*
* If the caller set mark_unused_now to true, we can remove dead tuples
@@ -1017,9 +1094,9 @@ heap_prune_record_unused(PruneState *prstate, OffsetNumber offnum)
/*
- * Perform the actual page changes needed by heap_page_prune.
- * It is expected that the caller has a full cleanup lock on the
- * buffer.
+ * Perform the actual page pruning modifications needed by
+ * heap_page_prune_and_freeze(). It is expected that the caller has a full
+ * cleanup lock on the buffer.
*/
void
heap_page_prune_execute(Buffer buffer,
@@ -1133,11 +1210,11 @@ heap_page_prune_execute(Buffer buffer,
#ifdef USE_ASSERT_CHECKING
/*
- * When heap_page_prune() was called, mark_unused_now may have been
- * passed as true, which allows would-be LP_DEAD items to be made
- * LP_UNUSED instead. This is only possible if the relation has no
- * indexes. If there are any dead items, then mark_unused_now was not
- * true and every item being marked LP_UNUSED must refer to a
+ * When heap_page_prune_and_freeze() was called, mark_unused_now may
+ * have been passed as true, which allows would-be LP_DEAD items to be
+ * made LP_UNUSED instead. This is only possible if the relation has
+ * no indexes. If there are any dead items, then mark_unused_now was
+ * not true and every item being marked LP_UNUSED must refer to a
* heap-only tuple.
*/
if (ndead > 0)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index abbb7ab3ada..6dd8d457c9c 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -269,9 +269,6 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
-static TransactionId heap_frz_conflict_horizon(PruneResult *presult,
- HeapPageFreeze *pagefrz);
-
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
*
@@ -432,12 +429,13 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
* as an upper bound on the XIDs stored in the pages we'll actually scan
* (NewRelfrozenXid tracking must never be allowed to miss unfrozen XIDs).
*
- * Next acquire vistest, a related cutoff that's used in heap_page_prune.
- * We expect vistest will always make heap_page_prune remove any deleted
- * tuple whose xmax is < OldestXmin. lazy_scan_prune must never become
- * confused about whether a tuple should be frozen or removed. (In the
- * future we might want to teach lazy_scan_prune to recompute vistest from
- * time to time, to increase the number of dead tuples it can prune away.)
+ * Next acquire vistest, a related cutoff that's used in
+ * heap_page_prune_and_freeze(). We expect vistest will always make
+ * heap_page_prune_and_freeze() remove any deleted tuple whose xmax is <
+ * OldestXmin. lazy_scan_prune must never become confused about whether a
+ * tuple should be frozen or removed. (In the future we might want to
+ * teach lazy_scan_prune to recompute vistest from time to time, to
+ * increase the number of dead tuples it can prune away.)
*/
vacrel->aggressive = vacuum_get_cutoffs(rel, params, &vacrel->cutoffs);
vacrel->rel_pages = orig_rel_pages = RelationGetNumberOfBlocks(rel);
@@ -1379,8 +1377,8 @@ lazy_scan_new_or_empty(LVRelState *vacrel, Buffer buf, BlockNumber blkno,
* Determine the snapshotConflictHorizon for freezing. Must only be called
* after pruning and determining if the page is freezable.
*/
-static TransactionId
-heap_frz_conflict_horizon(PruneResult *presult, HeapPageFreeze *pagefrz)
+TransactionId
+heap_frz_conflict_horizon(PruneFreezeResult *presult, HeapPageFreeze *pagefrz)
{
TransactionId result;
@@ -1407,21 +1405,21 @@ heap_frz_conflict_horizon(PruneResult *presult, HeapPageFreeze *pagefrz)
*
* Caller must hold pin and buffer cleanup lock on the buffer.
*
- * Prior to PostgreSQL 14 there were very rare cases where heap_page_prune()
- * was allowed to disagree with our HeapTupleSatisfiesVacuum() call about
- * whether or not a tuple should be considered DEAD. This happened when an
- * inserting transaction concurrently aborted (after our heap_page_prune()
- * call, before our HeapTupleSatisfiesVacuum() call). There was rather a lot
- * of complexity just so we could deal with tuples that were DEAD to VACUUM,
- * but nevertheless were left with storage after pruning.
+ * Prior to PostgreSQL 14 there were very rare cases where
+ * heap_page_prune_and_freeze() was allowed to disagree with our
+ * HeapTupleSatisfiesVacuum() call about whether or not a tuple should be
+ * considered DEAD. This happened when an inserting transaction concurrently
+ * aborted (after our heap_page_prune_and_freeze() call, before our
+ * HeapTupleSatisfiesVacuum() call). There was rather a lot of complexity just
+ * so we could deal with tuples that were DEAD to VACUUM, but nevertheless were
+ * left with storage after pruning.
*
* As of Postgres 17, we circumvent this problem altogether by reusing the
- * result of heap_page_prune()'s visibility check. Without the second call to
- * HeapTupleSatisfiesVacuum(), there is no new HTSV_Result and there can be no
- * disagreement. We'll just handle such tuples as if they had become fully dead
- * right after this operation completes instead of in the middle of it. Note that
- * any tuple that becomes dead after the call to heap_page_prune() can't need to
- * be frozen, because it was visible to another session when vacuum started.
+ * result of heap_page_prune_and_freeze()'s visibility check. Without the
+ * second call to HeapTupleSatisfiesVacuum(), there is no new HTSV_Result and
+ * there can be no disagreement. We'll just handle such tuples as if they had
+ * become fully dead right after this operation completes instead of in the
+ * middle of it.
*
* vmbuffer is the buffer containing the VM block with visibility information
* for the heap block, blkno. all_visible_according_to_vm is the saved
@@ -1444,26 +1442,24 @@ lazy_scan_prune(LVRelState *vacrel,
OffsetNumber offnum,
maxoff;
ItemId itemid;
- PruneResult presult;
+ PruneFreezeResult presult;
int lpdead_items,
live_tuples,
recently_dead_tuples;
HeapPageFreeze pagefrz;
bool hastup = false;
- bool do_freeze;
- int64 fpi_before = pgWalUsage.wal_fpi;
OffsetNumber deadoffsets[MaxHeapTuplesPerPage];
Assert(BufferGetBlockNumber(buf) == blkno);
/*
* maxoff might be reduced following line pointer array truncation in
- * heap_page_prune. That's safe for us to ignore, since the reclaimed
- * space will continue to look like LP_UNUSED items below.
+ * heap_page_prune_and_freeze(). That's safe for us to ignore, since the
+ * reclaimed space will continue to look like LP_UNUSED items below.
*/
maxoff = PageGetMaxOffsetNumber(page);
- /* Initialize (or reset) page-level state */
+ /* Initialize pagefrz */
pagefrz.freeze_required = false;
pagefrz.FreezePageRelfrozenXid = vacrel->NewRelfrozenXid;
pagefrz.FreezePageRelminMxid = vacrel->NewRelminMxid;
@@ -1475,7 +1471,7 @@ lazy_scan_prune(LVRelState *vacrel,
recently_dead_tuples = 0;
/*
- * Prune all HOT-update chains in this page.
+ * Prune all HOT-update chains and potentially freeze tuples on this page.
*
* We count the number of tuples removed from the page by the pruning step
* in presult.ndeleted. It should not be confused with lpdead_items;
@@ -1486,8 +1482,8 @@ lazy_scan_prune(LVRelState *vacrel,
* items LP_UNUSED, so mark_unused_now should be true if no indexes and
* false otherwise.
*/
- heap_page_prune(rel, buf, vacrel->vistest, vacrel->nindexes == 0,
- &pagefrz, &presult, &vacrel->offnum);
+ heap_page_prune_and_freeze(rel, buf, vacrel->vistest, vacrel->nindexes == 0,
+ &pagefrz, &presult, &vacrel->offnum);
/*
* Now scan the page to collect LP_DEAD items and check for tuples
@@ -1604,72 +1600,23 @@ lazy_scan_prune(LVRelState *vacrel,
vacrel->offnum = InvalidOffsetNumber;
- /*
- * Freeze the page when heap_prepare_freeze_tuple indicates that at least
- * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
- * freeze when pruning generated an FPI, if doing so means that we set the
- * page all-frozen afterwards (might not happen until final heap pass).
- */
- do_freeze = pagefrz.freeze_required ||
- (presult.all_visible_except_removable && presult.all_frozen &&
- presult.nfrozen > 0 &&
- fpi_before != pgWalUsage.wal_fpi);
+ Assert(MultiXactIdIsValid(presult.new_relminmxid));
+ vacrel->NewRelfrozenXid = presult.new_relfrozenxid;
+ Assert(TransactionIdIsValid(presult.new_relfrozenxid));
+ vacrel->NewRelminMxid = presult.new_relminmxid;
- if (do_freeze)
+ if (presult.nfrozen > 0)
{
- TransactionId snapshotConflictHorizon;
-
/*
- * We're freezing the page. Our final NewRelfrozenXid doesn't need to
- * be affected by the XIDs that are just about to be frozen anyway.
+ * We never increment the frozen_pages instrumentation counter when
+ * nfrozen == 0, since it only counts pages with newly frozen tuples
+ * (don't confuse that with pages newly set all-frozen in VM).
*/
- vacrel->NewRelfrozenXid = pagefrz.FreezePageRelfrozenXid;
- vacrel->NewRelminMxid = pagefrz.FreezePageRelminMxid;
-
vacrel->frozen_pages++;
- snapshotConflictHorizon = heap_frz_conflict_horizon(&presult, &pagefrz);
-
/* Using same cutoff when setting VM is now unnecessary */
- if (presult.all_visible_except_removable && presult.all_frozen)
+ if (presult.all_frozen)
presult.frz_conflict_horizon = InvalidTransactionId;
-
- /* Execute all freeze plans for page as a single atomic action */
- heap_freeze_execute_prepared(vacrel->rel, buf,
- snapshotConflictHorizon,
- presult.frozen, presult.nfrozen);
- }
- else if (presult.all_frozen && presult.nfrozen == 0)
- {
- /* Page should be all visible except to-be-removed tuples */
- Assert(presult.all_visible_except_removable);
-
- /*
- * We have no freeze plans to execute, so there's no added cost from
- * following the freeze path. That's why it was chosen. This is
- * important in the case where the page only contains totally frozen
- * tuples at this point (perhaps only following pruning). Such pages
- * can be marked all-frozen in the VM by our caller, even though none
- * of its tuples were newly frozen here (note that the "no freeze"
- * path never sets pages all-frozen).
- *
- * We never increment the frozen_pages instrumentation counter here,
- * since it only counts pages with newly frozen tuples (don't confuse
- * that with pages newly set all-frozen in VM).
- */
- vacrel->NewRelfrozenXid = pagefrz.FreezePageRelfrozenXid;
- vacrel->NewRelminMxid = pagefrz.FreezePageRelminMxid;
- }
- else
- {
- /*
- * Page requires "no freeze" processing. It might be set all-visible
- * in the visibility map, but it can never be set all-frozen.
- */
- vacrel->NewRelfrozenXid = pagefrz.NoFreezePageRelfrozenXid;
- vacrel->NewRelminMxid = pagefrz.NoFreezePageRelminMxid;
- presult.all_frozen = false;
- presult.nfrozen = 0; /* avoid miscounts in instrumentation */
}
/*
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index b3cd248fb64..88a6d504dff 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -1715,9 +1715,9 @@ TransactionIdIsActive(TransactionId xid)
* Note: the approximate horizons (see definition of GlobalVisState) are
* updated by the computations done here. That's currently required for
* correctness and a small optimization. Without doing so it's possible that
- * heap vacuum's call to heap_page_prune() uses a more conservative horizon
- * than later when deciding which tuples can be removed - which the code
- * doesn't expect (breaking HOT).
+ * heap vacuum's call to heap_page_prune_and_freeze() uses a more conservative
+ * horizon than later when deciding which tuples can be removed - which the
+ * code doesn't expect (breaking HOT).
*/
static void
ComputeXidHorizons(ComputeXidHorizonsResult *h)
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 2339abfd28a..45c4ae22e6a 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -195,7 +195,7 @@ typedef struct HeapPageFreeze
/*
* Per-page state returned from pruning
*/
-typedef struct PruneResult
+typedef struct PruneFreezeResult
{
int ndeleted; /* Number of tuples deleted from the page */
int nnewlpdead; /* Number of newly LP_DEAD items */
@@ -210,9 +210,10 @@ typedef struct PruneResult
/*
* Tuple visibility is only computed once for each tuple, for correctness
- * and efficiency reasons; see comment in heap_page_prune() for details.
- * This is of type int8[], instead of HTSV_Result[], so we can use -1 to
- * indicate no visibility has been computed, e.g. for LP_DEAD items.
+ * and efficiency reasons; see comment in heap_page_prune_and_freeze() for
+ * details. This is of type int8[], instead of HTSV_Result[], so we can
+ * use -1 to indicate no visibility has been computed, e.g. for LP_DEAD
+ * items.
*
* This needs to be MaxHeapTuplesPerPage + 1 long as FirstOffsetNumber is
* 1. Otherwise every access would need to subtract 1.
@@ -220,17 +221,18 @@ typedef struct PruneResult
int8 htsv[MaxHeapTuplesPerPage + 1];
- /*
- * One entry for every tuple that we may freeze.
- */
- HeapTupleFreeze frozen[MaxHeapTuplesPerPage];
-} PruneResult;
+ /* New value of relfrozenxid found by heap_page_prune_and_freeze() */
+ TransactionId new_relfrozenxid;
+
+ /* New value of relminmxid found by heap_page_prune_and_freeze() */
+ MultiXactId new_relminmxid;
+} PruneFreezeResult;
/*
* Pruning calculates tuple visibility once and saves the results in an array
- * of int8. See PruneResult.htsv for details. This helper function is meant to
- * guard against examining visibility status array members which have not yet
- * been computed.
+ * of int8. See PruneFreezeResult.htsv for details. This helper function is
+ * meant to guard against examining visibility status array members which have
+ * not yet been computed.
*/
static inline HTSV_Result
htsv_get_valid_status(int status)
@@ -306,6 +308,9 @@ extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
Buffer *buffer, struct TM_FailureData *tmfd);
extern void heap_inplace_update(Relation relation, HeapTuple tuple);
+
+extern TransactionId heap_frz_conflict_horizon(PruneFreezeResult *presult,
+ HeapPageFreeze *pagefrz);
extern bool heap_prepare_freeze_tuple(HeapTupleHeader tuple,
HeapPageFreeze *pagefrz,
HeapTupleFreeze *frz, bool *totally_frozen);
@@ -332,12 +337,12 @@ extern TransactionId heap_index_delete_tuples(Relation rel,
/* in heap/pruneheap.c */
struct GlobalVisState;
extern void heap_page_prune_opt(Relation relation, Buffer buffer);
-extern void heap_page_prune(Relation relation, Buffer buffer,
- struct GlobalVisState *vistest,
- bool mark_unused_now,
- HeapPageFreeze *pagefrz,
- PruneResult *presult,
- OffsetNumber *off_loc);
+extern void heap_page_prune_and_freeze(Relation relation, Buffer buffer,
+ struct GlobalVisState *vistest,
+ bool mark_unused_now,
+ HeapPageFreeze *pagefrz,
+ PruneFreezeResult *presult,
+ OffsetNumber *off_loc);
extern void heap_page_prune_execute(Buffer buffer,
OffsetNumber *redirected, int nredirected,
OffsetNumber *nowdead, int ndead,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index aa7a25b8f8c..1c1a4d305d6 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2175,7 +2175,7 @@ ProjectionPath
PromptInterruptContext
ProtocolVersion
PrsStorage
-PruneResult
+PruneFreezeResult
PruneState
PruneStepResult
PsqlScanCallbacks
--
2.40.1
[text/x-diff] v3-0008-Make-opp-freeze-heuristic-compatible-with-prune-f.patch (4.4K, ../20240315005658.y7bvbzvnlciqmhd6@liskov/9-v3-0008-Make-opp-freeze-heuristic-compatible-with-prune-f.patch)
download | inline diff:
From 4a72b89445a3952e06a0648a7f0c7e6eba2f1edc Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Sun, 7 Jan 2024 16:11:35 -0500
Subject: [PATCH v3 08/17] Make opp freeze heuristic compatible with
prune+freeze record
Once the prune and freeze records are combined, we will no longer be
able to use a test of whether or not pruning emitted an FPI to decide
whether or not to opportunistically freeze a freezable page.
While this heuristic should be improved, for now, approximate the
previous logic by keeping track of whether or not a hint bit FPI was
emitted during visibility checks (when checksums are on) and combine
that with checking XLogCheckBufferNeedsBackup(). If we just finished
deciding whether or not to prune and the current buffer seems to need an
FPI after modification, it is likely that pruning would have emitted an
FPI.
---
src/backend/access/heap/pruneheap.c | 58 +++++++++++++++++++++--------
1 file changed, 43 insertions(+), 15 deletions(-)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index abf6bdb2d99..f164b7957ed 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -255,6 +255,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
PruneState prstate;
HeapTupleData tup;
bool do_freeze;
+ bool do_prune;
+ bool whole_page_freezable;
+ bool hint_bit_fpi;
+ bool prune_fpi = false;
int64 fpi_before = pgWalUsage.wal_fpi;
TransactionId frz_conflict_horizon = InvalidTransactionId;
@@ -424,6 +428,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
}
}
+ /*
+ * If checksums are enabled, heap_prune_satisfies_vacuum() may have caused
+ * an FPI to be emitted. Then reset fpi_before for no prune case.
+ */
+ hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi;
+ fpi_before = pgWalUsage.wal_fpi;
+
/*
* For vacuum, if the whole page will become frozen, we consider
* opportunistically freezing tuples. Dead tuples which will be removed by
@@ -481,11 +492,42 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
if (off_loc)
*off_loc = InvalidOffsetNumber;
+ do_prune = prstate.nredirected > 0 ||
+ prstate.ndead > 0 ||
+ prstate.nunused > 0;
+
+ /*
+ * Only incur overhead of checking if we will do an FPI if we might use
+ * the information.
+ */
+ if (do_prune && pagefrz)
+ prune_fpi = XLogCheckBufferNeedsBackup(buffer);
+
+ /* Is the whole page freezable? And is there something to freeze */
+ whole_page_freezable = presult->all_visible_except_removable &&
+ presult->all_frozen;
+
+ /*
+ * Freeze the page when heap_prepare_freeze_tuple indicates that at least
+ * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
+ * freeze when pruning generated an FPI, if doing so means that we set the
+ * page all-frozen afterwards (might not happen until final heap pass).
+ * XXX: Previously, we knew if pruning emitted an FPI by checking
+ * pgWalUsage.wal_fpi before and after pruning. Once the freeze and prune
+ * records are combined, this heuristic couldn't be used anymore. The
+ * opportunistic freeze heuristic must be improved; however, for now, try
+ * to approximate it.
+ */
+
+ do_freeze = pagefrz &&
+ (pagefrz->freeze_required ||
+ (whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi)));
+
/* Any error while applying the changes is critical */
START_CRIT_SECTION();
/* Have we found any prunable items? */
- if (prstate.nredirected > 0 || prstate.ndead > 0 || prstate.nunused > 0)
+ if (do_prune)
{
/*
* Apply the planned item changes, then repair page fragmentation, and
@@ -577,20 +619,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
/* Record number of newly-set-LP_DEAD items for caller */
presult->nnewlpdead = prstate.ndead;
- /*
- * Freeze the page when heap_prepare_freeze_tuple indicates that at least
- * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
- * freeze when pruning generated an FPI, if doing so means that we set the
- * page all-frozen afterwards (might not happen until final heap pass).
- */
- if (pagefrz)
- do_freeze = pagefrz->freeze_required ||
- (presult->all_visible_except_removable && presult->all_frozen &&
- presult->nfrozen > 0 &&
- fpi_before != pgWalUsage.wal_fpi);
- else
- do_freeze = false;
-
if (do_freeze)
{
frz_conflict_horizon = heap_frz_conflict_horizon(presult, pagefrz);
--
2.40.1
[text/x-diff] v3-0009-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch (5.6K, ../20240315005658.y7bvbzvnlciqmhd6@liskov/10-v3-0009-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch)
download | inline diff:
From 07e6620bc79878c1f6a2eed4dd6b338045cd9b40 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Sun, 7 Jan 2024 16:53:45 -0500
Subject: [PATCH v3 09/17] Separate tuple pre freeze checks and invoke earlier
When combining the prune and freeze records their critical sections will
have to be combined. heap_freeze_execute_prepared() does a set of pre
freeze validations before starting its critical section. Move these
validations into a helper function, heap_pre_freeze_checks(), and invoke
it in heap_page_prune() before the pruning critical section.
Also move up the calculation of the freeze snapshot conflict horizon.
---
src/backend/access/heap/heapam.c | 58 ++++++++++++++++-------------
src/backend/access/heap/pruneheap.c | 8 +++-
src/include/access/heapam.h | 3 ++
3 files changed, 42 insertions(+), 27 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 7261c4988d7..16e3f2520a4 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -6659,35 +6659,19 @@ heap_execute_freeze_tuple(HeapTupleHeader tuple, HeapTupleFreeze *frz)
}
/*
- * heap_freeze_execute_prepared
- *
- * Executes freezing of one or more heap tuples on a page on behalf of caller.
- * Caller passes an array of tuple plans from heap_prepare_freeze_tuple.
- * Caller must set 'offset' in each plan for us. Note that we destructively
- * sort caller's tuples array in-place, so caller had better be done with it.
- *
- * WAL-logs the changes so that VACUUM can advance the rel's relfrozenxid
- * later on without any risk of unsafe pg_xact lookups, even following a hard
- * crash (or when querying from a standby). We represent freezing by setting
- * infomask bits in tuple headers, but this shouldn't be thought of as a hint.
- * See section on buffer access rules in src/backend/storage/buffer/README.
- */
+* Perform xmin/xmax XID status sanity checks before calling
+* heap_freeze_execute_prepared().
+*
+* heap_prepare_freeze_tuple doesn't perform these checks directly because
+* pg_xact lookups are relatively expensive. They shouldn't be repeated
+* by successive VACUUMs that each decide against freezing the same page.
+*/
void
-heap_freeze_execute_prepared(Relation rel, Buffer buffer,
- TransactionId snapshotConflictHorizon,
- HeapTupleFreeze *tuples, int ntuples)
+heap_pre_freeze_checks(Buffer buffer,
+ HeapTupleFreeze *tuples, int ntuples)
{
Page page = BufferGetPage(buffer);
- Assert(ntuples > 0);
-
- /*
- * Perform xmin/xmax XID status sanity checks before critical section.
- *
- * heap_prepare_freeze_tuple doesn't perform these checks directly because
- * pg_xact lookups are relatively expensive. They shouldn't be repeated
- * by successive VACUUMs that each decide against freezing the same page.
- */
for (int i = 0; i < ntuples; i++)
{
HeapTupleFreeze *frz = tuples + i;
@@ -6726,6 +6710,30 @@ heap_freeze_execute_prepared(Relation rel, Buffer buffer,
xmax)));
}
}
+}
+
+/*
+ * heap_freeze_execute_prepared
+ *
+ * Executes freezing of one or more heap tuples on a page on behalf of caller.
+ * Caller passes an array of tuple plans from heap_prepare_freeze_tuple.
+ * Caller must set 'offset' in each plan for us. Note that we destructively
+ * sort caller's tuples array in-place, so caller had better be done with it.
+ *
+ * WAL-logs the changes so that VACUUM can advance the rel's relfrozenxid
+ * later on without any risk of unsafe pg_xact lookups, even following a hard
+ * crash (or when querying from a standby). We represent freezing by setting
+ * infomask bits in tuple headers, but this shouldn't be thought of as a hint.
+ * See section on buffer access rules in src/backend/storage/buffer/README.
+ */
+void
+heap_freeze_execute_prepared(Relation rel, Buffer buffer,
+ TransactionId snapshotConflictHorizon,
+ HeapTupleFreeze *tuples, int ntuples)
+{
+ Page page = BufferGetPage(buffer);
+
+ Assert(ntuples > 0);
START_CRIT_SECTION();
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index f164b7957ed..bc0a23da61b 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -523,6 +523,12 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
(pagefrz->freeze_required ||
(whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi)));
+ if (do_freeze)
+ {
+ heap_pre_freeze_checks(buffer, frozen, presult->nfrozen);
+ frz_conflict_horizon = heap_frz_conflict_horizon(presult, pagefrz);
+ }
+
/* Any error while applying the changes is critical */
START_CRIT_SECTION();
@@ -621,8 +627,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
if (do_freeze)
{
- frz_conflict_horizon = heap_frz_conflict_horizon(presult, pagefrz);
-
/* Execute all freeze plans for page as a single atomic action */
heap_freeze_execute_prepared(relation, buffer,
frz_conflict_horizon,
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 45c4ae22e6a..dffbbd3cd3e 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -314,6 +314,9 @@ extern TransactionId heap_frz_conflict_horizon(PruneFreezeResult *presult,
extern bool heap_prepare_freeze_tuple(HeapTupleHeader tuple,
HeapPageFreeze *pagefrz,
HeapTupleFreeze *frz, bool *totally_frozen);
+
+extern void heap_pre_freeze_checks(Buffer buffer,
+ HeapTupleFreeze *tuples, int ntuples);
extern void heap_freeze_execute_prepared(Relation rel, Buffer buffer,
TransactionId snapshotConflictHorizon,
HeapTupleFreeze *tuples, int ntuples);
--
2.40.1
[text/x-diff] v3-0010-Inline-heap_freeze_execute_prepared.patch (8.4K, ../20240315005658.y7bvbzvnlciqmhd6@liskov/11-v3-0010-Inline-heap_freeze_execute_prepared.patch)
download | inline diff:
From 4554c87895dbc0566b345ad590af0fc033142f28 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Sun, 7 Jan 2024 17:03:17 -0500
Subject: [PATCH v3 10/17] Inline heap_freeze_execute_prepared()
In order to merge freeze and prune records, the execution of tuple
freezing and the WAL logging of the changes to the page must be
separated so that the WAL logging can be combined with prune WAL
logging. This commit makes a helper for the tuple freezing and then
inlines the contents of heap_freeze_execute_prepared() where it is
called in heap_page_prune(). The original function,
heap_freeze_execute_prepared() is retained because the "no prune" case
in heap_page_prune() must still be able to emit a freeze record.
---
src/backend/access/heap/heapam.c | 61 +++++++++++++++++------------
src/backend/access/heap/pruneheap.c | 51 ++++++++++++++++++++++--
src/include/access/heapam.h | 8 ++++
3 files changed, 90 insertions(+), 30 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 16e3f2520a4..a3691584c55 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -91,9 +91,6 @@ static void compute_new_xmax_infomask(TransactionId xmax, uint16 old_infomask,
static TM_Result heap_lock_updated_tuple(Relation rel, HeapTuple tuple,
ItemPointer ctid, TransactionId xid,
LockTupleMode mode);
-static int heap_log_freeze_plan(HeapTupleFreeze *tuples, int ntuples,
- xl_heap_freeze_plan *plans_out,
- OffsetNumber *offsets_out);
static void GetMultiXactIdHintBits(MultiXactId multi, uint16 *new_infomask,
uint16 *new_infomask2);
static TransactionId MultiXactIdGetUpdateXid(TransactionId xmax,
@@ -6713,30 +6710,17 @@ heap_pre_freeze_checks(Buffer buffer,
}
/*
- * heap_freeze_execute_prepared
- *
- * Executes freezing of one or more heap tuples on a page on behalf of caller.
- * Caller passes an array of tuple plans from heap_prepare_freeze_tuple.
- * Caller must set 'offset' in each plan for us. Note that we destructively
- * sort caller's tuples array in-place, so caller had better be done with it.
- *
- * WAL-logs the changes so that VACUUM can advance the rel's relfrozenxid
- * later on without any risk of unsafe pg_xact lookups, even following a hard
- * crash (or when querying from a standby). We represent freezing by setting
- * infomask bits in tuple headers, but this shouldn't be thought of as a hint.
- * See section on buffer access rules in src/backend/storage/buffer/README.
+ * Helper which executes freezing of one or more heap tuples on a page on
+ * behalf of caller. Caller passes an array of tuple plans from
+ * heap_prepare_freeze_tuple. Caller must set 'offset' in each plan for us.
+ * Must be called in a critical section that also marks the buffer dirty and,
+ * if needed, emits WAL.
*/
void
-heap_freeze_execute_prepared(Relation rel, Buffer buffer,
- TransactionId snapshotConflictHorizon,
- HeapTupleFreeze *tuples, int ntuples)
+heap_freeze_prepared_tuples(Buffer buffer, HeapTupleFreeze *tuples, int ntuples)
{
Page page = BufferGetPage(buffer);
- Assert(ntuples > 0);
-
- START_CRIT_SECTION();
-
for (int i = 0; i < ntuples; i++)
{
HeapTupleFreeze *frz = tuples + i;
@@ -6746,6 +6730,29 @@ heap_freeze_execute_prepared(Relation rel, Buffer buffer,
htup = (HeapTupleHeader) PageGetItem(page, itemid);
heap_execute_freeze_tuple(htup, frz);
}
+}
+
+/*
+ * heap_freeze_execute_prepared
+ *
+ * Execute freezing of prepared tuples and WAL-logs the changes so that VACUUM
+ * can advance the rel's relfrozenxid later on without any risk of unsafe
+ * pg_xact lookups, even following a hard crash (or when querying from a
+ * standby). We represent freezing by setting infomask bits in tuple headers,
+ * but this shouldn't be thought of as a hint. See section on buffer access
+ * rules in src/backend/storage/buffer/README. Must be called from within a
+ * critical section.
+ */
+void
+heap_freeze_execute_prepared(Relation rel, Buffer buffer,
+ TransactionId snapshotConflictHorizon,
+ HeapTupleFreeze *tuples, int ntuples)
+{
+ Page page = BufferGetPage(buffer);
+
+ Assert(ntuples > 0);
+
+ heap_freeze_prepared_tuples(buffer, tuples, ntuples);
MarkBufferDirty(buffer);
@@ -6758,7 +6765,11 @@ heap_freeze_execute_prepared(Relation rel, Buffer buffer,
xl_heap_freeze_page xlrec;
XLogRecPtr recptr;
- /* Prepare deduplicated representation for use in WAL record */
+ /*
+ * Prepare deduplicated representation for use in WAL record
+ * Destructively sorts tuples array in-place, so caller had better be
+ * done with it.
+ */
nplans = heap_log_freeze_plan(tuples, ntuples, plans, offsets);
xlrec.snapshotConflictHorizon = snapshotConflictHorizon;
@@ -6783,8 +6794,6 @@ heap_freeze_execute_prepared(Relation rel, Buffer buffer,
PageSetLSN(page, recptr);
}
-
- END_CRIT_SECTION();
}
/*
@@ -6874,7 +6883,7 @@ heap_log_freeze_new_plan(xl_heap_freeze_plan *plan, HeapTupleFreeze *frz)
* (actually there is one array per freeze plan, but that's not of immediate
* concern to our caller).
*/
-static int
+int
heap_log_freeze_plan(HeapTupleFreeze *tuples, int ntuples,
xl_heap_freeze_plan *plans_out,
OffsetNumber *offsets_out)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index bc0a23da61b..d4356e0bce9 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -627,10 +627,53 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
if (do_freeze)
{
- /* Execute all freeze plans for page as a single atomic action */
- heap_freeze_execute_prepared(relation, buffer,
- frz_conflict_horizon,
- frozen, presult->nfrozen);
+ START_CRIT_SECTION();
+
+ Assert(presult->nfrozen > 0);
+
+ heap_freeze_prepared_tuples(buffer, frozen, presult->nfrozen);
+
+ MarkBufferDirty(buffer);
+
+ /* Now WAL-log freezing if necessary */
+ if (RelationNeedsWAL(relation))
+ {
+ xl_heap_freeze_plan plans[MaxHeapTuplesPerPage];
+ OffsetNumber offsets[MaxHeapTuplesPerPage];
+ int nplans;
+ xl_heap_freeze_page xlrec;
+ XLogRecPtr recptr;
+
+ /*
+ * Prepare deduplicated representation for use in WAL record
+ * Destructively sorts tuples array in-place.
+ */
+ nplans = heap_log_freeze_plan(frozen, presult->nfrozen, plans, offsets);
+
+ xlrec.snapshotConflictHorizon = frz_conflict_horizon;
+ xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation);
+ xlrec.nplans = nplans;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfHeapFreezePage);
+
+ /*
+ * The freeze plan array and offset array are not actually in the
+ * buffer, but pretend that they are. When XLogInsert stores the
+ * whole buffer, the arrays need not be stored too.
+ */
+ XLogRegisterBuffer(0, buffer, REGBUF_STANDARD);
+ XLogRegisterBufData(0, (char *) plans,
+ nplans * sizeof(xl_heap_freeze_plan));
+ XLogRegisterBufData(0, (char *) offsets,
+ presult->nfrozen * sizeof(OffsetNumber));
+
+ recptr = XLogInsert(RM_HEAP2_ID, XLOG_HEAP2_FREEZE_PAGE);
+
+ PageSetLSN(page, recptr);
+ }
+
+ END_CRIT_SECTION();
}
else if (!pagefrz || !presult->all_frozen || presult->nfrozen > 0)
{
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index dffbbd3cd3e..8a6bc071345 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -14,6 +14,7 @@
#ifndef HEAPAM_H
#define HEAPAM_H
+#include "access/heapam_xlog.h"
#include "access/relation.h" /* for backward compatibility */
#include "access/relscan.h"
#include "access/sdir.h"
@@ -320,9 +321,16 @@ extern void heap_pre_freeze_checks(Buffer buffer,
extern void heap_freeze_execute_prepared(Relation rel, Buffer buffer,
TransactionId snapshotConflictHorizon,
HeapTupleFreeze *tuples, int ntuples);
+
+extern void heap_freeze_prepared_tuples(Buffer buffer,
+ HeapTupleFreeze *tuples, int ntuples);
extern bool heap_freeze_tuple(HeapTupleHeader tuple,
TransactionId relfrozenxid, TransactionId relminmxid,
TransactionId FreezeLimit, TransactionId MultiXactCutoff);
+
+extern int heap_log_freeze_plan(HeapTupleFreeze *tuples, int ntuples,
+ xl_heap_freeze_plan *plans_out,
+ OffsetNumber *offsets_out);
extern bool heap_tuple_should_freeze(HeapTupleHeader tuple,
const struct VacuumCutoffs *cutoffs,
TransactionId *NoFreezePageRelfrozenXid,
--
2.40.1
[text/x-diff] v3-0011-Exit-heap_page_prune-early-if-no-prune.patch (8.3K, ../20240315005658.y7bvbzvnlciqmhd6@liskov/12-v3-0011-Exit-heap_page_prune-early-if-no-prune.patch)
download | inline diff:
From b8a3a4b7bc76a67ba8c9d132d1efd844862bb3dc Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Sun, 7 Jan 2024 17:42:05 -0500
Subject: [PATCH v3 11/17] Exit heap_page_prune() early if no prune
If there is nothing to be pruned on the page, heap_page_prune() will
consider whether or not to update the page's pd_prune_xid and whether or
not to freeze the page. In this case, if we decide to freeze the page,
we will need to emit a freeze record.
Future commits will emit a combined freeze+prune record for cases in
which we are both pruning and freezing. In the no prune case, we are
done with heap_page_prune() after checking whether or not to set
pd_prune_xid. By reversing the prune and no prune cases so that the no
prune case is first, we can exit early in the no prune case. This allows
us to reduce the indentation level of the remaining code and not have to
validate whether or not we are, in fact, pruning.
Since we now exit early in the no prune case, we must set nfrozen and
all_frozen to their final values before executing pruning or freezing.
---
src/backend/access/heap/pruneheap.c | 195 ++++++++++++++++------------
1 file changed, 111 insertions(+), 84 deletions(-)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index d4356e0bce9..d77270ad0d6 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -528,80 +528,27 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
heap_pre_freeze_checks(buffer, frozen, presult->nfrozen);
frz_conflict_horizon = heap_frz_conflict_horizon(presult, pagefrz);
}
-
- /* Any error while applying the changes is critical */
- START_CRIT_SECTION();
-
- /* Have we found any prunable items? */
- if (do_prune)
+ else if (!pagefrz || !presult->all_frozen || presult->nfrozen > 0)
{
/*
- * Apply the planned item changes, then repair page fragmentation, and
- * update the page's hint bit about whether it has free line pointers.
- */
- heap_page_prune_execute(buffer,
- prstate.redirected, prstate.nredirected,
- prstate.nowdead, prstate.ndead,
- prstate.nowunused, prstate.nunused);
-
- /*
- * Update the page's pd_prune_xid field to either zero, or the lowest
- * XID of any soon-prunable tuple.
- */
- ((PageHeader) page)->pd_prune_xid = prstate.new_prune_xid;
-
- /*
- * Also clear the "page is full" flag, since there's no point in
- * repeating the prune/defrag process until something else happens to
- * the page.
- */
- PageClearFull(page);
-
- MarkBufferDirty(buffer);
-
- /*
- * Emit a WAL XLOG_HEAP2_PRUNE record showing what we did
+ * If we will neither freeze tuples on the page nor set the page all
+ * frozen in the visibility map, the page is not all frozen and there
+ * will be no newly frozen tuples.
*/
- if (RelationNeedsWAL(relation))
- {
- xl_heap_prune xlrec;
- XLogRecPtr recptr;
-
- xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation);
- xlrec.snapshotConflictHorizon = prstate.snapshotConflictHorizon;
- xlrec.nredirected = prstate.nredirected;
- xlrec.ndead = prstate.ndead;
-
- XLogBeginInsert();
- XLogRegisterData((char *) &xlrec, SizeOfHeapPrune);
-
- XLogRegisterBuffer(0, buffer, REGBUF_STANDARD);
-
- /*
- * The OffsetNumber arrays are not actually in the buffer, but we
- * pretend that they are. When XLogInsert stores the whole
- * buffer, the offset arrays need not be stored too.
- */
- if (prstate.nredirected > 0)
- XLogRegisterBufData(0, (char *) prstate.redirected,
- prstate.nredirected *
- sizeof(OffsetNumber) * 2);
-
- if (prstate.ndead > 0)
- XLogRegisterBufData(0, (char *) prstate.nowdead,
- prstate.ndead * sizeof(OffsetNumber));
+ presult->all_frozen = false;
+ presult->nfrozen = 0; /* avoid miscounts in instrumentation */
+ }
- if (prstate.nunused > 0)
- XLogRegisterBufData(0, (char *) prstate.nowunused,
- prstate.nunused * sizeof(OffsetNumber));
+ /* Record number of newly-set-LP_DEAD items for caller */
+ presult->nnewlpdead = prstate.ndead;
- recptr = XLogInsert(RM_HEAP2_ID, XLOG_HEAP2_PRUNE);
- PageSetLSN(BufferGetPage(buffer), recptr);
- }
- }
- else
+ /* Have we found any prunable items? */
+ if (!do_prune)
{
+ /* Any error while applying the changes is critical */
+ START_CRIT_SECTION();
+
/*
* If we didn't prune anything, but have found a new value for the
* pd_prune_xid field, update it and mark the buffer dirty. This is
@@ -618,17 +565,105 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
PageClearFull(page);
MarkBufferDirtyHint(buffer, true);
}
+
+ hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi;
+
+ /*
+ * We may have decided not to opportunistically freeze above because
+ * pruning would not emit an FPI. Now, however, if checksums are
+ * enabled, setting the hint bit may have emitted an FPI. Check again
+ * if we should freeze.
+ */
+ if (!do_freeze && hint_bit_fpi)
+ do_freeze = pagefrz &&
+ (pagefrz->freeze_required ||
+ (whole_page_freezable && presult->nfrozen > 0));
+
+ if (do_freeze)
+ {
+ heap_freeze_execute_prepared(relation, buffer,
+ frz_conflict_horizon,
+ frozen, presult->nfrozen);
+ }
+ else if (!pagefrz || !presult->all_frozen || presult->nfrozen > 0)
+ {
+ presult->all_frozen = false;
+ presult->nfrozen = 0;
+ }
+
+ END_CRIT_SECTION();
+
+ goto update_frozenxids;
}
- END_CRIT_SECTION();
+ START_CRIT_SECTION();
- /* Record number of newly-set-LP_DEAD items for caller */
- presult->nnewlpdead = prstate.ndead;
+ /*
+ * Apply the planned item changes, then repair page fragmentation, and
+ * update the page's hint bit about whether it has free line pointers.
+ */
+ heap_page_prune_execute(buffer,
+ prstate.redirected, prstate.nredirected,
+ prstate.nowdead, prstate.ndead,
+ prstate.nowunused, prstate.nunused);
- if (do_freeze)
+ /*
+ * Update the page's pd_prune_xid field to either zero, or the lowest XID
+ * of any soon-prunable tuple.
+ */
+ ((PageHeader) page)->pd_prune_xid = prstate.new_prune_xid;
+
+ /*
+ * Also clear the "page is full" flag, since there's no point in repeating
+ * the prune/defrag process until something else happens to the page.
+ */
+ PageClearFull(page);
+
+ MarkBufferDirty(buffer);
+
+ /*
+ * Emit a WAL XLOG_HEAP2_PRUNE record showing what we did
+ */
+ if (RelationNeedsWAL(relation))
{
- START_CRIT_SECTION();
+ xl_heap_prune xlrec;
+ XLogRecPtr recptr;
+
+ xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation);
+ xlrec.snapshotConflictHorizon = prstate.snapshotConflictHorizon;
+ xlrec.nredirected = prstate.nredirected;
+ xlrec.ndead = prstate.ndead;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfHeapPrune);
+
+ XLogRegisterBuffer(0, buffer, REGBUF_STANDARD);
+ /*
+ * The OffsetNumber arrays are not actually in the buffer, but we
+ * pretend that they are. When XLogInsert stores the whole buffer,
+ * the offset arrays need not be stored too.
+ */
+ if (prstate.nredirected > 0)
+ XLogRegisterBufData(0, (char *) prstate.redirected,
+ prstate.nredirected *
+ sizeof(OffsetNumber) * 2);
+
+ if (prstate.ndead > 0)
+ XLogRegisterBufData(0, (char *) prstate.nowdead,
+ prstate.ndead * sizeof(OffsetNumber));
+
+ if (prstate.nunused > 0)
+ XLogRegisterBufData(0, (char *) prstate.nowunused,
+ prstate.nunused * sizeof(OffsetNumber));
+
+ recptr = XLogInsert(RM_HEAP2_ID, XLOG_HEAP2_PRUNE);
+
+ PageSetLSN(BufferGetPage(buffer), recptr);
+ }
+
+ if (do_freeze)
+ {
Assert(presult->nfrozen > 0);
heap_freeze_prepared_tuples(buffer, frozen, presult->nfrozen);
@@ -672,20 +707,12 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
PageSetLSN(page, recptr);
}
-
- END_CRIT_SECTION();
- }
- else if (!pagefrz || !presult->all_frozen || presult->nfrozen > 0)
- {
- /*
- * If we will neither freeze tuples on the page nor set the page all
- * frozen in the visibility map, the page is not all frozen and there
- * will be no newly frozen tuples.
- */
- presult->all_frozen = false;
- presult->nfrozen = 0; /* avoid miscounts in instrumentation */
}
+ END_CRIT_SECTION();
+
+update_frozenxids:
+
/* Caller won't update new_relfrozenxid and new_relminmxid */
if (!pagefrz)
return;
--
2.40.1
[text/x-diff] v3-0012-Merge-prune-and-freeze-records.patch (8.9K, ../20240315005658.y7bvbzvnlciqmhd6@liskov/13-v3-0012-Merge-prune-and-freeze-records.patch)
download | inline diff:
From 4d8db3931ea86f1cb17e0687cc53e38824887cea Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Sun, 7 Jan 2024 17:55:31 -0500
Subject: [PATCH v3 12/17] Merge prune and freeze records
When there are both tuples to prune and freeze on a page, emit a single,
combined prune record containing the offsets for pruning and the freeze
plans and offsets for freezing. This will reduce the number of WAL
records emitted.
---
src/backend/access/heap/heapam.c | 42 ++++++++++++--
src/backend/access/heap/pruneheap.c | 85 +++++++++++++----------------
src/include/access/heapam_xlog.h | 20 +++++--
3 files changed, 90 insertions(+), 57 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index a3691584c55..a8f35eba3c9 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -8803,24 +8803,28 @@ heap_xlog_prune(XLogReaderState *record)
if (action == BLK_NEEDS_REDO)
{
Page page = (Page) BufferGetPage(buffer);
- OffsetNumber *end;
OffsetNumber *redirected;
OffsetNumber *nowdead;
OffsetNumber *nowunused;
int nredirected;
int ndead;
int nunused;
+ int nplans;
Size datalen;
+ xl_heap_freeze_plan *plans;
+ OffsetNumber *frz_offsets;
+ int curoff = 0;
- redirected = (OffsetNumber *) XLogRecGetBlockData(record, 0, &datalen);
-
+ nplans = xlrec->nplans;
nredirected = xlrec->nredirected;
ndead = xlrec->ndead;
- end = (OffsetNumber *) ((char *) redirected + datalen);
+ nunused = xlrec->nunused;
+
+ plans = (xl_heap_freeze_plan *) XLogRecGetBlockData(record, 0, &datalen);
+ redirected = (OffsetNumber *) &plans[nplans];
nowdead = redirected + (nredirected * 2);
nowunused = nowdead + ndead;
- nunused = (end - nowunused);
- Assert(nunused >= 0);
+ frz_offsets = nowunused + nunused;
/* Update all line pointers per the record, and repair fragmentation */
heap_page_prune_execute(buffer,
@@ -8828,6 +8832,32 @@ heap_xlog_prune(XLogReaderState *record)
nowdead, ndead,
nowunused, nunused);
+ for (int p = 0; p < nplans; p++)
+ {
+ HeapTupleFreeze frz;
+
+ /*
+ * Convert freeze plan representation from WAL record into
+ * per-tuple format used by heap_execute_freeze_tuple
+ */
+ frz.xmax = plans[p].xmax;
+ frz.t_infomask2 = plans[p].t_infomask2;
+ frz.t_infomask = plans[p].t_infomask;
+ frz.frzflags = plans[p].frzflags;
+ frz.offset = InvalidOffsetNumber; /* unused, but be tidy */
+
+ for (int i = 0; i < plans[p].ntuples; i++)
+ {
+ OffsetNumber offset = frz_offsets[curoff++];
+ ItemId lp;
+ HeapTupleHeader tuple;
+
+ lp = PageGetItemId(page, offset);
+ tuple = (HeapTupleHeader) PageGetItem(page, lp);
+ heap_execute_freeze_tuple(tuple, &frz);
+ }
+ }
+
/*
* Note: we don't worry about updating the page's prunability hints.
* At worst this will cause an extra prune cycle to occur soon.
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index d77270ad0d6..994cf75c54e 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -619,6 +619,9 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
*/
PageClearFull(page);
+ if (do_freeze)
+ heap_freeze_prepared_tuples(buffer, frozen, presult->nfrozen);
+
MarkBufferDirty(buffer);
/*
@@ -629,10 +632,37 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
xl_heap_prune xlrec;
XLogRecPtr recptr;
+ xl_heap_freeze_plan plans[MaxHeapTuplesPerPage];
+ OffsetNumber offsets[MaxHeapTuplesPerPage];
+
xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation);
- xlrec.snapshotConflictHorizon = prstate.snapshotConflictHorizon;
xlrec.nredirected = prstate.nredirected;
xlrec.ndead = prstate.ndead;
+ xlrec.nunused = prstate.nunused;
+ xlrec.nplans = 0;
+
+ /*
+ * The snapshotConflictHorizon for the whole record should be the most
+ * conservative of all the horizons calculated for any of the possible
+ * modifications. If this record will prune tuples, any transactions
+ * on the standby older than the youngest xmax of the most recently
+ * removed tuple this record will prune will conflict. If this record
+ * will freeze tuples, any transactions on the standby with xids older
+ * than the youngest tuple this record will freeze will conflict.
+ */
+ if (do_freeze)
+ xlrec.snapshotConflictHorizon = Max(prstate.snapshotConflictHorizon,
+ frz_conflict_horizon);
+ else
+ xlrec.snapshotConflictHorizon = prstate.snapshotConflictHorizon;
+
+ /*
+ * Prepare deduplicated representation for use in WAL record
+ * Destructively sorts tuples array in-place.
+ */
+ if (do_freeze)
+ xlrec.nplans = heap_log_freeze_plan(frozen,
+ presult->nfrozen, plans, offsets);
XLogBeginInsert();
XLogRegisterData((char *) &xlrec, SizeOfHeapPrune);
@@ -644,6 +674,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
* pretend that they are. When XLogInsert stores the whole buffer,
* the offset arrays need not be stored too.
*/
+ if (xlrec.nplans > 0)
+ XLogRegisterBufData(0, (char *) plans,
+ xlrec.nplans * sizeof(xl_heap_freeze_plan));
+
if (prstate.nredirected > 0)
XLogRegisterBufData(0, (char *) prstate.redirected,
prstate.nredirected *
@@ -657,56 +691,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
XLogRegisterBufData(0, (char *) prstate.nowunused,
prstate.nunused * sizeof(OffsetNumber));
- recptr = XLogInsert(RM_HEAP2_ID, XLOG_HEAP2_PRUNE);
-
- PageSetLSN(BufferGetPage(buffer), recptr);
- }
-
- if (do_freeze)
- {
- Assert(presult->nfrozen > 0);
-
- heap_freeze_prepared_tuples(buffer, frozen, presult->nfrozen);
-
- MarkBufferDirty(buffer);
-
- /* Now WAL-log freezing if necessary */
- if (RelationNeedsWAL(relation))
- {
- xl_heap_freeze_plan plans[MaxHeapTuplesPerPage];
- OffsetNumber offsets[MaxHeapTuplesPerPage];
- int nplans;
- xl_heap_freeze_page xlrec;
- XLogRecPtr recptr;
-
- /*
- * Prepare deduplicated representation for use in WAL record
- * Destructively sorts tuples array in-place.
- */
- nplans = heap_log_freeze_plan(frozen, presult->nfrozen, plans, offsets);
-
- xlrec.snapshotConflictHorizon = frz_conflict_horizon;
- xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation);
- xlrec.nplans = nplans;
-
- XLogBeginInsert();
- XLogRegisterData((char *) &xlrec, SizeOfHeapFreezePage);
-
- /*
- * The freeze plan array and offset array are not actually in the
- * buffer, but pretend that they are. When XLogInsert stores the
- * whole buffer, the arrays need not be stored too.
- */
- XLogRegisterBuffer(0, buffer, REGBUF_STANDARD);
- XLogRegisterBufData(0, (char *) plans,
- nplans * sizeof(xl_heap_freeze_plan));
+ if (xlrec.nplans > 0)
XLogRegisterBufData(0, (char *) offsets,
presult->nfrozen * sizeof(OffsetNumber));
- recptr = XLogInsert(RM_HEAP2_ID, XLOG_HEAP2_FREEZE_PAGE);
+ recptr = XLogInsert(RM_HEAP2_ID, XLOG_HEAP2_PRUNE);
- PageSetLSN(page, recptr);
- }
+ PageSetLSN(BufferGetPage(buffer), recptr);
}
END_CRIT_SECTION();
diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h
index 6488dad5e64..22f236bb52a 100644
--- a/src/include/access/heapam_xlog.h
+++ b/src/include/access/heapam_xlog.h
@@ -231,23 +231,35 @@ typedef struct xl_heap_update
* during opportunistic pruning)
*
* The array of OffsetNumbers following the fixed part of the record contains:
+ * * for each freeze plan: the freeze plan
* * for each redirected item: the item offset, then the offset redirected to
* * for each now-dead item: the item offset
* * for each now-unused item: the item offset
- * The total number of OffsetNumbers is therefore 2*nredirected+ndead+nunused.
- * Note that nunused is not explicitly stored, but may be found by reference
- * to the total record length.
+ * * for each tuple frozen by the freeze plans: the offset of the item corresponding to that tuple
+ * The total number of OffsetNumbers is therefore
+ * (2*nredirected) + ndead + nunused + (sum[plan.ntuples for plan in plans])
*
* Acquires a full cleanup lock.
*/
typedef struct xl_heap_prune
{
TransactionId snapshotConflictHorizon;
+ uint16 nplans;
uint16 nredirected;
uint16 ndead;
+ uint16 nunused;
bool isCatalogRel; /* to handle recovery conflict during logical
* decoding on standby */
- /* OFFSET NUMBERS are in the block reference 0 */
+ /*
+ * OFFSET NUMBERS and freeze plans are in the block reference 0 in the
+ * following order:
+ *
+ * * xl_heap_freeze_plan plans[nplans];
+ * * OffsetNumber redirected[2 * nredirected];
+ * * OffsetNumber nowdead[ndead];
+ * * OffsetNumber nowunused[nunused];
+ * * OffsetNumber frz_offsets[...];
+ */
} xl_heap_prune;
#define SizeOfHeapPrune (offsetof(xl_heap_prune, isCatalogRel) + sizeof(bool))
--
2.40.1
[text/x-diff] v3-0013-Set-hastup-in-heap_page_prune.patch (7.3K, ../20240315005658.y7bvbzvnlciqmhd6@liskov/14-v3-0013-Set-hastup-in-heap_page_prune.patch)
download | inline diff:
From 46dc299c2bc878799e2c56ed4c240d5c5284b986 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 22 Jan 2024 14:53:36 -0500
Subject: [PATCH v3 13/17] Set hastup in heap_page_prune
lazy_scan_prune() loops through the line pointers and tuple visibility
information for each tuple on a page, setting hastup to true if there
are any LP_REDIRECT line pointers or tuples with storage which will not
be removed. We want to remove this extra loop from lazy_scan_prune(),
and we know about non-removable tuples during heap_page_prune() anyway.
Set hastup when recording LP_REDIRECT line pointers in
heap_prune_chain() and when LP_NORMAL line pointers refer to tuples
whose visibility status is not HEAPTUPLE_DEAD.
---
src/backend/access/heap/pruneheap.c | 33 ++++++++++++++++++++++++----
src/backend/access/heap/vacuumlazy.c | 25 ++-------------------
src/include/access/heapam.h | 3 +++
3 files changed, 34 insertions(+), 27 deletions(-)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index 994cf75c54e..2fee9aa509c 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -70,7 +70,8 @@ static void prune_prepare_freeze_tuple(Page page, OffsetNumber offnum,
PruneFreezeResult *presult);
static void heap_prune_record_prunable(PruneState *prstate, TransactionId xid);
static void heap_prune_record_redirect(PruneState *prstate,
- OffsetNumber offnum, OffsetNumber rdoffnum);
+ OffsetNumber offnum, OffsetNumber rdoffnum,
+ PruneFreezeResult *presult);
static void heap_prune_record_dead(PruneState *prstate, OffsetNumber offnum,
PruneFreezeResult *presult);
static void heap_prune_record_dead_or_unused(PruneState *prstate, OffsetNumber offnum,
@@ -294,6 +295,8 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
presult->nnewlpdead = 0;
presult->nfrozen = 0;
+ presult->hastup = false;
+
/*
* Keep track of whether or not the page is all_visible in case the caller
* wants to use this information to update the VM.
@@ -474,18 +477,37 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
prune_prepare_freeze_tuple(page, offnum,
pagefrz, frozen, presult);
+ itemid = PageGetItemId(page, offnum);
+
+ if (ItemIdIsNormal(itemid) &&
+ presult->htsv[offnum] != HEAPTUPLE_DEAD)
+ {
+ Assert(presult->htsv[offnum] != -1);
+
+ /*
+ * Deliberately don't set hastup for LP_DEAD items. We make the
+ * soft assumption that any LP_DEAD items encountered here will
+ * become LP_UNUSED later on, before count_nondeletable_pages is
+ * reached. If we don't make this assumption then rel truncation
+ * will only happen every other VACUUM, at most. Besides, VACUUM
+ * must treat hastup/nonempty_pages as provisional no matter how
+ * LP_DEAD items are handled (handled here, or handled later on).
+ */
+ presult->hastup = true;
+ }
+
/* Ignore items already processed as part of an earlier chain */
if (prstate.marked[offnum])
continue;
/* Nothing to do if slot is empty */
- itemid = PageGetItemId(page, offnum);
if (!ItemIdIsUsed(itemid))
continue;
/* Process this item or chain of items */
presult->ndeleted += heap_prune_chain(buffer, offnum,
&prstate, presult);
+
}
/* Clear the offset information once we have processed the given page. */
@@ -1040,7 +1062,7 @@ heap_prune_chain(Buffer buffer, OffsetNumber rootoffnum,
if (i >= nchain)
heap_prune_record_dead_or_unused(prstate, rootoffnum, presult);
else
- heap_prune_record_redirect(prstate, rootoffnum, chainitems[i]);
+ heap_prune_record_redirect(prstate, rootoffnum, chainitems[i], presult);
}
else if (nchain < 2 && ItemIdIsRedirected(rootlp))
{
@@ -1122,7 +1144,8 @@ heap_prune_record_prunable(PruneState *prstate, TransactionId xid)
/* Record line pointer to be redirected */
static void
heap_prune_record_redirect(PruneState *prstate,
- OffsetNumber offnum, OffsetNumber rdoffnum)
+ OffsetNumber offnum, OffsetNumber rdoffnum,
+ PruneFreezeResult *presult)
{
Assert(prstate->nredirected < MaxHeapTuplesPerPage);
prstate->redirected[prstate->nredirected * 2] = offnum;
@@ -1132,6 +1155,8 @@ heap_prune_record_redirect(PruneState *prstate,
prstate->marked[offnum] = true;
Assert(!prstate->marked[rdoffnum]);
prstate->marked[rdoffnum] = true;
+
+ presult->hastup = true;
}
/* Record line pointer to be marked dead */
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 6dd8d457c9c..aac38f54c0a 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -1447,7 +1447,6 @@ lazy_scan_prune(LVRelState *vacrel,
live_tuples,
recently_dead_tuples;
HeapPageFreeze pagefrz;
- bool hastup = false;
OffsetNumber deadoffsets[MaxHeapTuplesPerPage];
Assert(BufferGetBlockNumber(buf) == blkno);
@@ -1491,7 +1490,6 @@ lazy_scan_prune(LVRelState *vacrel,
* the VM after collecting LP_DEAD items and freezing tuples. Pruning will
* have determined whether or not the page is all_visible and able to
* become all_frozen.
- *
*/
for (offnum = FirstOffsetNumber;
offnum <= maxoff;
@@ -1504,28 +1502,12 @@ lazy_scan_prune(LVRelState *vacrel,
vacrel->offnum = offnum;
itemid = PageGetItemId(page, offnum);
- if (!ItemIdIsUsed(itemid))
- continue;
-
/* Redirect items mustn't be touched */
- if (ItemIdIsRedirected(itemid))
- {
- /* page makes rel truncation unsafe */
- hastup = true;
+ if (ItemIdIsRedirected(itemid) || !ItemIdIsUsed(itemid))
continue;
- }
if (ItemIdIsDead(itemid))
{
- /*
- * Deliberately don't set hastup for LP_DEAD items. We make the
- * soft assumption that any LP_DEAD items encountered here will
- * become LP_UNUSED later on, before count_nondeletable_pages is
- * reached. If we don't make this assumption then rel truncation
- * will only happen every other VACUUM, at most. Besides, VACUUM
- * must treat hastup/nonempty_pages as provisional no matter how
- * LP_DEAD items are handled (handled here, or handled later on).
- */
deadoffsets[lpdead_items++] = offnum;
continue;
}
@@ -1593,9 +1575,6 @@ lazy_scan_prune(LVRelState *vacrel,
elog(ERROR, "unexpected HeapTupleSatisfiesVacuum result");
break;
}
-
- hastup = true; /* page makes rel truncation unsafe */
-
}
vacrel->offnum = InvalidOffsetNumber;
@@ -1677,7 +1656,7 @@ lazy_scan_prune(LVRelState *vacrel,
vacrel->recently_dead_tuples += recently_dead_tuples;
/* Can't truncate this page */
- if (hastup)
+ if (presult.hastup)
vacrel->nonempty_pages = blkno + 1;
/* Did we find LP_DEAD items? */
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 8a6bc071345..7ad46696d66 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -202,11 +202,14 @@ typedef struct PruneFreezeResult
int nnewlpdead; /* Number of newly LP_DEAD items */
bool all_visible; /* Whether or not the page is all visible */
bool all_visible_except_removable;
+ bool hastup; /* Does page make rel truncation unsafe */
+
/* Whether or not the page can be set all frozen in the VM */
bool all_frozen;
/* Number of newly frozen tuples */
int nfrozen;
+
TransactionId frz_conflict_horizon; /* Newest xmin on the page */
/*
--
2.40.1
[text/x-diff] v3-0014-Count-tuples-for-vacuum-logging-in-heap_page_prun.patch (15.1K, ../20240315005658.y7bvbzvnlciqmhd6@liskov/15-v3-0014-Count-tuples-for-vacuum-logging-in-heap_page_prun.patch)
download | inline diff:
From 444ba496410ba547564f2fed1fb7bea4e8d1e020 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 22 Jan 2024 17:25:56 -0500
Subject: [PATCH v3 14/17] Count tuples for vacuum logging in heap_page_prune
lazy_scan_prune() loops through all of the tuple visibility information
that was recorded in heap_page_prune() and then counts live and recently
dead tuples. That information is available in heap_page_prune(), so just
record it there. Add live and recently dead tuple counters to the
PruneResult. Doing this counting in heap_page_prune() eliminates the
need for saving the tuple visibility status information in the
PruneResult. Instead, save it in the PruneState where it can be
referenced by heap_prune_chain().
---
src/backend/access/heap/pruneheap.c | 110 +++++++++++++++++++++++----
src/backend/access/heap/vacuumlazy.c | 77 +------------------
src/include/access/heapam.h | 29 +------
3 files changed, 99 insertions(+), 117 deletions(-)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index 2fee9aa509c..575cbcb13a3 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -55,6 +55,18 @@ typedef struct
* 1. Otherwise every access would need to subtract 1.
*/
bool marked[MaxHeapTuplesPerPage + 1];
+
+ /*
+ * Tuple visibility is only computed once for each tuple, for correctness
+ * and efficiency reasons; see comment in heap_page_prune_and_freeze() for
+ * details. This is of type int8[], instead of HTSV_Result[], so we can
+ * use -1 to indicate no visibility has been computed, e.g. for LP_DEAD
+ * items.
+ *
+ * This needs to be MaxHeapTuplesPerPage + 1 long as FirstOffsetNumber is
+ * 1. Otherwise every access would need to subtract 1.
+ */
+ int8 htsv[MaxHeapTuplesPerPage + 1];
} PruneState;
/* Local functions */
@@ -65,7 +77,8 @@ static int heap_prune_chain(Buffer buffer,
OffsetNumber rootoffnum,
PruneState *prstate, PruneFreezeResult *presult);
-static void prune_prepare_freeze_tuple(Page page, OffsetNumber offnum,
+static inline HTSV_Result htsv_get_valid_status(int status);
+static void prune_prepare_freeze_tuple(Page page, OffsetNumber offnum, PruneState *prstate,
HeapPageFreeze *pagefrz, HeapTupleFreeze *frozen,
PruneFreezeResult *presult);
static void heap_prune_record_prunable(PruneState *prstate, TransactionId xid);
@@ -297,6 +310,9 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
presult->hastup = false;
+ presult->live_tuples = 0;
+ presult->recently_dead_tuples = 0;
+
/*
* Keep track of whether or not the page is all_visible in case the caller
* wants to use this information to update the VM.
@@ -342,7 +358,7 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
/* Nothing to do if slot doesn't contain a tuple */
if (!ItemIdIsNormal(itemid))
{
- presult->htsv[offnum] = -1;
+ prstate.htsv[offnum] = -1;
continue;
}
@@ -358,9 +374,30 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
if (off_loc)
*off_loc = offnum;
- presult->htsv[offnum] = heap_prune_satisfies_vacuum(&prstate, &tup,
- buffer);
- switch (presult->htsv[offnum])
+ prstate.htsv[offnum] = heap_prune_satisfies_vacuum(&prstate, &tup,
+ buffer);
+ Assert(ItemIdIsNormal(itemid));
+
+ /*
+ * The criteria for counting a tuple as live in this block need to
+ * match what analyze.c's acquire_sample_rows() does, otherwise VACUUM
+ * and ANALYZE may produce wildly different reltuples values, e.g.
+ * when there are many recently-dead tuples.
+ *
+ * The logic here is a bit simpler than acquire_sample_rows(), as
+ * VACUUM can't run inside a transaction block, which makes some cases
+ * impossible (e.g. in-progress insert from the same transaction).
+ *
+ * We treat LP_DEAD items (which are the closest thing to DEAD tuples
+ * that might be seen here) differently, too: we assume that they'll
+ * become LP_UNUSED before VACUUM finishes. This difference is only
+ * superficial. VACUUM effectively agrees with ANALYZE about DEAD
+ * items, in the end. VACUUM won't remember LP_DEAD items, but only
+ * because they're not supposed to be left behind when it is done.
+ * (Cases where we bypass index vacuuming will violate this optimistic
+ * assumption, but the overall impact of that should be negligible.)
+ */
+ switch (prstate.htsv[offnum])
{
case HEAPTUPLE_DEAD:
@@ -380,6 +417,12 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
break;
case HEAPTUPLE_LIVE:
+ /*
+ * Count it as live. Not only is this natural, but it's also
+ * what acquire_sample_rows() does.
+ */
+ presult->live_tuples++;
+
/*
* Is the tuple definitely visible to all transactions?
*
@@ -416,13 +459,34 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
}
break;
case HEAPTUPLE_RECENTLY_DEAD:
+
+ /*
+ * If tuple is recently dead then we must not remove it from
+ * the relation. (We only remove items that are LP_DEAD from
+ * pruning.)
+ */
+ presult->recently_dead_tuples++;
presult->all_visible = false;
break;
case HEAPTUPLE_INSERT_IN_PROGRESS:
+
+ /*
+ * We do not count these rows as live, because we expect the
+ * inserting transaction to update the counters at commit, and
+ * we assume that will happen only after we report our
+ * results. This assumption is a bit shaky, but it is what
+ * acquire_sample_rows() does, so be consistent.
+ */
presult->all_visible = false;
break;
case HEAPTUPLE_DELETE_IN_PROGRESS:
- /* This is an expected case during concurrent vacuum */
+
+ /*
+ * This an expected case during concurrent vacuum. Count such
+ * rows as live. As above, we assume the deleting transaction
+ * will commit and update the counters after we report.
+ */
+ presult->live_tuples++;
presult->all_visible = false;
break;
default:
@@ -474,15 +538,15 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
*off_loc = offnum;
if (pagefrz)
- prune_prepare_freeze_tuple(page, offnum,
+ prune_prepare_freeze_tuple(page, offnum, &prstate,
pagefrz, frozen, presult);
itemid = PageGetItemId(page, offnum);
if (ItemIdIsNormal(itemid) &&
- presult->htsv[offnum] != HEAPTUPLE_DEAD)
+ prstate.htsv[offnum] != HEAPTUPLE_DEAD)
{
- Assert(presult->htsv[offnum] != -1);
+ Assert(prstate.htsv[offnum] != -1);
/*
* Deliberately don't set hastup for LP_DEAD items. We make the
@@ -770,10 +834,24 @@ heap_prune_satisfies_vacuum(PruneState *prstate, HeapTuple tup, Buffer buffer)
}
+/*
+ * Pruning calculates tuple visibility once and saves the results in an array
+ * of int8. See PruneState.htsv for details. This helper function is meant to
+ * guard against examining visibility status array members which have not yet
+ * been computed.
+ */
+static inline HTSV_Result
+htsv_get_valid_status(int status)
+{
+ Assert(status >= HEAPTUPLE_DEAD &&
+ status <= HEAPTUPLE_DELETE_IN_PROGRESS);
+ return (HTSV_Result) status;
+}
+
/*
* Prune specified line pointer or a HOT chain originating at line pointer.
*
- * Tuple visibility information is provided in presult->htsv.
+ * Tuple visibility information is provided in prstate->htsv.
*
* If the item is an index-referenced tuple (i.e. not a heap-only tuple),
* the HOT chain is pruned by removing all DEAD tuples at the start of the HOT
@@ -824,7 +902,7 @@ heap_prune_chain(Buffer buffer, OffsetNumber rootoffnum,
*/
if (ItemIdIsNormal(rootlp))
{
- Assert(presult->htsv[rootoffnum] != -1);
+ Assert(prstate->htsv[rootoffnum] != -1);
htup = (HeapTupleHeader) PageGetItem(dp, rootlp);
if (HeapTupleHeaderIsHeapOnly(htup))
@@ -847,7 +925,7 @@ heap_prune_chain(Buffer buffer, OffsetNumber rootoffnum,
* either here or while following a chain below. Whichever path
* gets there first will mark the tuple unused.
*/
- if (presult->htsv[rootoffnum] == HEAPTUPLE_DEAD &&
+ if (prstate->htsv[rootoffnum] == HEAPTUPLE_DEAD &&
!HeapTupleHeaderIsHotUpdated(htup))
{
heap_prune_record_unused(prstate, rootoffnum);
@@ -948,7 +1026,7 @@ heap_prune_chain(Buffer buffer, OffsetNumber rootoffnum,
*/
tupdead = recent_dead = false;
- switch (htsv_get_valid_status(presult->htsv[offnum]))
+ switch (htsv_get_valid_status(prstate->htsv[offnum]))
{
case HEAPTUPLE_DEAD:
tupdead = true;
@@ -1086,7 +1164,7 @@ heap_prune_chain(Buffer buffer, OffsetNumber rootoffnum,
* want to consider freezing normal tuples which will not be removed.
*/
static void
-prune_prepare_freeze_tuple(Page page, OffsetNumber offnum,
+prune_prepare_freeze_tuple(Page page, OffsetNumber offnum, PruneState *prstate,
HeapPageFreeze *pagefrz,
HeapTupleFreeze *frozen,
PruneFreezeResult *presult)
@@ -1103,8 +1181,8 @@ prune_prepare_freeze_tuple(Page page, OffsetNumber offnum,
return;
/* We do not consider freezing tuples which will be removed. */
- if (presult->htsv[offnum] == HEAPTUPLE_DEAD ||
- presult->htsv[offnum] == -1)
+ if (prstate->htsv[offnum] == HEAPTUPLE_DEAD ||
+ prstate->htsv[offnum] == -1)
return;
htup = (HeapTupleHeader) PageGetItem(page, itemid);
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index aac38f54c0a..634f4da9a17 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -1442,10 +1442,8 @@ lazy_scan_prune(LVRelState *vacrel,
OffsetNumber offnum,
maxoff;
ItemId itemid;
+ int lpdead_items = 0;
PruneFreezeResult presult;
- int lpdead_items,
- live_tuples,
- recently_dead_tuples;
HeapPageFreeze pagefrz;
OffsetNumber deadoffsets[MaxHeapTuplesPerPage];
@@ -1465,9 +1463,6 @@ lazy_scan_prune(LVRelState *vacrel,
pagefrz.NoFreezePageRelfrozenXid = vacrel->NewRelfrozenXid;
pagefrz.NoFreezePageRelminMxid = vacrel->NewRelminMxid;
pagefrz.cutoffs = &vacrel->cutoffs;
- lpdead_items = 0;
- live_tuples = 0;
- recently_dead_tuples = 0;
/*
* Prune all HOT-update chains and potentially freeze tuples on this page.
@@ -1502,9 +1497,6 @@ lazy_scan_prune(LVRelState *vacrel,
vacrel->offnum = offnum;
itemid = PageGetItemId(page, offnum);
- /* Redirect items mustn't be touched */
- if (ItemIdIsRedirected(itemid) || !ItemIdIsUsed(itemid))
- continue;
if (ItemIdIsDead(itemid))
{
@@ -1512,69 +1504,6 @@ lazy_scan_prune(LVRelState *vacrel,
continue;
}
- Assert(ItemIdIsNormal(itemid));
-
- /*
- * The criteria for counting a tuple as live in this block need to
- * match what analyze.c's acquire_sample_rows() does, otherwise VACUUM
- * and ANALYZE may produce wildly different reltuples values, e.g.
- * when there are many recently-dead tuples.
- *
- * The logic here is a bit simpler than acquire_sample_rows(), as
- * VACUUM can't run inside a transaction block, which makes some cases
- * impossible (e.g. in-progress insert from the same transaction).
- *
- * We treat LP_DEAD items (which are the closest thing to DEAD tuples
- * that might be seen here) differently, too: we assume that they'll
- * become LP_UNUSED before VACUUM finishes. This difference is only
- * superficial. VACUUM effectively agrees with ANALYZE about DEAD
- * items, in the end. VACUUM won't remember LP_DEAD items, but only
- * because they're not supposed to be left behind when it is done.
- * (Cases where we bypass index vacuuming will violate this optimistic
- * assumption, but the overall impact of that should be negligible.)
- */
- switch (htsv_get_valid_status(presult.htsv[offnum]))
- {
- case HEAPTUPLE_LIVE:
-
- /*
- * Count it as live. Not only is this natural, but it's also
- * what acquire_sample_rows() does.
- */
- live_tuples++;
- break;
- case HEAPTUPLE_RECENTLY_DEAD:
-
- /*
- * If tuple is recently dead then we must not remove it from
- * the relation. (We only remove items that are LP_DEAD from
- * pruning.)
- */
- recently_dead_tuples++;
- break;
- case HEAPTUPLE_INSERT_IN_PROGRESS:
-
- /*
- * We do not count these rows as live, because we expect the
- * inserting transaction to update the counters at commit, and
- * we assume that will happen only after we report our
- * results. This assumption is a bit shaky, but it is what
- * acquire_sample_rows() does, so be consistent.
- */
- break;
- case HEAPTUPLE_DELETE_IN_PROGRESS:
-
- /*
- * This an expected case during concurrent vacuum. Count such
- * rows as live. As above, we assume the deleting transaction
- * will commit and update the counters after we report.
- */
- live_tuples++;
- break;
- default:
- elog(ERROR, "unexpected HeapTupleSatisfiesVacuum result");
- break;
- }
}
vacrel->offnum = InvalidOffsetNumber;
@@ -1652,8 +1581,8 @@ lazy_scan_prune(LVRelState *vacrel,
vacrel->tuples_deleted += presult.ndeleted;
vacrel->tuples_frozen += presult.nfrozen;
vacrel->lpdead_items += lpdead_items;
- vacrel->live_tuples += live_tuples;
- vacrel->recently_dead_tuples += recently_dead_tuples;
+ vacrel->live_tuples += presult.live_tuples;
+ vacrel->recently_dead_tuples += presult.recently_dead_tuples;
/* Can't truncate this page */
if (presult.hastup)
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 7ad46696d66..22a2494a3f8 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -198,6 +198,8 @@ typedef struct HeapPageFreeze
*/
typedef struct PruneFreezeResult
{
+ int live_tuples;
+ int recently_dead_tuples;
int ndeleted; /* Number of tuples deleted from the page */
int nnewlpdead; /* Number of newly LP_DEAD items */
bool all_visible; /* Whether or not the page is all visible */
@@ -212,19 +214,6 @@ typedef struct PruneFreezeResult
TransactionId frz_conflict_horizon; /* Newest xmin on the page */
- /*
- * Tuple visibility is only computed once for each tuple, for correctness
- * and efficiency reasons; see comment in heap_page_prune_and_freeze() for
- * details. This is of type int8[], instead of HTSV_Result[], so we can
- * use -1 to indicate no visibility has been computed, e.g. for LP_DEAD
- * items.
- *
- * This needs to be MaxHeapTuplesPerPage + 1 long as FirstOffsetNumber is
- * 1. Otherwise every access would need to subtract 1.
- */
- int8 htsv[MaxHeapTuplesPerPage + 1];
-
-
/* New value of relfrozenxid found by heap_page_prune_and_freeze() */
TransactionId new_relfrozenxid;
@@ -232,20 +221,6 @@ typedef struct PruneFreezeResult
MultiXactId new_relminmxid;
} PruneFreezeResult;
-/*
- * Pruning calculates tuple visibility once and saves the results in an array
- * of int8. See PruneFreezeResult.htsv for details. This helper function is
- * meant to guard against examining visibility status array members which have
- * not yet been computed.
- */
-static inline HTSV_Result
-htsv_get_valid_status(int status)
-{
- Assert(status >= HEAPTUPLE_DEAD &&
- status <= HEAPTUPLE_DELETE_IN_PROGRESS);
- return (HTSV_Result) status;
-}
-
/* ----------------
* function prototypes for heap access method
*
--
2.40.1
[text/x-diff] v3-0015-Save-dead-tuple-offsets-during-heap_page_prune.patch (7.0K, ../20240315005658.y7bvbzvnlciqmhd6@liskov/16-v3-0015-Save-dead-tuple-offsets-during-heap_page_prune.patch)
download | inline diff:
From 8a09af63ceedd3ea24ab975470c09b20383410bf Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 22 Jan 2024 16:55:28 -0500
Subject: [PATCH v3 15/17] Save dead tuple offsets during heap_page_prune
After heap_page_prune() returned, lazy_scan_prune() looped through all
of the offsets of LP_DEAD items which it later added to
LVRelState->dead_items. Instead take care of this when marking a line
pointer or when an existing non-removable LP_DEAD item is encountered in
heap_prune_chain().
---
src/backend/access/heap/pruneheap.c | 7 ++++
src/backend/access/heap/vacuumlazy.c | 60 ++++++----------------------
src/include/access/heapam.h | 2 +
3 files changed, 22 insertions(+), 47 deletions(-)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index 575cbcb13a3..fa628e410e6 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -312,6 +312,7 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
presult->live_tuples = 0;
presult->recently_dead_tuples = 0;
+ presult->lpdead_items = 0;
/*
* Keep track of whether or not the page is all_visible in case the caller
@@ -1001,7 +1002,10 @@ heap_prune_chain(Buffer buffer, OffsetNumber rootoffnum,
if (unlikely(prstate->mark_unused_now))
heap_prune_record_unused(prstate, offnum);
else
+ {
presult->all_visible = false;
+ presult->deadoffsets[presult->lpdead_items++] = offnum;
+ }
break;
}
@@ -1253,6 +1257,9 @@ heap_prune_record_dead(PruneState *prstate, OffsetNumber offnum,
* all_visible.
*/
presult->all_visible = false;
+
+ /* Record the dead offset for vacuum */
+ presult->deadoffsets[presult->lpdead_items++] = offnum;
}
/*
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 634f4da9a17..4b45e8be1ad 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -1439,23 +1439,11 @@ lazy_scan_prune(LVRelState *vacrel,
bool *has_lpdead_items)
{
Relation rel = vacrel->rel;
- OffsetNumber offnum,
- maxoff;
- ItemId itemid;
- int lpdead_items = 0;
PruneFreezeResult presult;
HeapPageFreeze pagefrz;
- OffsetNumber deadoffsets[MaxHeapTuplesPerPage];
Assert(BufferGetBlockNumber(buf) == blkno);
- /*
- * maxoff might be reduced following line pointer array truncation in
- * heap_page_prune_and_freeze(). That's safe for us to ignore, since the
- * reclaimed space will continue to look like LP_UNUSED items below.
- */
- maxoff = PageGetMaxOffsetNumber(page);
-
/* Initialize pagefrz */
pagefrz.freeze_required = false;
pagefrz.FreezePageRelfrozenXid = vacrel->NewRelfrozenXid;
@@ -1468,9 +1456,9 @@ lazy_scan_prune(LVRelState *vacrel,
* Prune all HOT-update chains and potentially freeze tuples on this page.
*
* We count the number of tuples removed from the page by the pruning step
- * in presult.ndeleted. It should not be confused with lpdead_items;
- * lpdead_items's final value can be thought of as the number of tuples
- * that were deleted from indexes.
+ * in presult.ndeleted. It should not be confused with
+ * presult.lpdead_items; presult.lpdead_items's final value can be thought
+ * of as the number of tuples that were deleted from indexes.
*
* If the relation has no indexes, we can immediately mark would-be dead
* items LP_UNUSED, so mark_unused_now should be true if no indexes and
@@ -1480,32 +1468,10 @@ lazy_scan_prune(LVRelState *vacrel,
&pagefrz, &presult, &vacrel->offnum);
/*
- * Now scan the page to collect LP_DEAD items and check for tuples
- * requiring freezing among remaining tuples with storage. We will update
- * the VM after collecting LP_DEAD items and freezing tuples. Pruning will
- * have determined whether or not the page is all_visible and able to
- * become all_frozen.
+ * We will update the VM after collecting LP_DEAD items and freezing
+ * tuples. Pruning will have determined whether or not the page is
+ * all_visible.
*/
- for (offnum = FirstOffsetNumber;
- offnum <= maxoff;
- offnum = OffsetNumberNext(offnum))
- {
- /*
- * Set the offset number so that we can display it along with any
- * error that occurred while processing this tuple.
- */
- vacrel->offnum = offnum;
- itemid = PageGetItemId(page, offnum);
-
-
- if (ItemIdIsDead(itemid))
- {
- deadoffsets[lpdead_items++] = offnum;
- continue;
- }
-
- }
-
vacrel->offnum = InvalidOffsetNumber;
Assert(MultiXactIdIsValid(presult.new_relminmxid));
@@ -1541,7 +1507,7 @@ lazy_scan_prune(LVRelState *vacrel,
TransactionId debug_cutoff;
bool debug_all_frozen;
- Assert(lpdead_items == 0);
+ Assert(presult.lpdead_items == 0);
if (!heap_page_is_all_visible(vacrel, buf,
&debug_cutoff, &debug_all_frozen))
@@ -1557,7 +1523,7 @@ lazy_scan_prune(LVRelState *vacrel,
/*
* Now save details of the LP_DEAD items from the page in vacrel
*/
- if (lpdead_items > 0)
+ if (presult.lpdead_items > 0)
{
VacDeadItems *dead_items = vacrel->dead_items;
ItemPointerData tmp;
@@ -1566,9 +1532,9 @@ lazy_scan_prune(LVRelState *vacrel,
ItemPointerSetBlockNumber(&tmp, blkno);
- for (int i = 0; i < lpdead_items; i++)
+ for (int i = 0; i < presult.lpdead_items; i++)
{
- ItemPointerSetOffsetNumber(&tmp, deadoffsets[i]);
+ ItemPointerSetOffsetNumber(&tmp, presult.deadoffsets[i]);
dead_items->items[dead_items->num_items++] = tmp;
}
@@ -1580,7 +1546,7 @@ lazy_scan_prune(LVRelState *vacrel,
/* Finally, add page-local counts to whole-VACUUM counts */
vacrel->tuples_deleted += presult.ndeleted;
vacrel->tuples_frozen += presult.nfrozen;
- vacrel->lpdead_items += lpdead_items;
+ vacrel->lpdead_items += presult.lpdead_items;
vacrel->live_tuples += presult.live_tuples;
vacrel->recently_dead_tuples += presult.recently_dead_tuples;
@@ -1589,7 +1555,7 @@ lazy_scan_prune(LVRelState *vacrel,
vacrel->nonempty_pages = blkno + 1;
/* Did we find LP_DEAD items? */
- *has_lpdead_items = (lpdead_items > 0);
+ *has_lpdead_items = (presult.lpdead_items > 0);
Assert(!presult.all_visible || !(*has_lpdead_items));
@@ -1657,7 +1623,7 @@ lazy_scan_prune(LVRelState *vacrel,
* There should never be LP_DEAD items on a page with PD_ALL_VISIBLE set,
* however.
*/
- else if (lpdead_items > 0 && PageIsAllVisible(page))
+ else if (presult.lpdead_items > 0 && PageIsAllVisible(page))
{
elog(WARNING, "page containing LP_DEAD items is marked as all-visible in relation \"%s\" page %u",
vacrel->relname, blkno);
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 22a2494a3f8..cc3071644c3 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -219,6 +219,8 @@ typedef struct PruneFreezeResult
/* New value of relminmxid found by heap_page_prune_and_freeze() */
MultiXactId new_relminmxid;
+ int lpdead_items; /* includes existing LP_DEAD items */
+ OffsetNumber deadoffsets[MaxHeapTuplesPerPage];
} PruneFreezeResult;
/* ----------------
--
2.40.1
[text/x-diff] v3-0016-Obsolete-XLOG_HEAP2_FREEZE_PAGE.patch (13.4K, ../20240315005658.y7bvbzvnlciqmhd6@liskov/17-v3-0016-Obsolete-XLOG_HEAP2_FREEZE_PAGE.patch)
download | inline diff:
From 5871dbff01dc89c0a1dfc09db341fff8314451c0 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Tue, 12 Mar 2024 19:07:38 -0400
Subject: [PATCH v3 16/17] Obsolete XLOG_HEAP2_FREEZE_PAGE
When vacuum freezes tuples, the information needed to replay these
changes is saved in the xl_heap_prune record. As such, we no longer need
to create new xl_heap_freeze records. We can get rid of
heap_freeze_execute_prepared() as well as the special case in
heap_page_prune_and_freeze() for when only freezing is done.
We must retain the xl_heap_freeze_page record and
heap_xlog_freeze_page() in order to replay old freeze records.
---
src/backend/access/heap/heapam.c | 78 ++-----------
src/backend/access/heap/pruneheap.c | 163 ++++++++++++++--------------
src/include/access/heapam.h | 7 +-
3 files changed, 88 insertions(+), 160 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index a8f35eba3c9..12a1a7805f4 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -6340,7 +6340,7 @@ FreezeMultiXactId(MultiXactId multi, uint16 t_infomask,
* XIDs or MultiXactIds that will need to be processed by a future VACUUM.
*
* VACUUM caller must assemble HeapTupleFreeze freeze plan entries for every
- * tuple that we returned true for, and call heap_freeze_execute_prepared to
+ * tuple that we returned true for, and call heap_page_prune_and_freeze() to
* execute freezing. Caller must initialize pagefrz fields for page as a
* whole before first call here for each heap page.
*
@@ -6656,8 +6656,7 @@ heap_execute_freeze_tuple(HeapTupleHeader tuple, HeapTupleFreeze *frz)
}
/*
-* Perform xmin/xmax XID status sanity checks before calling
-* heap_freeze_execute_prepared().
+* Perform xmin/xmax XID status sanity checks before executing freezing.
*
* heap_prepare_freeze_tuple doesn't perform these checks directly because
* pg_xact lookups are relatively expensive. They shouldn't be repeated
@@ -6732,70 +6731,6 @@ heap_freeze_prepared_tuples(Buffer buffer, HeapTupleFreeze *tuples, int ntuples)
}
}
-/*
- * heap_freeze_execute_prepared
- *
- * Execute freezing of prepared tuples and WAL-logs the changes so that VACUUM
- * can advance the rel's relfrozenxid later on without any risk of unsafe
- * pg_xact lookups, even following a hard crash (or when querying from a
- * standby). We represent freezing by setting infomask bits in tuple headers,
- * but this shouldn't be thought of as a hint. See section on buffer access
- * rules in src/backend/storage/buffer/README. Must be called from within a
- * critical section.
- */
-void
-heap_freeze_execute_prepared(Relation rel, Buffer buffer,
- TransactionId snapshotConflictHorizon,
- HeapTupleFreeze *tuples, int ntuples)
-{
- Page page = BufferGetPage(buffer);
-
- Assert(ntuples > 0);
-
- heap_freeze_prepared_tuples(buffer, tuples, ntuples);
-
- MarkBufferDirty(buffer);
-
- /* Now WAL-log freezing if necessary */
- if (RelationNeedsWAL(rel))
- {
- xl_heap_freeze_plan plans[MaxHeapTuplesPerPage];
- OffsetNumber offsets[MaxHeapTuplesPerPage];
- int nplans;
- xl_heap_freeze_page xlrec;
- XLogRecPtr recptr;
-
- /*
- * Prepare deduplicated representation for use in WAL record
- * Destructively sorts tuples array in-place, so caller had better be
- * done with it.
- */
- nplans = heap_log_freeze_plan(tuples, ntuples, plans, offsets);
-
- xlrec.snapshotConflictHorizon = snapshotConflictHorizon;
- xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(rel);
- xlrec.nplans = nplans;
-
- XLogBeginInsert();
- XLogRegisterData((char *) &xlrec, SizeOfHeapFreezePage);
-
- /*
- * The freeze plan array and offset array are not actually in the
- * buffer, but pretend that they are. When XLogInsert stores the
- * whole buffer, the arrays need not be stored too.
- */
- XLogRegisterBuffer(0, buffer, REGBUF_STANDARD);
- XLogRegisterBufData(0, (char *) plans,
- nplans * sizeof(xl_heap_freeze_plan));
- XLogRegisterBufData(0, (char *) offsets,
- ntuples * sizeof(OffsetNumber));
-
- recptr = XLogInsert(RM_HEAP2_ID, XLOG_HEAP2_FREEZE_PAGE);
-
- PageSetLSN(page, recptr);
- }
-}
-
/*
* Comparator used to deduplicate XLOG_HEAP2_FREEZE_PAGE freeze plans
*/
@@ -8827,10 +8762,11 @@ heap_xlog_prune(XLogReaderState *record)
frz_offsets = nowunused + nunused;
/* Update all line pointers per the record, and repair fragmentation */
- heap_page_prune_execute(buffer,
- redirected, nredirected,
- nowdead, ndead,
- nowunused, nunused);
+ if (nredirected > 0 || ndead > 0 || nunused > 0)
+ heap_page_prune_execute(buffer,
+ redirected, nredirected,
+ nowdead, ndead,
+ nowunused, nunused);
for (int p = 0; p < nplans; p++)
{
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index fa628e410e6..6f45e5c37f0 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -270,6 +270,7 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
HeapTupleData tup;
bool do_freeze;
bool do_prune;
+ bool do_hint;
bool whole_page_freezable;
bool hint_bit_fpi;
bool prune_fpi = false;
@@ -583,6 +584,9 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
prstate.ndead > 0 ||
prstate.nunused > 0;
+ /* Record number of newly-set-LP_DEAD items for caller */
+ presult->nnewlpdead = prstate.ndead;
+
/*
* Only incur overhead of checking if we will do an FPI if we might use
* the information.
@@ -590,7 +594,15 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
if (do_prune && pagefrz)
prune_fpi = XLogCheckBufferNeedsBackup(buffer);
- /* Is the whole page freezable? And is there something to freeze */
+ /*
+ * Even if we don't prune anything, if we found a new value for the
+ * pd_prune_xid field or the page was marked full, we will update the hint
+ * bit.
+ */
+ do_hint = ((PageHeader) page)->pd_prune_xid != prstate.new_prune_xid ||
+ PageIsFull(page);
+
+ /* Is the whole page freezable? And is there something to freeze? */
whole_page_freezable = presult->all_visible_except_removable &&
presult->all_frozen;
@@ -605,55 +617,63 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
* opportunistic freeze heuristic must be improved; however, for now, try
* to approximate it.
*/
-
do_freeze = pagefrz &&
(pagefrz->freeze_required ||
(whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi)));
- if (do_freeze)
- {
+ /*
+ * Validate the tuples we are considering freezing. We do this even if
+ * pruning and hint bit setting have not emitted an FPI so far because we
+ * still may emit an FPI while setting the page hint bit later. But we
+ * want to avoid doing the pre-freeze checks in a critical section.
+ */
+ if (pagefrz &&
+ (pagefrz->freeze_required ||
+ (whole_page_freezable && presult->nfrozen > 0)))
heap_pre_freeze_checks(buffer, frozen, presult->nfrozen);
- frz_conflict_horizon = heap_frz_conflict_horizon(presult, pagefrz);
- }
- else if (!pagefrz || !presult->all_frozen || presult->nfrozen > 0)
- {
- /*
- * If we will neither freeze tuples on the page nor set the page all
- * frozen in the visibility map, the page is not all frozen and there
- * will be no newly frozen tuples.
- */
- presult->all_frozen = false;
- presult->nfrozen = 0; /* avoid miscounts in instrumentation */
- }
- /* Record number of newly-set-LP_DEAD items for caller */
- presult->nnewlpdead = prstate.ndead;
+ /*
+ * If we are going to modify the page contents anyway, we will have to
+ * update more than hint bits.
+ */
+ if (do_freeze || do_prune)
+ do_hint = false;
+ START_CRIT_SECTION();
- /* Have we found any prunable items? */
- if (!do_prune)
- {
- /* Any error while applying the changes is critical */
- START_CRIT_SECTION();
+ /*
+ * Update the page's pd_prune_xid field to either zero, or the lowest XID
+ * of any soon-prunable tuple.
+ */
+ ((PageHeader) page)->pd_prune_xid = prstate.new_prune_xid;
- /*
- * If we didn't prune anything, but have found a new value for the
- * pd_prune_xid field, update it and mark the buffer dirty. This is
- * treated as a non-WAL-logged hint.
- *
- * Also clear the "page is full" flag if it is set, since there's no
- * point in repeating the prune/defrag process until something else
- * happens to the page.
- */
- if (((PageHeader) page)->pd_prune_xid != prstate.new_prune_xid ||
- PageIsFull(page))
- {
- ((PageHeader) page)->pd_prune_xid = prstate.new_prune_xid;
- PageClearFull(page);
- MarkBufferDirtyHint(buffer, true);
- }
+ /*
+ * If pruning, freezing, or updating the hint bit, clear the "page is
+ * full" flag if it is set since there's no point in repeating the
+ * prune/defrag process until something else happens to the page.
+ */
+ if (do_prune || do_freeze || do_hint)
+ PageClearFull(page);
- hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi;
+ /*
+ * Apply the planned item changes, then repair page fragmentation, and
+ * update the page's hint bit about whether it has free line pointers.
+ */
+ if (do_prune)
+ {
+ heap_page_prune_execute(buffer,
+ prstate.redirected, prstate.nredirected,
+ prstate.nowdead, prstate.ndead,
+ prstate.nowunused, prstate.nunused);
+ }
+
+ /*
+ * If we aren't pruning or freezing anything, but we updated pd_prune_xid,
+ * this is a non-WAL-logged hint.
+ */
+ if (do_hint)
+ {
+ MarkBufferDirtyHint(buffer, true);
/*
* We may have decided not to opportunistically freeze above because
@@ -661,60 +681,37 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
* enabled, setting the hint bit may have emitted an FPI. Check again
* if we should freeze.
*/
- if (!do_freeze && hint_bit_fpi)
+ hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi;
+
+ if (hint_bit_fpi)
do_freeze = pagefrz &&
(pagefrz->freeze_required ||
(whole_page_freezable && presult->nfrozen > 0));
-
- if (do_freeze)
- {
- heap_freeze_execute_prepared(relation, buffer,
- frz_conflict_horizon,
- frozen, presult->nfrozen);
- }
- else if (!pagefrz || !presult->all_frozen || presult->nfrozen > 0)
- {
- presult->all_frozen = false;
- presult->nfrozen = 0;
- }
-
- END_CRIT_SECTION();
-
- goto update_frozenxids;
}
- START_CRIT_SECTION();
-
- /*
- * Apply the planned item changes, then repair page fragmentation, and
- * update the page's hint bit about whether it has free line pointers.
- */
- heap_page_prune_execute(buffer,
- prstate.redirected, prstate.nredirected,
- prstate.nowdead, prstate.ndead,
- prstate.nowunused, prstate.nunused);
-
- /*
- * Update the page's pd_prune_xid field to either zero, or the lowest XID
- * of any soon-prunable tuple.
- */
- ((PageHeader) page)->pd_prune_xid = prstate.new_prune_xid;
-
- /*
- * Also clear the "page is full" flag, since there's no point in repeating
- * the prune/defrag process until something else happens to the page.
- */
- PageClearFull(page);
-
if (do_freeze)
+ {
+ frz_conflict_horizon = heap_frz_conflict_horizon(presult, pagefrz);
heap_freeze_prepared_tuples(buffer, frozen, presult->nfrozen);
+ }
+ else if ((!pagefrz || !presult->all_frozen || presult->nfrozen > 0))
+ {
+ /*
+ * If we will neither freeze tuples on the page nor set the page all
+ * frozen in the visibility map, the page is not all-frozen and there
+ * will be no newly frozen tuples.
+ */
+ presult->all_frozen = false;
+ presult->nfrozen = 0; /* avoid miscounts in instrumentation */
+ }
- MarkBufferDirty(buffer);
+ if (do_prune || do_freeze)
+ MarkBufferDirty(buffer);
/*
* Emit a WAL XLOG_HEAP2_PRUNE record showing what we did
*/
- if (RelationNeedsWAL(relation))
+ if ((do_prune || do_freeze) && RelationNeedsWAL(relation))
{
xl_heap_prune xlrec;
XLogRecPtr recptr;
@@ -789,8 +786,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
END_CRIT_SECTION();
-update_frozenxids:
-
/* Caller won't update new_relfrozenxid and new_relminmxid */
if (!pagefrz)
return;
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index cc3071644c3..c36623f53bd 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -102,7 +102,7 @@ typedef enum
} HTSV_Result;
/*
- * heap_prepare_freeze_tuple may request that heap_freeze_execute_prepared
+ * heap_prepare_freeze_tuple may request that the heap_page_prune_and_freeze()
* check any tuple's to-be-frozen xmin and/or xmax status using pg_xact
*/
#define HEAP_FREEZE_CHECK_XMIN_COMMITTED 0x01
@@ -155,7 +155,7 @@ typedef struct HeapPageFreeze
/*
* "Freeze" NewRelfrozenXid/NewRelminMxid trackers.
*
- * Trackers used when heap_freeze_execute_prepared freezes, or when there
+ * Trackers used when heap_page_prune_and_freeze() freezes, or when there
* are zero freeze plans for a page. It is always valid for vacuumlazy.c
* to freeze any page, by definition. This even includes pages that have
* no tuples with storage to consider in the first place. That way the
@@ -298,9 +298,6 @@ extern bool heap_prepare_freeze_tuple(HeapTupleHeader tuple,
extern void heap_pre_freeze_checks(Buffer buffer,
HeapTupleFreeze *tuples, int ntuples);
-extern void heap_freeze_execute_prepared(Relation rel, Buffer buffer,
- TransactionId snapshotConflictHorizon,
- HeapTupleFreeze *tuples, int ntuples);
extern void heap_freeze_prepared_tuples(Buffer buffer,
HeapTupleFreeze *tuples, int ntuples);
--
2.40.1
[text/x-diff] v3-0017-Streamline-XLOG_HEAP2_PRUNE-record.patch (20.2K, ../20240315005658.y7bvbzvnlciqmhd6@liskov/18-v3-0017-Streamline-XLOG_HEAP2_PRUNE-record.patch)
download | inline diff:
From 1dddbaea7de88911be06bae4ecfe47119c6812a1 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Wed, 13 Mar 2024 00:28:57 -0400
Subject: [PATCH v3 17/17] Streamline XLOG_HEAP2_PRUNE record
xl_heap_prune struct for the XLOG_HEAP2_PRUNE record type had members
for counting the number of freeze plans and number of redirected, dead,
and newly unused line pointers. However, only some of those are used in
many XLOG_HEAP2_PRUNE records. As part of a refactor to use
XLOG_HEAP2_PRUNE record types instead of XLOG_HEAP2_FREEZE_PAGE records
when only freezing is being done, eliminate those members and instead
use flags to indicate which of those types of modifications will be
done. The resulting record will contain only data about modifications
that must be done.
ci-os-only:
---
src/backend/access/heap/heapam.c | 121 ++++++++++++++++++-----
src/backend/access/heap/pruneheap.c | 86 ++++++++++++----
src/backend/access/rmgrdesc/heapdesc.c | 130 +++++++++++++++++++------
src/include/access/heapam_xlog.h | 122 ++++++++++++++---------
src/tools/pgindent/typedefs.list | 2 +
5 files changed, 337 insertions(+), 124 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 12a1a7805f4..11aa176b6c3 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -8703,10 +8703,73 @@ ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
return key_tuple;
}
+/*
+ * Given a MAXALIGNed buffer returned by XLogRecGetBlockData() and pointed to
+ * by cursor and any xl_heap_prune flags, deserialize the arrays of
+ * OffsetNumbers contained in an xl_heap_prune record.
+ */
+static void
+heap_xlog_deserialize_prune_and_freeze(char *cursor, uint8 flags,
+ int *nredirected, OffsetNumber **redirected,
+ int *ndead, OffsetNumber **nowdead,
+ int *nunused, OffsetNumber **nowunused,
+ int *nplans, xl_heap_freeze_plan **plans,
+ OffsetNumber **frz_offsets)
+{
+ if (flags & XLHP_HAS_FREEZE_PLANS)
+ {
+ xlhp_freeze *freeze = (xlhp_freeze *) cursor;
+
+ *nplans = freeze->nplans;
+ Assert(*nplans > 0);
+ *plans = freeze->plans;
+
+ cursor += offsetof(xlhp_freeze, plans);
+ cursor += sizeof(xl_heap_freeze_plan) * freeze->nplans;
+ }
+
+ if (flags & XLHP_HAS_REDIRECTIONS)
+ {
+ xlhp_prune_items *subrecord = (xlhp_prune_items *) cursor;
+
+ *nredirected = subrecord->ntargets;
+ Assert(nredirected > 0);
+ *redirected = &subrecord->data[0];
+
+ cursor += offsetof(xlhp_prune_items, data);
+ cursor += sizeof(OffsetNumber[2]) * *nredirected;
+ }
+
+ if (flags & XLHP_HAS_DEAD_ITEMS)
+ {
+ xlhp_prune_items *subrecord = (xlhp_prune_items *) cursor;
+
+ *ndead = subrecord->ntargets;
+ Assert(ndead > 0);
+ *nowdead = subrecord->data;
+
+ cursor += offsetof(xlhp_prune_items, data);
+ cursor += sizeof(OffsetNumber) * *ndead;
+ }
+
+ if (flags & XLHP_HAS_NOW_UNUSED_ITEMS)
+ {
+ xlhp_prune_items *subrecord = (xlhp_prune_items *) cursor;
+
+ *nunused = subrecord->ntargets;
+ Assert(nunused > 0);
+ *nowunused = subrecord->data;
+
+ cursor += offsetof(xlhp_prune_items, data);
+ cursor += sizeof(OffsetNumber) * *nunused;
+ }
+
+ if (nplans > 0)
+ *frz_offsets = (OffsetNumber *) cursor;
+}
+
/*
* Handles XLOG_HEAP2_PRUNE record type.
- *
- * Acquires a full cleanup lock.
*/
static void
heap_xlog_prune(XLogReaderState *record)
@@ -8717,49 +8780,54 @@ heap_xlog_prune(XLogReaderState *record)
RelFileLocator rlocator;
BlockNumber blkno;
XLogRedoAction action;
+ bool get_cleanup_lock;
XLogRecGetBlockTag(record, 0, &rlocator, NULL, &blkno);
+ /*
+ * If there are dead, redirected, or unused items,
+ * heap_page_prune_execute() will call PageRepairFragementation() which
+ * expects a full cleanup lock.
+ */
+ get_cleanup_lock = xlrec->flags & XLHP_HAS_REDIRECTIONS ||
+ xlrec->flags & XLHP_HAS_DEAD_ITEMS ||
+ xlrec->flags & XLHP_HAS_NOW_UNUSED_ITEMS;
+
/*
* We're about to remove tuples. In Hot Standby mode, ensure that there's
* no queries running for which the removed tuples are still visible.
*/
if (InHotStandby)
ResolveRecoveryConflictWithSnapshot(xlrec->snapshotConflictHorizon,
- xlrec->isCatalogRel,
+ xlrec->flags & XLHP_IS_CATALOG_REL,
rlocator);
/*
- * If we have a full-page image, restore it (using a cleanup lock) and
- * we're done.
+ * If we have a full-page image, restore it and we're done.
*/
- action = XLogReadBufferForRedoExtended(record, 0, RBM_NORMAL, true,
- &buffer);
+ action = XLogReadBufferForRedoExtended(record, 0, RBM_NORMAL,
+ get_cleanup_lock, &buffer);
+
if (action == BLK_NEEDS_REDO)
{
Page page = (Page) BufferGetPage(buffer);
- OffsetNumber *redirected;
- OffsetNumber *nowdead;
- OffsetNumber *nowunused;
- int nredirected;
- int ndead;
- int nunused;
- int nplans;
Size datalen;
- xl_heap_freeze_plan *plans;
- OffsetNumber *frz_offsets;
+ OffsetNumber *redirected = NULL;
+ OffsetNumber *nowdead = NULL;
+ OffsetNumber *nowunused = NULL;
+ int nredirected = 0;
+ int ndead = 0;
+ int nunused = 0;
+ int nplans = 0;
+ xl_heap_freeze_plan *plans = NULL;
+ OffsetNumber *frz_offsets = NULL;
int curoff = 0;
- nplans = xlrec->nplans;
- nredirected = xlrec->nredirected;
- ndead = xlrec->ndead;
- nunused = xlrec->nunused;
+ char *cursor = XLogRecGetBlockData(record, 0, &datalen);
- plans = (xl_heap_freeze_plan *) XLogRecGetBlockData(record, 0, &datalen);
- redirected = (OffsetNumber *) &plans[nplans];
- nowdead = redirected + (nredirected * 2);
- nowunused = nowdead + ndead;
- frz_offsets = nowunused + nunused;
+ heap_xlog_deserialize_prune_and_freeze(cursor, xlrec->flags,
+ &nredirected, &redirected, &ndead, &nowdead,
+ &nunused, &nowunused, &nplans, &plans, &frz_offsets);
/* Update all line pointers per the record, and repair fragmentation */
if (nredirected > 0 || ndead > 0 || nunused > 0)
@@ -8798,7 +8866,6 @@ heap_xlog_prune(XLogReaderState *record)
* Note: we don't worry about updating the page's prunability hints.
* At worst this will cause an extra prune cycle to occur soon.
*/
-
PageSetLSN(page, lsn);
MarkBufferDirty(buffer);
}
@@ -8810,7 +8877,7 @@ heap_xlog_prune(XLogReaderState *record)
UnlockReleaseBuffer(buffer);
/*
- * After pruning records from a page, it's useful to update the FSM
+ * After modifying records on a page, it's useful to update the FSM
* about it, as it may cause the page become target for insertions
* later even if vacuum decides not to visit it (which is possible if
* gets marked all-visible.)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index 6f45e5c37f0..d3643b1ecc6 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -715,15 +715,19 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
{
xl_heap_prune xlrec;
XLogRecPtr recptr;
+ xlhp_freeze freeze;
+ xlhp_prune_items redirect,
+ dead,
+ unused;
+ int nplans = 0;
xl_heap_freeze_plan plans[MaxHeapTuplesPerPage];
- OffsetNumber offsets[MaxHeapTuplesPerPage];
+ OffsetNumber frz_offsets[MaxHeapTuplesPerPage];
- xlrec.isCatalogRel = RelationIsAccessibleInLogicalDecoding(relation);
- xlrec.nredirected = prstate.nredirected;
- xlrec.ndead = prstate.ndead;
- xlrec.nunused = prstate.nunused;
- xlrec.nplans = 0;
+ xlrec.flags = 0;
+
+ if (RelationIsAccessibleInLogicalDecoding(relation))
+ xlrec.flags |= XLHP_IS_CATALOG_REL;
/*
* The snapshotConflictHorizon for the whole record should be the most
@@ -745,8 +749,11 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
* Destructively sorts tuples array in-place.
*/
if (do_freeze)
- xlrec.nplans = heap_log_freeze_plan(frozen,
- presult->nfrozen, plans, offsets);
+ nplans = heap_log_freeze_plan(frozen,
+ presult->nfrozen, plans,
+ frz_offsets);
+ if (nplans > 0)
+ xlrec.flags |= XLHP_HAS_FREEZE_PLANS;
XLogBeginInsert();
XLogRegisterData((char *) &xlrec, SizeOfHeapPrune);
@@ -758,26 +765,71 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
* pretend that they are. When XLogInsert stores the whole buffer,
* the offset arrays need not be stored too.
*/
- if (xlrec.nplans > 0)
+ if (nplans > 0)
+ {
+ freeze = (xlhp_freeze)
+ {
+ .nplans = nplans
+ };
+
+ XLogRegisterBufData(0, (char *) &freeze, offsetof(xlhp_freeze, plans));
+
XLogRegisterBufData(0, (char *) plans,
- xlrec.nplans * sizeof(xl_heap_freeze_plan));
+ sizeof(xl_heap_freeze_plan) * freeze.nplans);
+ }
+
if (prstate.nredirected > 0)
+ {
+ xlrec.flags |= XLHP_HAS_REDIRECTIONS;
+
+ redirect = (xlhp_prune_items)
+ {
+ .ntargets = prstate.nredirected
+ };
+
+ XLogRegisterBufData(0, (char *) &redirect,
+ offsetof(xlhp_prune_items, data));
+
XLogRegisterBufData(0, (char *) prstate.redirected,
- prstate.nredirected *
- sizeof(OffsetNumber) * 2);
+ sizeof(OffsetNumber[2]) * prstate.nredirected);
+ }
if (prstate.ndead > 0)
+ {
+ xlrec.flags |= XLHP_HAS_DEAD_ITEMS;
+
+ dead = (xlhp_prune_items)
+ {
+ .ntargets = prstate.ndead
+ };
+
+ XLogRegisterBufData(0, (char *) &dead,
+ offsetof(xlhp_prune_items, data));
+
XLogRegisterBufData(0, (char *) prstate.nowdead,
- prstate.ndead * sizeof(OffsetNumber));
+ sizeof(OffsetNumber) * dead.ntargets);
+ }
if (prstate.nunused > 0)
+ {
+ xlrec.flags |= XLHP_HAS_NOW_UNUSED_ITEMS;
+
+ unused = (xlhp_prune_items)
+ {
+ .ntargets = prstate.nunused
+ };
+
+ XLogRegisterBufData(0, (char *) &unused,
+ offsetof(xlhp_prune_items, data));
+
XLogRegisterBufData(0, (char *) prstate.nowunused,
- prstate.nunused * sizeof(OffsetNumber));
+ sizeof(OffsetNumber) * unused.ntargets);
+ }
- if (xlrec.nplans > 0)
- XLogRegisterBufData(0, (char *) offsets,
- presult->nfrozen * sizeof(OffsetNumber));
+ if (nplans > 0)
+ XLogRegisterBufData(0, (char *) frz_offsets,
+ sizeof(OffsetNumber) * presult->nfrozen);
recptr = XLogInsert(RM_HEAP2_ID, XLOG_HEAP2_PRUNE);
diff --git a/src/backend/access/rmgrdesc/heapdesc.c b/src/backend/access/rmgrdesc/heapdesc.c
index 36a3d83c8c2..462b0d74f80 100644
--- a/src/backend/access/rmgrdesc/heapdesc.c
+++ b/src/backend/access/rmgrdesc/heapdesc.c
@@ -179,43 +179,109 @@ heap2_desc(StringInfo buf, XLogReaderState *record)
{
xl_heap_prune *xlrec = (xl_heap_prune *) rec;
- appendStringInfo(buf, "snapshotConflictHorizon: %u, nredirected: %u, ndead: %u, isCatalogRel: %c",
+ appendStringInfo(buf, "snapshotConflictHorizon: %u, isCatalogRel: %c",
xlrec->snapshotConflictHorizon,
- xlrec->nredirected,
- xlrec->ndead,
- xlrec->isCatalogRel ? 'T' : 'F');
+ xlrec->flags & XLHP_IS_CATALOG_REL ? 'T' : 'F');
if (XLogRecHasBlockData(record, 0))
{
- OffsetNumber *end;
- OffsetNumber *redirected;
- OffsetNumber *nowdead;
- OffsetNumber *nowunused;
- int nredirected;
- int nunused;
Size datalen;
-
- redirected = (OffsetNumber *) XLogRecGetBlockData(record, 0,
- &datalen);
-
- nredirected = xlrec->nredirected;
- end = (OffsetNumber *) ((char *) redirected + datalen);
- nowdead = redirected + (nredirected * 2);
- nowunused = nowdead + xlrec->ndead;
- nunused = (end - nowunused);
- Assert(nunused >= 0);
-
- appendStringInfo(buf, ", nunused: %d", nunused);
-
- appendStringInfoString(buf, ", redirected:");
- array_desc(buf, redirected, sizeof(OffsetNumber) * 2,
- nredirected, &redirect_elem_desc, NULL);
- appendStringInfoString(buf, ", dead:");
- array_desc(buf, nowdead, sizeof(OffsetNumber), xlrec->ndead,
- &offset_elem_desc, NULL);
- appendStringInfoString(buf, ", unused:");
- array_desc(buf, nowunused, sizeof(OffsetNumber), nunused,
- &offset_elem_desc, NULL);
+ OffsetNumber *redirected = NULL;
+ OffsetNumber *nowdead = NULL;
+ OffsetNumber *nowunused = NULL;
+ int nredirected = 0;
+ int nunused = 0;
+ int ndead = 0;
+ int nplans = 0;
+ xl_heap_freeze_plan *plans = NULL;
+ OffsetNumber *frz_offsets;
+
+ char *cursor = XLogRecGetBlockData(record, 0, &datalen);
+
+ if (xlrec->flags & XLHP_HAS_FREEZE_PLANS)
+ {
+ xlhp_freeze *freeze = (xlhp_freeze *) cursor;
+
+ nplans = freeze->nplans;
+ Assert(nplans > 0);
+ plans = freeze->plans;
+
+ cursor += offsetof(xlhp_freeze, plans);
+ cursor += sizeof(xl_heap_freeze_plan) * freeze->nplans;
+ }
+
+ if (xlrec->flags & XLHP_HAS_REDIRECTIONS)
+ {
+ xlhp_prune_items *subrecord = (xlhp_prune_items *) cursor;
+
+ nredirected = subrecord->ntargets;
+ Assert(nredirected > 0);
+ redirected = &subrecord->data[0];
+
+ cursor += offsetof(xlhp_prune_items, data);
+ cursor += sizeof(OffsetNumber[2]) * nredirected;
+ }
+
+ if (xlrec->flags & XLHP_HAS_DEAD_ITEMS)
+ {
+ xlhp_prune_items *subrecord = (xlhp_prune_items *) cursor;
+
+ ndead = subrecord->ntargets;
+ Assert(ndead > 0);
+ nowdead = subrecord->data;
+
+ cursor += offsetof(xlhp_prune_items, data);
+ cursor += sizeof(OffsetNumber) * ndead;
+ }
+
+ if (xlrec->flags & XLHP_HAS_NOW_UNUSED_ITEMS)
+ {
+ xlhp_prune_items *subrecord = (xlhp_prune_items *) cursor;
+
+ nunused = subrecord->ntargets;
+ Assert(nunused > 0);
+ nowunused = subrecord->data;
+
+ cursor += offsetof(xlhp_prune_items, data);
+ cursor += sizeof(OffsetNumber) * nunused;
+ }
+
+ if (nplans > 0)
+ frz_offsets = (OffsetNumber *) cursor;
+
+ appendStringInfo(buf, ", nredirected: %u, ndead: %u, nunused: %u, nplans: %u,",
+ nredirected,
+ ndead,
+ nunused,
+ nplans);
+
+ if (nredirected > 0)
+ {
+ appendStringInfoString(buf, ", redirected:");
+ array_desc(buf, redirected, sizeof(OffsetNumber) * 2,
+ nredirected, &redirect_elem_desc, NULL);
+ }
+
+ if (ndead > 0)
+ {
+ appendStringInfoString(buf, ", dead:");
+ array_desc(buf, nowdead, sizeof(OffsetNumber), ndead,
+ &offset_elem_desc, NULL);
+ }
+
+ if (nunused > 0)
+ {
+ appendStringInfoString(buf, ", unused:");
+ array_desc(buf, nowunused, sizeof(OffsetNumber), nunused,
+ &offset_elem_desc, NULL);
+ }
+
+ if (nplans > 0)
+ {
+ appendStringInfoString(buf, ", plans:");
+ array_desc(buf, plans, sizeof(xl_heap_freeze_plan), nplans,
+ &plan_elem_desc, &frz_offsets);
+ }
}
}
else if (info == XLOG_HEAP2_VACUUM)
diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h
index 22f236bb52a..bebd93422d5 100644
--- a/src/include/access/heapam_xlog.h
+++ b/src/include/access/heapam_xlog.h
@@ -227,42 +227,84 @@ typedef struct xl_heap_update
#define SizeOfHeapUpdate (offsetof(xl_heap_update, new_offnum) + sizeof(OffsetNumber))
/*
- * This is what we need to know about page pruning (both during VACUUM and
- * during opportunistic pruning)
+ * XXX: As of Postgres 17, XLOG_HEAP2_PRUNE records replace
+ * XLOG_HEAP2_FREEZE_PAGE record types
+ */
+
+/*
+ * This is what we need to know about page pruning and freezing, both during
+ * VACUUM and during opportunistic pruning.
*
- * The array of OffsetNumbers following the fixed part of the record contains:
- * * for each freeze plan: the freeze plan
- * * for each redirected item: the item offset, then the offset redirected to
- * * for each now-dead item: the item offset
- * * for each now-unused item: the item offset
- * * for each tuple frozen by the freeze plans: the offset of the item corresponding to that tuple
- * The total number of OffsetNumbers is therefore
- * (2*nredirected) + ndead + nunused + (sum[plan.ntuples for plan in plans])
+ * If XLPH_HAS_REDIRECTIONS, XLHP_HAS_DEAD_ITEMS, or XLHP_HAS_NOW_UNUSED is set,
+ * acquires a full cleanup lock. Otherwise an ordinary exclusive lock is
+ * enough. This can happen if freezing was the only modification to the page.
*
- * Acquires a full cleanup lock.
+ * The data for block reference 0 contains "sub-records" depending on which
+ * of the XLHP_HAS_* flags are set. See xlhp_* struct definitions below.
+ *
+ * The layout is in the same order as the XLHP_* flags.
*/
typedef struct xl_heap_prune
{
TransactionId snapshotConflictHorizon;
- uint16 nplans;
- uint16 nredirected;
- uint16 ndead;
- uint16 nunused;
- bool isCatalogRel; /* to handle recovery conflict during logical
- * decoding on standby */
- /*
- * OFFSET NUMBERS and freeze plans are in the block reference 0 in the
- * following order:
- *
- * * xl_heap_freeze_plan plans[nplans];
- * * OffsetNumber redirected[2 * nredirected];
- * * OffsetNumber nowdead[ndead];
- * * OffsetNumber nowunused[nunused];
- * * OffsetNumber frz_offsets[...];
- */
+ uint8 flags;
} xl_heap_prune;
-#define SizeOfHeapPrune (offsetof(xl_heap_prune, isCatalogRel) + sizeof(bool))
+#define XLHP_IS_CATALOG_REL 0x01 /* to handle recovery conflict
+ * during logical decoding on
+ * standby */
+#define XLHP_HAS_FREEZE_PLANS 0x02
+#define XLHP_HAS_REDIRECTIONS 0x04
+#define XLHP_HAS_DEAD_ITEMS 0x08
+#define XLHP_HAS_NOW_UNUSED_ITEMS 0x10
+
+#define SizeOfHeapPrune (offsetof(xl_heap_prune, flags) + sizeof(uint8))
+
+/*
+ * This struct represents a 'freeze plan', which describes how to freeze a
+ * group of one or more heap tuples (appears in xl_heap_freeze_page and
+ * xl_heap_prune's xlhp_freeze records)
+ */
+/* 0x01 was XLH_FREEZE_XMIN */
+#define XLH_FREEZE_XVAC 0x02
+#define XLH_INVALID_XVAC 0x04
+
+typedef struct xl_heap_freeze_plan
+{
+ TransactionId xmax;
+ uint16 t_infomask2;
+ uint16 t_infomask;
+ uint8 frzflags;
+
+ /* Length of individual page offset numbers array for this plan */
+ uint16 ntuples;
+} xl_heap_freeze_plan;
+
+/*
+ * This is what we need to know about a block being frozen during vacuum
+ *
+ * Backup block 0's data contains an array of xl_heap_freeze_plan structs
+ * (with nplans elements), followed by one or more page offset number arrays.
+ * Each such page offset number array corresponds to a single freeze plan
+ * (REDO routine freezes corresponding heap tuples using freeze plan).
+ */
+typedef struct xlhp_freeze
+{
+ uint16 nplans;
+ xl_heap_freeze_plan plans[FLEXIBLE_ARRAY_MEMBER];
+} xlhp_freeze;
+
+/*
+ * Sub-record type contained in block reference 0 of a prune record if
+ * XLHP_HAS_REDIRECTIONS/XLHP_HAS_DEAD_ITEMS/XLHP_HAS_NOW_UNUSED_ITEMS is set.
+ * Note that in the XLHP_HAS_REDIRECTIONS variant, there are actually 2 *
+ * length number of OffsetNumbers in the data.
+ */
+typedef struct xlhp_prune_items
+{
+ uint16 ntargets;
+ OffsetNumber data[FLEXIBLE_ARRAY_MEMBER];
+} xlhp_prune_items;
/*
* The vacuum page record is similar to the prune record, but can only mark
@@ -326,26 +368,6 @@ typedef struct xl_heap_inplace
} xl_heap_inplace;
#define SizeOfHeapInplace (offsetof(xl_heap_inplace, offnum) + sizeof(OffsetNumber))
-
-/*
- * This struct represents a 'freeze plan', which describes how to freeze a
- * group of one or more heap tuples (appears in xl_heap_freeze_page record)
- */
-/* 0x01 was XLH_FREEZE_XMIN */
-#define XLH_FREEZE_XVAC 0x02
-#define XLH_INVALID_XVAC 0x04
-
-typedef struct xl_heap_freeze_plan
-{
- TransactionId xmax;
- uint16 t_infomask2;
- uint16 t_infomask;
- uint8 frzflags;
-
- /* Length of individual page offset numbers array for this plan */
- uint16 ntuples;
-} xl_heap_freeze_plan;
-
/*
* This is what we need to know about a block being frozen during vacuum
*
@@ -353,6 +375,10 @@ typedef struct xl_heap_freeze_plan
* (with nplans elements), followed by one or more page offset number arrays.
* Each such page offset number array corresponds to a single freeze plan
* (REDO routine freezes corresponding heap tuples using freeze plan).
+ *
+ * This is for backwards compatability for reading individual freeze records.
+ * As of Postgres 17, xl_heap_freeze_plan records occur in xl_heap_prune
+ * records.
*/
typedef struct xl_heap_freeze_page
{
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 1c1a4d305d6..2702f211d90 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -4002,6 +4002,8 @@ xl_xact_stats_items
xl_xact_subxacts
xl_xact_twophase
xl_xact_xinfo
+xlhp_freeze
+xlhp_prune_items
xmlBuffer
xmlBufferPtr
xmlChar
--
2.40.1
view thread (4+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: Combine Prune and Freeze records emitted by vacuum
In-Reply-To: <20240315005658.y7bvbzvnlciqmhd6@liskov>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox