public inbox for [email protected]
help / color / mirror / Atom feedRe: PG 16 draft release notes ready
14+ messages / 3 participants
[nested] [flat]
* Re: PG 16 draft release notes ready
@ 2023-05-19 02:41 jian he <[email protected]>
0 siblings, 1 reply; 14+ messages in thread
From: jian he @ 2023-05-19 02:41 UTC (permalink / raw)
To: Bruce Momjian <[email protected]>; +Cc: pgsql-hackers
On Fri, May 19, 2023 at 4:49 AM Bruce Momjian <[email protected]> wrote:
> I have completed the first draft of the PG 16 release notes. You can
> see the output here:
>
> https://momjian.us/pgsql_docs/release-16.html
>
> I will adjust it to the feedback I receive; that URL will quickly show
> all updates.
>
> I learned a few things creating it this time:
>
> * I can get confused over C function names and SQL function names in
> commit messages.
>
> * The sections and ordering of the entries can greatly clarify the
> items.
>
> * The feature count is slightly higher than recent releases:
>
> release-10: 189
> release-11: 170
> release-12: 180
> release-13: 178
> release-14: 220
> release-15: 184
> --> release-16: 200
>
> --
> Bruce Momjian <[email protected]> https://momjian.us
> EDB https://enterprisedb.com
>
> Only you can decide what is important to you.
* When granting role membership, require the granted-by role to be a role
> that has appropriate permissions (Robert Haas)
> This is a requirement even when the superuser is granting role membership.
an exception would be the granted-by is the bootstrap superuser.
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: PG 16 draft release notes ready
@ 2023-05-19 03:15 Bruce Momjian <[email protected]>
parent: jian he <[email protected]>
0 siblings, 0 replies; 14+ messages in thread
From: Bruce Momjian @ 2023-05-19 03:15 UTC (permalink / raw)
To: jian he <[email protected]>; +Cc: pgsql-hackers
On Fri, May 19, 2023 at 10:41:59AM +0800, jian he wrote:
> On Fri, May 19, 2023 at 4:49 AM Bruce Momjian <[email protected]> wrote:
> * When granting role membership, require the granted-by role to be a role
> that has appropriate permissions (Robert Haas)
> This is a requirement even when the superuser is granting role membership.
>
>
> an exception would be the granted-by is the bootstrap superuser.
Okay, updated text is:
<listitem>
<para>
When granting role membership, require the granted-by role to be a role that has appropriate permissions (Robert Haas)
</para>
<para>
This is a requirement even when a non-bootstrap superuser is granting role membership.
</para>
</listitem>
--
Bruce Momjian <[email protected]> https://momjian.us
EDB https://enterprisedb.com
Only you can decide what is important to you.
^ permalink raw reply [nested|flat] 14+ messages in thread
* [PATCH v3 08/17] Make opp freeze heuristic compatible with prune+freeze record
@ 2024-01-07 21:11 Melanie Plageman <[email protected]>
0 siblings, 0 replies; 14+ messages in thread
From: Melanie Plageman @ 2024-01-07 21:11 UTC (permalink / raw)
Once the prune and freeze records are combined, we will no longer be
able to use a test of whether or not pruning emitted an FPI to decide
whether or not to opportunistically freeze a freezable page.
While this heuristic should be improved, for now, approximate the
previous logic by keeping track of whether or not a hint bit FPI was
emitted during visibility checks (when checksums are on) and combine
that with checking XLogCheckBufferNeedsBackup(). If we just finished
deciding whether or not to prune and the current buffer seems to need an
FPI after modification, it is likely that pruning would have emitted an
FPI.
---
src/backend/access/heap/pruneheap.c | 58 +++++++++++++++++++++--------
1 file changed, 43 insertions(+), 15 deletions(-)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index abf6bdb2d99..f164b7957ed 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -255,6 +255,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
PruneState prstate;
HeapTupleData tup;
bool do_freeze;
+ bool do_prune;
+ bool whole_page_freezable;
+ bool hint_bit_fpi;
+ bool prune_fpi = false;
int64 fpi_before = pgWalUsage.wal_fpi;
TransactionId frz_conflict_horizon = InvalidTransactionId;
@@ -424,6 +428,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
}
}
+ /*
+ * If checksums are enabled, heap_prune_satisfies_vacuum() may have caused
+ * an FPI to be emitted. Then reset fpi_before for no prune case.
+ */
+ hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi;
+ fpi_before = pgWalUsage.wal_fpi;
+
/*
* For vacuum, if the whole page will become frozen, we consider
* opportunistically freezing tuples. Dead tuples which will be removed by
@@ -481,11 +492,42 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
if (off_loc)
*off_loc = InvalidOffsetNumber;
+ do_prune = prstate.nredirected > 0 ||
+ prstate.ndead > 0 ||
+ prstate.nunused > 0;
+
+ /*
+ * Only incur overhead of checking if we will do an FPI if we might use
+ * the information.
+ */
+ if (do_prune && pagefrz)
+ prune_fpi = XLogCheckBufferNeedsBackup(buffer);
+
+ /* Is the whole page freezable? And is there something to freeze */
+ whole_page_freezable = presult->all_visible_except_removable &&
+ presult->all_frozen;
+
+ /*
+ * Freeze the page when heap_prepare_freeze_tuple indicates that at least
+ * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
+ * freeze when pruning generated an FPI, if doing so means that we set the
+ * page all-frozen afterwards (might not happen until final heap pass).
+ * XXX: Previously, we knew if pruning emitted an FPI by checking
+ * pgWalUsage.wal_fpi before and after pruning. Once the freeze and prune
+ * records are combined, this heuristic couldn't be used anymore. The
+ * opportunistic freeze heuristic must be improved; however, for now, try
+ * to approximate it.
+ */
+
+ do_freeze = pagefrz &&
+ (pagefrz->freeze_required ||
+ (whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi)));
+
/* Any error while applying the changes is critical */
START_CRIT_SECTION();
/* Have we found any prunable items? */
- if (prstate.nredirected > 0 || prstate.ndead > 0 || prstate.nunused > 0)
+ if (do_prune)
{
/*
* Apply the planned item changes, then repair page fragmentation, and
@@ -577,20 +619,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
/* Record number of newly-set-LP_DEAD items for caller */
presult->nnewlpdead = prstate.ndead;
- /*
- * Freeze the page when heap_prepare_freeze_tuple indicates that at least
- * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
- * freeze when pruning generated an FPI, if doing so means that we set the
- * page all-frozen afterwards (might not happen until final heap pass).
- */
- if (pagefrz)
- do_freeze = pagefrz->freeze_required ||
- (presult->all_visible_except_removable && presult->all_frozen &&
- presult->nfrozen > 0 &&
- fpi_before != pgWalUsage.wal_fpi);
- else
- do_freeze = false;
-
if (do_freeze)
{
frz_conflict_horizon = heap_frz_conflict_horizon(presult, pagefrz);
--
2.40.1
--racicctn4wry6xe5
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v3-0009-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch"
^ permalink raw reply [nested|flat] 14+ messages in thread
* [PATCH v4 10/19] Make opp freeze heuristic compatible with prune+freeze record
@ 2024-01-07 21:11 Melanie Plageman <[email protected]>
0 siblings, 0 replies; 14+ messages in thread
From: Melanie Plageman @ 2024-01-07 21:11 UTC (permalink / raw)
Once the prune and freeze records are combined, we will no longer be
able to use a test of whether or not pruning emitted an FPI to decide
whether or not to opportunistically freeze a freezable page.
While this heuristic should be improved, for now, approximate the
previous logic by keeping track of whether or not a hint bit FPI was
emitted during visibility checks (when checksums are on) and combine
that with checking XLogCheckBufferNeedsBackup(). If we just finished
deciding whether or not to prune and the current buffer seems to need an
FPI after modification, it is likely that pruning would have emitted an
FPI.
---
src/backend/access/heap/pruneheap.c | 57 +++++++++++++++++++++--------
1 file changed, 42 insertions(+), 15 deletions(-)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index 20907ba5408..9edf6bf72d7 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -246,6 +246,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
PruneState prstate;
HeapTupleData tup;
bool do_freeze;
+ bool do_prune;
+ bool whole_page_freezable;
+ bool hint_bit_fpi;
+ bool prune_fpi = false;
int64 fpi_before = pgWalUsage.wal_fpi;
/*
@@ -439,6 +443,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
}
}
+ /*
+ * If checksums are enabled, heap_prune_satisfies_vacuum() may have caused
+ * an FPI to be emitted. Then reset fpi_before for no prune case.
+ */
+ hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi;
+ fpi_before = pgWalUsage.wal_fpi;
+
/*
* For vacuum, if the whole page will become frozen, we consider
* opportunistically freezing tuples. Dead tuples which will be removed by
@@ -483,11 +494,41 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
if (off_loc)
*off_loc = InvalidOffsetNumber;
+ do_prune = prstate.nredirected > 0 ||
+ prstate.ndead > 0 ||
+ prstate.nunused > 0;
+
+ /*
+ * Only incur overhead of checking if we will do an FPI if we might use
+ * the information.
+ */
+ if (do_prune && pagefrz)
+ prune_fpi = XLogCheckBufferNeedsBackup(buffer);
+
+ /* Is the whole page freezable? And is there something to freeze */
+ whole_page_freezable = presult->all_visible_except_removable &&
+ presult->all_frozen;
+
+ /*
+ * Freeze the page when heap_prepare_freeze_tuple indicates that at least
+ * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
+ * freeze when pruning generated an FPI, if doing so means that we set the
+ * page all-frozen afterwards (might not happen until final heap pass).
+ * XXX: Previously, we knew if pruning emitted an FPI by checking
+ * pgWalUsage.wal_fpi before and after pruning. Once the freeze and prune
+ * records are combined, this heuristic couldn't be used anymore. The
+ * opportunistic freeze heuristic must be improved; however, for now, try
+ * to approximate it.
+ */
+ do_freeze = pagefrz &&
+ (pagefrz->freeze_required ||
+ (whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi)));
+
/* Any error while applying the changes is critical */
START_CRIT_SECTION();
/* Have we found any prunable items? */
- if (prstate.nredirected > 0 || prstate.ndead > 0 || prstate.nunused > 0)
+ if (do_prune)
{
/*
* Apply the planned item changes, then repair page fragmentation, and
@@ -579,20 +620,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
/* Record number of newly-set-LP_DEAD items for caller */
presult->nnewlpdead = prstate.ndead;
- /*
- * Freeze the page when heap_prepare_freeze_tuple indicates that at least
- * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
- * freeze when pruning generated an FPI, if doing so means that we set the
- * page all-frozen afterwards (might not happen until final heap pass).
- */
- if (pagefrz)
- do_freeze = pagefrz->freeze_required ||
- (presult->all_visible_except_removable && presult->all_frozen &&
- presult->nfrozen > 0 &&
- fpi_before != pgWalUsage.wal_fpi);
- else
- do_freeze = false;
-
if (do_freeze)
{
/*
--
2.40.1
--tez7m2a73jtztiij
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0011-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch"
^ permalink raw reply [nested|flat] 14+ messages in thread
* [PATCH v2 08/17] Make opp freeze heuristic compatible with prune+freeze record
@ 2024-01-07 21:11 Melanie Plageman <[email protected]>
0 siblings, 0 replies; 14+ messages in thread
From: Melanie Plageman @ 2024-01-07 21:11 UTC (permalink / raw)
Once the prune and freeze records are combined, we will no longer be
able to use a test of whether or not pruning emitted an FPI to decide
whether or not to opportunistically freeze a freezable page.
While this heuristic should be improved, for now, approximate the
previous logic by keeping track of whether or not a hint bit FPI was
emitted during visibility checks (when checksums are on) and combine
that with checking XLogCheckBufferNeedsBackup(). If we just finished
deciding whether or not to prune and the current buffer seems to need an
FPI after modification, it is likely that pruning would have emitted an
FPI.
---
src/backend/access/heap/pruneheap.c | 58 +++++++++++++++++++++--------
1 file changed, 43 insertions(+), 15 deletions(-)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index 9c709315192..e715fc29a83 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -241,6 +241,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
PruneState prstate;
HeapTupleData tup;
bool do_freeze;
+ bool do_prune;
+ bool whole_page_freezable;
+ bool hint_bit_fpi;
+ bool prune_fpi = false;
int64 fpi_before = pgWalUsage.wal_fpi;
TransactionId frz_conflict_horizon = InvalidTransactionId;
@@ -410,6 +414,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
}
}
+ /*
+ * If checksums are enabled, heap_prune_satisfies_vacuum() may have caused
+ * an FPI to be emitted. Then reset fpi_before for no prune case.
+ */
+ hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi;
+ fpi_before = pgWalUsage.wal_fpi;
+
/*
* For vacuum, if the whole page will become frozen, we consider
* opportunistically freezing tuples. Dead tuples which will be removed by
@@ -467,11 +478,42 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
if (off_loc)
*off_loc = InvalidOffsetNumber;
+ do_prune = prstate.nredirected > 0 ||
+ prstate.ndead > 0 ||
+ prstate.nunused > 0;
+
+ /*
+ * Only incur overhead of checking if we will do an FPI if we might use
+ * the information.
+ */
+ if (do_prune && pagefrz)
+ prune_fpi = XLogCheckBufferNeedsBackup(buffer);
+
+ /* Is the whole page freezable? And is there something to freeze */
+ whole_page_freezable = presult->all_visible_except_removable &&
+ presult->all_frozen;
+
+ /*
+ * Freeze the page when heap_prepare_freeze_tuple indicates that at least
+ * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
+ * freeze when pruning generated an FPI, if doing so means that we set the
+ * page all-frozen afterwards (might not happen until final heap pass).
+ * XXX: Previously, we knew if pruning emitted an FPI by checking
+ * pgWalUsage.wal_fpi before and after pruning. Once the freeze and prune
+ * records are combined, this heuristic couldn't be used anymore. The
+ * opportunistic freeze heuristic must be improved; however, for now, try
+ * to approximate it.
+ */
+
+ do_freeze = pagefrz &&
+ (pagefrz->freeze_required ||
+ (whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi)));
+
/* Any error while applying the changes is critical */
START_CRIT_SECTION();
/* Have we found any prunable items? */
- if (prstate.nredirected > 0 || prstate.ndead > 0 || prstate.nunused > 0)
+ if (do_prune)
{
/*
* Apply the planned item changes, then repair page fragmentation, and
@@ -563,20 +605,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
/* Record number of newly-set-LP_DEAD items for caller */
presult->nnewlpdead = prstate.ndead;
- /*
- * Freeze the page when heap_prepare_freeze_tuple indicates that at least
- * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
- * freeze when pruning generated an FPI, if doing so means that we set the
- * page all-frozen afterwards (might not happen until final heap pass).
- */
- if (pagefrz)
- do_freeze = pagefrz->freeze_required ||
- (presult->all_visible_except_removable && presult->all_frozen &&
- presult->nfrozen > 0 &&
- fpi_before != pgWalUsage.wal_fpi);
- else
- do_freeze = false;
-
if (do_freeze)
{
frz_conflict_horizon = heap_frz_conflict_horizon(presult, pagefrz);
--
2.40.1
--rdqtp5puvxqotfdw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v2-0009-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch"
^ permalink raw reply [nested|flat] 14+ messages in thread
* [PATCH v3 08/17] Make opp freeze heuristic compatible with prune+freeze record
@ 2024-01-07 21:11 Melanie Plageman <[email protected]>
0 siblings, 0 replies; 14+ messages in thread
From: Melanie Plageman @ 2024-01-07 21:11 UTC (permalink / raw)
Once the prune and freeze records are combined, we will no longer be
able to use a test of whether or not pruning emitted an FPI to decide
whether or not to opportunistically freeze a freezable page.
While this heuristic should be improved, for now, approximate the
previous logic by keeping track of whether or not a hint bit FPI was
emitted during visibility checks (when checksums are on) and combine
that with checking XLogCheckBufferNeedsBackup(). If we just finished
deciding whether or not to prune and the current buffer seems to need an
FPI after modification, it is likely that pruning would have emitted an
FPI.
---
src/backend/access/heap/pruneheap.c | 58 +++++++++++++++++++++--------
1 file changed, 43 insertions(+), 15 deletions(-)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index abf6bdb2d99..f164b7957ed 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -255,6 +255,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
PruneState prstate;
HeapTupleData tup;
bool do_freeze;
+ bool do_prune;
+ bool whole_page_freezable;
+ bool hint_bit_fpi;
+ bool prune_fpi = false;
int64 fpi_before = pgWalUsage.wal_fpi;
TransactionId frz_conflict_horizon = InvalidTransactionId;
@@ -424,6 +428,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
}
}
+ /*
+ * If checksums are enabled, heap_prune_satisfies_vacuum() may have caused
+ * an FPI to be emitted. Then reset fpi_before for no prune case.
+ */
+ hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi;
+ fpi_before = pgWalUsage.wal_fpi;
+
/*
* For vacuum, if the whole page will become frozen, we consider
* opportunistically freezing tuples. Dead tuples which will be removed by
@@ -481,11 +492,42 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
if (off_loc)
*off_loc = InvalidOffsetNumber;
+ do_prune = prstate.nredirected > 0 ||
+ prstate.ndead > 0 ||
+ prstate.nunused > 0;
+
+ /*
+ * Only incur overhead of checking if we will do an FPI if we might use
+ * the information.
+ */
+ if (do_prune && pagefrz)
+ prune_fpi = XLogCheckBufferNeedsBackup(buffer);
+
+ /* Is the whole page freezable? And is there something to freeze */
+ whole_page_freezable = presult->all_visible_except_removable &&
+ presult->all_frozen;
+
+ /*
+ * Freeze the page when heap_prepare_freeze_tuple indicates that at least
+ * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
+ * freeze when pruning generated an FPI, if doing so means that we set the
+ * page all-frozen afterwards (might not happen until final heap pass).
+ * XXX: Previously, we knew if pruning emitted an FPI by checking
+ * pgWalUsage.wal_fpi before and after pruning. Once the freeze and prune
+ * records are combined, this heuristic couldn't be used anymore. The
+ * opportunistic freeze heuristic must be improved; however, for now, try
+ * to approximate it.
+ */
+
+ do_freeze = pagefrz &&
+ (pagefrz->freeze_required ||
+ (whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi)));
+
/* Any error while applying the changes is critical */
START_CRIT_SECTION();
/* Have we found any prunable items? */
- if (prstate.nredirected > 0 || prstate.ndead > 0 || prstate.nunused > 0)
+ if (do_prune)
{
/*
* Apply the planned item changes, then repair page fragmentation, and
@@ -577,20 +619,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
/* Record number of newly-set-LP_DEAD items for caller */
presult->nnewlpdead = prstate.ndead;
- /*
- * Freeze the page when heap_prepare_freeze_tuple indicates that at least
- * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
- * freeze when pruning generated an FPI, if doing so means that we set the
- * page all-frozen afterwards (might not happen until final heap pass).
- */
- if (pagefrz)
- do_freeze = pagefrz->freeze_required ||
- (presult->all_visible_except_removable && presult->all_frozen &&
- presult->nfrozen > 0 &&
- fpi_before != pgWalUsage.wal_fpi);
- else
- do_freeze = false;
-
if (do_freeze)
{
frz_conflict_horizon = heap_frz_conflict_horizon(presult, pagefrz);
--
2.40.1
--racicctn4wry6xe5
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v3-0009-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch"
^ permalink raw reply [nested|flat] 14+ messages in thread
* [PATCH v2 08/17] Make opp freeze heuristic compatible with prune+freeze record
@ 2024-01-07 21:11 Melanie Plageman <[email protected]>
0 siblings, 0 replies; 14+ messages in thread
From: Melanie Plageman @ 2024-01-07 21:11 UTC (permalink / raw)
Once the prune and freeze records are combined, we will no longer be
able to use a test of whether or not pruning emitted an FPI to decide
whether or not to opportunistically freeze a freezable page.
While this heuristic should be improved, for now, approximate the
previous logic by keeping track of whether or not a hint bit FPI was
emitted during visibility checks (when checksums are on) and combine
that with checking XLogCheckBufferNeedsBackup(). If we just finished
deciding whether or not to prune and the current buffer seems to need an
FPI after modification, it is likely that pruning would have emitted an
FPI.
---
src/backend/access/heap/pruneheap.c | 58 +++++++++++++++++++++--------
1 file changed, 43 insertions(+), 15 deletions(-)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index 9c709315192..e715fc29a83 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -241,6 +241,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
PruneState prstate;
HeapTupleData tup;
bool do_freeze;
+ bool do_prune;
+ bool whole_page_freezable;
+ bool hint_bit_fpi;
+ bool prune_fpi = false;
int64 fpi_before = pgWalUsage.wal_fpi;
TransactionId frz_conflict_horizon = InvalidTransactionId;
@@ -410,6 +414,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
}
}
+ /*
+ * If checksums are enabled, heap_prune_satisfies_vacuum() may have caused
+ * an FPI to be emitted. Then reset fpi_before for no prune case.
+ */
+ hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi;
+ fpi_before = pgWalUsage.wal_fpi;
+
/*
* For vacuum, if the whole page will become frozen, we consider
* opportunistically freezing tuples. Dead tuples which will be removed by
@@ -467,11 +478,42 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
if (off_loc)
*off_loc = InvalidOffsetNumber;
+ do_prune = prstate.nredirected > 0 ||
+ prstate.ndead > 0 ||
+ prstate.nunused > 0;
+
+ /*
+ * Only incur overhead of checking if we will do an FPI if we might use
+ * the information.
+ */
+ if (do_prune && pagefrz)
+ prune_fpi = XLogCheckBufferNeedsBackup(buffer);
+
+ /* Is the whole page freezable? And is there something to freeze */
+ whole_page_freezable = presult->all_visible_except_removable &&
+ presult->all_frozen;
+
+ /*
+ * Freeze the page when heap_prepare_freeze_tuple indicates that at least
+ * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
+ * freeze when pruning generated an FPI, if doing so means that we set the
+ * page all-frozen afterwards (might not happen until final heap pass).
+ * XXX: Previously, we knew if pruning emitted an FPI by checking
+ * pgWalUsage.wal_fpi before and after pruning. Once the freeze and prune
+ * records are combined, this heuristic couldn't be used anymore. The
+ * opportunistic freeze heuristic must be improved; however, for now, try
+ * to approximate it.
+ */
+
+ do_freeze = pagefrz &&
+ (pagefrz->freeze_required ||
+ (whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi)));
+
/* Any error while applying the changes is critical */
START_CRIT_SECTION();
/* Have we found any prunable items? */
- if (prstate.nredirected > 0 || prstate.ndead > 0 || prstate.nunused > 0)
+ if (do_prune)
{
/*
* Apply the planned item changes, then repair page fragmentation, and
@@ -563,20 +605,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
/* Record number of newly-set-LP_DEAD items for caller */
presult->nnewlpdead = prstate.ndead;
- /*
- * Freeze the page when heap_prepare_freeze_tuple indicates that at least
- * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
- * freeze when pruning generated an FPI, if doing so means that we set the
- * page all-frozen afterwards (might not happen until final heap pass).
- */
- if (pagefrz)
- do_freeze = pagefrz->freeze_required ||
- (presult->all_visible_except_removable && presult->all_frozen &&
- presult->nfrozen > 0 &&
- fpi_before != pgWalUsage.wal_fpi);
- else
- do_freeze = false;
-
if (do_freeze)
{
frz_conflict_horizon = heap_frz_conflict_horizon(presult, pagefrz);
--
2.40.1
--rdqtp5puvxqotfdw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v2-0009-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch"
^ permalink raw reply [nested|flat] 14+ messages in thread
* [PATCH v2 08/17] Make opp freeze heuristic compatible with prune+freeze record
@ 2024-01-07 21:11 Melanie Plageman <[email protected]>
0 siblings, 0 replies; 14+ messages in thread
From: Melanie Plageman @ 2024-01-07 21:11 UTC (permalink / raw)
Once the prune and freeze records are combined, we will no longer be
able to use a test of whether or not pruning emitted an FPI to decide
whether or not to opportunistically freeze a freezable page.
While this heuristic should be improved, for now, approximate the
previous logic by keeping track of whether or not a hint bit FPI was
emitted during visibility checks (when checksums are on) and combine
that with checking XLogCheckBufferNeedsBackup(). If we just finished
deciding whether or not to prune and the current buffer seems to need an
FPI after modification, it is likely that pruning would have emitted an
FPI.
---
src/backend/access/heap/pruneheap.c | 58 +++++++++++++++++++++--------
1 file changed, 43 insertions(+), 15 deletions(-)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index 9c709315192..e715fc29a83 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -241,6 +241,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
PruneState prstate;
HeapTupleData tup;
bool do_freeze;
+ bool do_prune;
+ bool whole_page_freezable;
+ bool hint_bit_fpi;
+ bool prune_fpi = false;
int64 fpi_before = pgWalUsage.wal_fpi;
TransactionId frz_conflict_horizon = InvalidTransactionId;
@@ -410,6 +414,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
}
}
+ /*
+ * If checksums are enabled, heap_prune_satisfies_vacuum() may have caused
+ * an FPI to be emitted. Then reset fpi_before for no prune case.
+ */
+ hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi;
+ fpi_before = pgWalUsage.wal_fpi;
+
/*
* For vacuum, if the whole page will become frozen, we consider
* opportunistically freezing tuples. Dead tuples which will be removed by
@@ -467,11 +478,42 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
if (off_loc)
*off_loc = InvalidOffsetNumber;
+ do_prune = prstate.nredirected > 0 ||
+ prstate.ndead > 0 ||
+ prstate.nunused > 0;
+
+ /*
+ * Only incur overhead of checking if we will do an FPI if we might use
+ * the information.
+ */
+ if (do_prune && pagefrz)
+ prune_fpi = XLogCheckBufferNeedsBackup(buffer);
+
+ /* Is the whole page freezable? And is there something to freeze */
+ whole_page_freezable = presult->all_visible_except_removable &&
+ presult->all_frozen;
+
+ /*
+ * Freeze the page when heap_prepare_freeze_tuple indicates that at least
+ * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
+ * freeze when pruning generated an FPI, if doing so means that we set the
+ * page all-frozen afterwards (might not happen until final heap pass).
+ * XXX: Previously, we knew if pruning emitted an FPI by checking
+ * pgWalUsage.wal_fpi before and after pruning. Once the freeze and prune
+ * records are combined, this heuristic couldn't be used anymore. The
+ * opportunistic freeze heuristic must be improved; however, for now, try
+ * to approximate it.
+ */
+
+ do_freeze = pagefrz &&
+ (pagefrz->freeze_required ||
+ (whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi)));
+
/* Any error while applying the changes is critical */
START_CRIT_SECTION();
/* Have we found any prunable items? */
- if (prstate.nredirected > 0 || prstate.ndead > 0 || prstate.nunused > 0)
+ if (do_prune)
{
/*
* Apply the planned item changes, then repair page fragmentation, and
@@ -563,20 +605,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
/* Record number of newly-set-LP_DEAD items for caller */
presult->nnewlpdead = prstate.ndead;
- /*
- * Freeze the page when heap_prepare_freeze_tuple indicates that at least
- * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
- * freeze when pruning generated an FPI, if doing so means that we set the
- * page all-frozen afterwards (might not happen until final heap pass).
- */
- if (pagefrz)
- do_freeze = pagefrz->freeze_required ||
- (presult->all_visible_except_removable && presult->all_frozen &&
- presult->nfrozen > 0 &&
- fpi_before != pgWalUsage.wal_fpi);
- else
- do_freeze = false;
-
if (do_freeze)
{
frz_conflict_horizon = heap_frz_conflict_horizon(presult, pagefrz);
--
2.40.1
--rdqtp5puvxqotfdw
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v2-0009-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch"
^ permalink raw reply [nested|flat] 14+ messages in thread
* [PATCH v3 08/17] Make opp freeze heuristic compatible with prune+freeze record
@ 2024-01-07 21:11 Melanie Plageman <[email protected]>
0 siblings, 0 replies; 14+ messages in thread
From: Melanie Plageman @ 2024-01-07 21:11 UTC (permalink / raw)
Once the prune and freeze records are combined, we will no longer be
able to use a test of whether or not pruning emitted an FPI to decide
whether or not to opportunistically freeze a freezable page.
While this heuristic should be improved, for now, approximate the
previous logic by keeping track of whether or not a hint bit FPI was
emitted during visibility checks (when checksums are on) and combine
that with checking XLogCheckBufferNeedsBackup(). If we just finished
deciding whether or not to prune and the current buffer seems to need an
FPI after modification, it is likely that pruning would have emitted an
FPI.
---
src/backend/access/heap/pruneheap.c | 58 +++++++++++++++++++++--------
1 file changed, 43 insertions(+), 15 deletions(-)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index abf6bdb2d99..f164b7957ed 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -255,6 +255,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
PruneState prstate;
HeapTupleData tup;
bool do_freeze;
+ bool do_prune;
+ bool whole_page_freezable;
+ bool hint_bit_fpi;
+ bool prune_fpi = false;
int64 fpi_before = pgWalUsage.wal_fpi;
TransactionId frz_conflict_horizon = InvalidTransactionId;
@@ -424,6 +428,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
}
}
+ /*
+ * If checksums are enabled, heap_prune_satisfies_vacuum() may have caused
+ * an FPI to be emitted. Then reset fpi_before for no prune case.
+ */
+ hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi;
+ fpi_before = pgWalUsage.wal_fpi;
+
/*
* For vacuum, if the whole page will become frozen, we consider
* opportunistically freezing tuples. Dead tuples which will be removed by
@@ -481,11 +492,42 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
if (off_loc)
*off_loc = InvalidOffsetNumber;
+ do_prune = prstate.nredirected > 0 ||
+ prstate.ndead > 0 ||
+ prstate.nunused > 0;
+
+ /*
+ * Only incur overhead of checking if we will do an FPI if we might use
+ * the information.
+ */
+ if (do_prune && pagefrz)
+ prune_fpi = XLogCheckBufferNeedsBackup(buffer);
+
+ /* Is the whole page freezable? And is there something to freeze */
+ whole_page_freezable = presult->all_visible_except_removable &&
+ presult->all_frozen;
+
+ /*
+ * Freeze the page when heap_prepare_freeze_tuple indicates that at least
+ * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
+ * freeze when pruning generated an FPI, if doing so means that we set the
+ * page all-frozen afterwards (might not happen until final heap pass).
+ * XXX: Previously, we knew if pruning emitted an FPI by checking
+ * pgWalUsage.wal_fpi before and after pruning. Once the freeze and prune
+ * records are combined, this heuristic couldn't be used anymore. The
+ * opportunistic freeze heuristic must be improved; however, for now, try
+ * to approximate it.
+ */
+
+ do_freeze = pagefrz &&
+ (pagefrz->freeze_required ||
+ (whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi)));
+
/* Any error while applying the changes is critical */
START_CRIT_SECTION();
/* Have we found any prunable items? */
- if (prstate.nredirected > 0 || prstate.ndead > 0 || prstate.nunused > 0)
+ if (do_prune)
{
/*
* Apply the planned item changes, then repair page fragmentation, and
@@ -577,20 +619,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
/* Record number of newly-set-LP_DEAD items for caller */
presult->nnewlpdead = prstate.ndead;
- /*
- * Freeze the page when heap_prepare_freeze_tuple indicates that at least
- * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
- * freeze when pruning generated an FPI, if doing so means that we set the
- * page all-frozen afterwards (might not happen until final heap pass).
- */
- if (pagefrz)
- do_freeze = pagefrz->freeze_required ||
- (presult->all_visible_except_removable && presult->all_frozen &&
- presult->nfrozen > 0 &&
- fpi_before != pgWalUsage.wal_fpi);
- else
- do_freeze = false;
-
if (do_freeze)
{
frz_conflict_horizon = heap_frz_conflict_horizon(presult, pagefrz);
--
2.40.1
--racicctn4wry6xe5
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v3-0009-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch"
^ permalink raw reply [nested|flat] 14+ messages in thread
* [PATCH v4 10/19] Make opp freeze heuristic compatible with prune+freeze record
@ 2024-01-07 21:11 Melanie Plageman <[email protected]>
0 siblings, 0 replies; 14+ messages in thread
From: Melanie Plageman @ 2024-01-07 21:11 UTC (permalink / raw)
Once the prune and freeze records are combined, we will no longer be
able to use a test of whether or not pruning emitted an FPI to decide
whether or not to opportunistically freeze a freezable page.
While this heuristic should be improved, for now, approximate the
previous logic by keeping track of whether or not a hint bit FPI was
emitted during visibility checks (when checksums are on) and combine
that with checking XLogCheckBufferNeedsBackup(). If we just finished
deciding whether or not to prune and the current buffer seems to need an
FPI after modification, it is likely that pruning would have emitted an
FPI.
---
src/backend/access/heap/pruneheap.c | 57 +++++++++++++++++++++--------
1 file changed, 42 insertions(+), 15 deletions(-)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index 20907ba5408..9edf6bf72d7 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -246,6 +246,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
PruneState prstate;
HeapTupleData tup;
bool do_freeze;
+ bool do_prune;
+ bool whole_page_freezable;
+ bool hint_bit_fpi;
+ bool prune_fpi = false;
int64 fpi_before = pgWalUsage.wal_fpi;
/*
@@ -439,6 +443,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
}
}
+ /*
+ * If checksums are enabled, heap_prune_satisfies_vacuum() may have caused
+ * an FPI to be emitted. Then reset fpi_before for no prune case.
+ */
+ hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi;
+ fpi_before = pgWalUsage.wal_fpi;
+
/*
* For vacuum, if the whole page will become frozen, we consider
* opportunistically freezing tuples. Dead tuples which will be removed by
@@ -483,11 +494,41 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
if (off_loc)
*off_loc = InvalidOffsetNumber;
+ do_prune = prstate.nredirected > 0 ||
+ prstate.ndead > 0 ||
+ prstate.nunused > 0;
+
+ /*
+ * Only incur overhead of checking if we will do an FPI if we might use
+ * the information.
+ */
+ if (do_prune && pagefrz)
+ prune_fpi = XLogCheckBufferNeedsBackup(buffer);
+
+ /* Is the whole page freezable? And is there something to freeze */
+ whole_page_freezable = presult->all_visible_except_removable &&
+ presult->all_frozen;
+
+ /*
+ * Freeze the page when heap_prepare_freeze_tuple indicates that at least
+ * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
+ * freeze when pruning generated an FPI, if doing so means that we set the
+ * page all-frozen afterwards (might not happen until final heap pass).
+ * XXX: Previously, we knew if pruning emitted an FPI by checking
+ * pgWalUsage.wal_fpi before and after pruning. Once the freeze and prune
+ * records are combined, this heuristic couldn't be used anymore. The
+ * opportunistic freeze heuristic must be improved; however, for now, try
+ * to approximate it.
+ */
+ do_freeze = pagefrz &&
+ (pagefrz->freeze_required ||
+ (whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi)));
+
/* Any error while applying the changes is critical */
START_CRIT_SECTION();
/* Have we found any prunable items? */
- if (prstate.nredirected > 0 || prstate.ndead > 0 || prstate.nunused > 0)
+ if (do_prune)
{
/*
* Apply the planned item changes, then repair page fragmentation, and
@@ -579,20 +620,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
/* Record number of newly-set-LP_DEAD items for caller */
presult->nnewlpdead = prstate.ndead;
- /*
- * Freeze the page when heap_prepare_freeze_tuple indicates that at least
- * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
- * freeze when pruning generated an FPI, if doing so means that we set the
- * page all-frozen afterwards (might not happen until final heap pass).
- */
- if (pagefrz)
- do_freeze = pagefrz->freeze_required ||
- (presult->all_visible_except_removable && presult->all_frozen &&
- presult->nfrozen > 0 &&
- fpi_before != pgWalUsage.wal_fpi);
- else
- do_freeze = false;
-
if (do_freeze)
{
/*
--
2.40.1
--tez7m2a73jtztiij
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0011-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch"
^ permalink raw reply [nested|flat] 14+ messages in thread
* [PATCH v4 10/19] Make opp freeze heuristic compatible with prune+freeze record
@ 2024-01-07 21:11 Melanie Plageman <[email protected]>
0 siblings, 0 replies; 14+ messages in thread
From: Melanie Plageman @ 2024-01-07 21:11 UTC (permalink / raw)
Once the prune and freeze records are combined, we will no longer be
able to use a test of whether or not pruning emitted an FPI to decide
whether or not to opportunistically freeze a freezable page.
While this heuristic should be improved, for now, approximate the
previous logic by keeping track of whether or not a hint bit FPI was
emitted during visibility checks (when checksums are on) and combine
that with checking XLogCheckBufferNeedsBackup(). If we just finished
deciding whether or not to prune and the current buffer seems to need an
FPI after modification, it is likely that pruning would have emitted an
FPI.
---
src/backend/access/heap/pruneheap.c | 57 +++++++++++++++++++++--------
1 file changed, 42 insertions(+), 15 deletions(-)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index 20907ba5408..9edf6bf72d7 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -246,6 +246,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
PruneState prstate;
HeapTupleData tup;
bool do_freeze;
+ bool do_prune;
+ bool whole_page_freezable;
+ bool hint_bit_fpi;
+ bool prune_fpi = false;
int64 fpi_before = pgWalUsage.wal_fpi;
/*
@@ -439,6 +443,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
}
}
+ /*
+ * If checksums are enabled, heap_prune_satisfies_vacuum() may have caused
+ * an FPI to be emitted. Then reset fpi_before for no prune case.
+ */
+ hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi;
+ fpi_before = pgWalUsage.wal_fpi;
+
/*
* For vacuum, if the whole page will become frozen, we consider
* opportunistically freezing tuples. Dead tuples which will be removed by
@@ -483,11 +494,41 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
if (off_loc)
*off_loc = InvalidOffsetNumber;
+ do_prune = prstate.nredirected > 0 ||
+ prstate.ndead > 0 ||
+ prstate.nunused > 0;
+
+ /*
+ * Only incur overhead of checking if we will do an FPI if we might use
+ * the information.
+ */
+ if (do_prune && pagefrz)
+ prune_fpi = XLogCheckBufferNeedsBackup(buffer);
+
+ /* Is the whole page freezable? And is there something to freeze */
+ whole_page_freezable = presult->all_visible_except_removable &&
+ presult->all_frozen;
+
+ /*
+ * Freeze the page when heap_prepare_freeze_tuple indicates that at least
+ * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
+ * freeze when pruning generated an FPI, if doing so means that we set the
+ * page all-frozen afterwards (might not happen until final heap pass).
+ * XXX: Previously, we knew if pruning emitted an FPI by checking
+ * pgWalUsage.wal_fpi before and after pruning. Once the freeze and prune
+ * records are combined, this heuristic couldn't be used anymore. The
+ * opportunistic freeze heuristic must be improved; however, for now, try
+ * to approximate it.
+ */
+ do_freeze = pagefrz &&
+ (pagefrz->freeze_required ||
+ (whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi)));
+
/* Any error while applying the changes is critical */
START_CRIT_SECTION();
/* Have we found any prunable items? */
- if (prstate.nredirected > 0 || prstate.ndead > 0 || prstate.nunused > 0)
+ if (do_prune)
{
/*
* Apply the planned item changes, then repair page fragmentation, and
@@ -579,20 +620,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
/* Record number of newly-set-LP_DEAD items for caller */
presult->nnewlpdead = prstate.ndead;
- /*
- * Freeze the page when heap_prepare_freeze_tuple indicates that at least
- * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
- * freeze when pruning generated an FPI, if doing so means that we set the
- * page all-frozen afterwards (might not happen until final heap pass).
- */
- if (pagefrz)
- do_freeze = pagefrz->freeze_required ||
- (presult->all_visible_except_removable && presult->all_frozen &&
- presult->nfrozen > 0 &&
- fpi_before != pgWalUsage.wal_fpi);
- else
- do_freeze = false;
-
if (do_freeze)
{
/*
--
2.40.1
--tez7m2a73jtztiij
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0011-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch"
^ permalink raw reply [nested|flat] 14+ messages in thread
* [PATCH v7 09/16] Make opp freeze heuristic compatible with prune+freeze record
@ 2024-03-26 00:48 Melanie Plageman <[email protected]>
0 siblings, 0 replies; 14+ messages in thread
From: Melanie Plageman @ 2024-03-26 00:48 UTC (permalink / raw)
Once the prune and freeze records are combined, we will no longer be
able to use a test of whether or not pruning emitted an FPI to decide
whether or not to opportunistically freeze a freezable page.
While this heuristic should be improved, for now, approximate the
previous logic by keeping track of whether or not a hint bit FPI was
emitted during visibility checks (when checksums are on) and combine
that with checking XLogCheckBufferNeedsBackup(). If we just finished
deciding whether or not to prune and the current buffer seems to need an
FPI after modification, it is likely that pruning would have emitted an
FPI.
---
src/backend/access/heap/pruneheap.c | 57 +++++++++++++++++++++--------
1 file changed, 42 insertions(+), 15 deletions(-)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index e009c7579dd..d38de9b063d 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -247,6 +247,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
TransactionId visibility_cutoff_xid;
bool do_freeze;
bool all_visible_except_removable;
+ bool do_prune;
+ bool whole_page_freezable;
+ bool hint_bit_fpi;
+ bool prune_fpi = false;
int64 fpi_before = pgWalUsage.wal_fpi;
/*
@@ -456,6 +460,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
}
}
+ /*
+ * If checksums are enabled, heap_prune_satisfies_vacuum() may have caused
+ * an FPI to be emitted. Then reset fpi_before for no prune case.
+ */
+ hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi;
+ fpi_before = pgWalUsage.wal_fpi;
+
/*
* For vacuum, if the whole page will become frozen, we consider
* opportunistically freezing tuples. Dead tuples which will be removed by
@@ -500,11 +511,41 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
if (off_loc)
*off_loc = InvalidOffsetNumber;
+ do_prune = prstate.nredirected > 0 ||
+ prstate.ndead > 0 ||
+ prstate.nunused > 0;
+
+ /*
+ * Only incur overhead of checking if we will do an FPI if we might use
+ * the information.
+ */
+ if (do_prune && pagefrz)
+ prune_fpi = XLogCheckBufferNeedsBackup(buffer);
+
+ /* Is the whole page freezable? And is there something to freeze */
+ whole_page_freezable = all_visible_except_removable &&
+ presult->all_frozen;
+
+ /*
+ * Freeze the page when heap_prepare_freeze_tuple indicates that at least
+ * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
+ * freeze when pruning generated an FPI, if doing so means that we set the
+ * page all-frozen afterwards (might not happen until final heap pass).
+ * XXX: Previously, we knew if pruning emitted an FPI by checking
+ * pgWalUsage.wal_fpi before and after pruning. Once the freeze and prune
+ * records are combined, this heuristic couldn't be used anymore. The
+ * opportunistic freeze heuristic must be improved; however, for now, try
+ * to approximate it.
+ */
+ do_freeze = pagefrz &&
+ (pagefrz->freeze_required ||
+ (whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi)));
+
/* Any error while applying the changes is critical */
START_CRIT_SECTION();
/* Have we found any prunable items? */
- if (prstate.nredirected > 0 || prstate.ndead > 0 || prstate.nunused > 0)
+ if (do_prune)
{
/*
* Apply the planned item changes, then repair page fragmentation, and
@@ -569,20 +610,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
/* Record number of newly-set-LP_DEAD items for caller */
presult->nnewlpdead = prstate.ndead;
- /*
- * Freeze the page when heap_prepare_freeze_tuple indicates that at least
- * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
- * freeze when pruning generated an FPI, if doing so means that we set the
- * page all-frozen afterwards (might not happen until final heap pass).
- */
- if (pagefrz)
- do_freeze = pagefrz->freeze_required ||
- (all_visible_except_removable && presult->all_frozen &&
- presult->nfrozen > 0 &&
- fpi_before != pgWalUsage.wal_fpi);
- else
- do_freeze = false;
-
if (do_freeze)
{
TransactionId frz_conflict_horizon = InvalidTransactionId;
--
2.40.1
--ck6erxojvlx23byk
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0010-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch"
^ permalink raw reply [nested|flat] 14+ messages in thread
* [PATCH v7 09/16] Make opp freeze heuristic compatible with prune+freeze record
@ 2024-03-26 00:48 Melanie Plageman <[email protected]>
0 siblings, 0 replies; 14+ messages in thread
From: Melanie Plageman @ 2024-03-26 00:48 UTC (permalink / raw)
Once the prune and freeze records are combined, we will no longer be
able to use a test of whether or not pruning emitted an FPI to decide
whether or not to opportunistically freeze a freezable page.
While this heuristic should be improved, for now, approximate the
previous logic by keeping track of whether or not a hint bit FPI was
emitted during visibility checks (when checksums are on) and combine
that with checking XLogCheckBufferNeedsBackup(). If we just finished
deciding whether or not to prune and the current buffer seems to need an
FPI after modification, it is likely that pruning would have emitted an
FPI.
---
src/backend/access/heap/pruneheap.c | 57 +++++++++++++++++++++--------
1 file changed, 42 insertions(+), 15 deletions(-)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index e009c7579dd..d38de9b063d 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -247,6 +247,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
TransactionId visibility_cutoff_xid;
bool do_freeze;
bool all_visible_except_removable;
+ bool do_prune;
+ bool whole_page_freezable;
+ bool hint_bit_fpi;
+ bool prune_fpi = false;
int64 fpi_before = pgWalUsage.wal_fpi;
/*
@@ -456,6 +460,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
}
}
+ /*
+ * If checksums are enabled, heap_prune_satisfies_vacuum() may have caused
+ * an FPI to be emitted. Then reset fpi_before for no prune case.
+ */
+ hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi;
+ fpi_before = pgWalUsage.wal_fpi;
+
/*
* For vacuum, if the whole page will become frozen, we consider
* opportunistically freezing tuples. Dead tuples which will be removed by
@@ -500,11 +511,41 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
if (off_loc)
*off_loc = InvalidOffsetNumber;
+ do_prune = prstate.nredirected > 0 ||
+ prstate.ndead > 0 ||
+ prstate.nunused > 0;
+
+ /*
+ * Only incur overhead of checking if we will do an FPI if we might use
+ * the information.
+ */
+ if (do_prune && pagefrz)
+ prune_fpi = XLogCheckBufferNeedsBackup(buffer);
+
+ /* Is the whole page freezable? And is there something to freeze */
+ whole_page_freezable = all_visible_except_removable &&
+ presult->all_frozen;
+
+ /*
+ * Freeze the page when heap_prepare_freeze_tuple indicates that at least
+ * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
+ * freeze when pruning generated an FPI, if doing so means that we set the
+ * page all-frozen afterwards (might not happen until final heap pass).
+ * XXX: Previously, we knew if pruning emitted an FPI by checking
+ * pgWalUsage.wal_fpi before and after pruning. Once the freeze and prune
+ * records are combined, this heuristic couldn't be used anymore. The
+ * opportunistic freeze heuristic must be improved; however, for now, try
+ * to approximate it.
+ */
+ do_freeze = pagefrz &&
+ (pagefrz->freeze_required ||
+ (whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi)));
+
/* Any error while applying the changes is critical */
START_CRIT_SECTION();
/* Have we found any prunable items? */
- if (prstate.nredirected > 0 || prstate.ndead > 0 || prstate.nunused > 0)
+ if (do_prune)
{
/*
* Apply the planned item changes, then repair page fragmentation, and
@@ -569,20 +610,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
/* Record number of newly-set-LP_DEAD items for caller */
presult->nnewlpdead = prstate.ndead;
- /*
- * Freeze the page when heap_prepare_freeze_tuple indicates that at least
- * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
- * freeze when pruning generated an FPI, if doing so means that we set the
- * page all-frozen afterwards (might not happen until final heap pass).
- */
- if (pagefrz)
- do_freeze = pagefrz->freeze_required ||
- (all_visible_except_removable && presult->all_frozen &&
- presult->nfrozen > 0 &&
- fpi_before != pgWalUsage.wal_fpi);
- else
- do_freeze = false;
-
if (do_freeze)
{
TransactionId frz_conflict_horizon = InvalidTransactionId;
--
2.40.1
--ck6erxojvlx23byk
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v7-0010-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch"
^ permalink raw reply [nested|flat] 14+ messages in thread
* [PATCH v9 09/21] Make opp freeze heuristic compatible with prune+freeze record
@ 2024-03-26 00:48 Melanie Plageman <[email protected]>
0 siblings, 0 replies; 14+ messages in thread
From: Melanie Plageman @ 2024-03-26 00:48 UTC (permalink / raw)
Once the prune and freeze records are combined, we will no longer be
able to use a test of whether or not pruning emitted an FPI to decide
whether or not to opportunistically freeze a freezable page.
While this heuristic should be improved, for now, approximate the
previous logic by keeping track of whether or not a hint bit FPI was
emitted during visibility checks (when checksums are on) and combine
that with checking XLogCheckBufferNeedsBackup(). If we just finished
deciding whether or not to prune and the current buffer seems to need an
FPI after modification, it is likely that pruning would have emitted an
FPI.
---
src/backend/access/heap/pruneheap.c | 57 +++++++++++++++++++++--------
1 file changed, 42 insertions(+), 15 deletions(-)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index 312695f806c..f0decff35dc 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -247,6 +247,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
TransactionId visibility_cutoff_xid;
bool do_freeze;
bool all_visible_except_removable;
+ bool do_prune;
+ bool whole_page_freezable;
+ bool hint_bit_fpi;
+ bool prune_fpi = false;
int64 fpi_before = pgWalUsage.wal_fpi;
/*
@@ -456,6 +460,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
}
}
+ /*
+ * If checksums are enabled, heap_prune_satisfies_vacuum() may have caused
+ * an FPI to be emitted. Then reset fpi_before for no prune case.
+ */
+ hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi;
+ fpi_before = pgWalUsage.wal_fpi;
+
/*
* For vacuum, if the whole page will become frozen, we consider
* opportunistically freezing tuples. Dead tuples which will be removed by
@@ -500,11 +511,41 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
if (off_loc)
*off_loc = InvalidOffsetNumber;
+ do_prune = prstate.nredirected > 0 ||
+ prstate.ndead > 0 ||
+ prstate.nunused > 0;
+
+ /*
+ * Only incur overhead of checking if we will do an FPI if we might use
+ * the information.
+ */
+ if (do_prune && pagefrz)
+ prune_fpi = XLogCheckBufferNeedsBackup(buffer);
+
+ /* Is the whole page freezable? And is there something to freeze */
+ whole_page_freezable = all_visible_except_removable &&
+ presult->all_frozen;
+
+ /*
+ * Freeze the page when heap_prepare_freeze_tuple indicates that at least
+ * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
+ * freeze when pruning generated an FPI, if doing so means that we set the
+ * page all-frozen afterwards (might not happen until final heap pass).
+ * XXX: Previously, we knew if pruning emitted an FPI by checking
+ * pgWalUsage.wal_fpi before and after pruning. Once the freeze and prune
+ * records are combined, this heuristic couldn't be used anymore. The
+ * opportunistic freeze heuristic must be improved; however, for now, try
+ * to approximate it.
+ */
+ do_freeze = pagefrz &&
+ (pagefrz->freeze_required ||
+ (whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi)));
+
/* Any error while applying the changes is critical */
START_CRIT_SECTION();
/* Have we found any prunable items? */
- if (prstate.nredirected > 0 || prstate.ndead > 0 || prstate.nunused > 0)
+ if (do_prune)
{
/*
* Apply the planned item changes, then repair page fragmentation, and
@@ -569,20 +610,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer,
/* Record number of newly-set-LP_DEAD items for caller */
presult->nnewlpdead = prstate.ndead;
- /*
- * Freeze the page when heap_prepare_freeze_tuple indicates that at least
- * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
- * freeze when pruning generated an FPI, if doing so means that we set the
- * page all-frozen afterwards (might not happen until final heap pass).
- */
- if (pagefrz)
- do_freeze = pagefrz->freeze_required ||
- (all_visible_except_removable && presult->all_frozen &&
- presult->nfrozen > 0 &&
- fpi_before != pgWalUsage.wal_fpi);
- else
- do_freeze = false;
-
if (do_freeze)
{
TransactionId frz_conflict_horizon = InvalidTransactionId;
--
2.40.1
--caj67xgx3lukmr5f
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v9-0010-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch"
^ permalink raw reply [nested|flat] 14+ messages in thread
end of thread, other threads:[~2024-03-26 00:48 UTC | newest]
Thread overview: 14+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-05-19 02:41 Re: PG 16 draft release notes ready jian he <[email protected]>
2023-05-19 03:15 ` Bruce Momjian <[email protected]>
2024-01-07 21:11 [PATCH v2 08/17] Make opp freeze heuristic compatible with prune+freeze record Melanie Plageman <[email protected]>
2024-01-07 21:11 [PATCH v4 10/19] Make opp freeze heuristic compatible with prune+freeze record Melanie Plageman <[email protected]>
2024-01-07 21:11 [PATCH v4 10/19] Make opp freeze heuristic compatible with prune+freeze record Melanie Plageman <[email protected]>
2024-01-07 21:11 [PATCH v2 08/17] Make opp freeze heuristic compatible with prune+freeze record Melanie Plageman <[email protected]>
2024-01-07 21:11 [PATCH v2 08/17] Make opp freeze heuristic compatible with prune+freeze record Melanie Plageman <[email protected]>
2024-01-07 21:11 [PATCH v4 10/19] Make opp freeze heuristic compatible with prune+freeze record Melanie Plageman <[email protected]>
2024-01-07 21:11 [PATCH v3 08/17] Make opp freeze heuristic compatible with prune+freeze record Melanie Plageman <[email protected]>
2024-01-07 21:11 [PATCH v3 08/17] Make opp freeze heuristic compatible with prune+freeze record Melanie Plageman <[email protected]>
2024-01-07 21:11 [PATCH v3 08/17] Make opp freeze heuristic compatible with prune+freeze record Melanie Plageman <[email protected]>
2024-03-26 00:48 [PATCH v7 09/16] Make opp freeze heuristic compatible with prune+freeze record Melanie Plageman <[email protected]>
2024-03-26 00:48 [PATCH v7 09/16] Make opp freeze heuristic compatible with prune+freeze record Melanie Plageman <[email protected]>
2024-03-26 00:48 [PATCH v9 09/21] Make opp freeze heuristic compatible with prune+freeze record Melanie Plageman <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox