agora inbox for [email protected]help / color / mirror / Atom feed
[PATCH v2 08/17] Make opp freeze heuristic compatible with prune+freeze record 13+ messages / 2 participants [nested] [flat]
* [PATCH v2 08/17] Make opp freeze heuristic compatible with prune+freeze record @ 2024-01-07 21:11 Melanie Plageman <[email protected]> 0 siblings, 0 replies; 13+ messages in thread From: Melanie Plageman @ 2024-01-07 21:11 UTC (permalink / raw) Once the prune and freeze records are combined, we will no longer be able to use a test of whether or not pruning emitted an FPI to decide whether or not to opportunistically freeze a freezable page. While this heuristic should be improved, for now, approximate the previous logic by keeping track of whether or not a hint bit FPI was emitted during visibility checks (when checksums are on) and combine that with checking XLogCheckBufferNeedsBackup(). If we just finished deciding whether or not to prune and the current buffer seems to need an FPI after modification, it is likely that pruning would have emitted an FPI. --- src/backend/access/heap/pruneheap.c | 58 +++++++++++++++++++++-------- 1 file changed, 43 insertions(+), 15 deletions(-) diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 9c709315192..e715fc29a83 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -241,6 +241,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, PruneState prstate; HeapTupleData tup; bool do_freeze; + bool do_prune; + bool whole_page_freezable; + bool hint_bit_fpi; + bool prune_fpi = false; int64 fpi_before = pgWalUsage.wal_fpi; TransactionId frz_conflict_horizon = InvalidTransactionId; @@ -410,6 +414,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, } } + /* + * If checksums are enabled, heap_prune_satisfies_vacuum() may have caused + * an FPI to be emitted. Then reset fpi_before for no prune case. + */ + hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi; + fpi_before = pgWalUsage.wal_fpi; + /* * For vacuum, if the whole page will become frozen, we consider * opportunistically freezing tuples. Dead tuples which will be removed by @@ -467,11 +478,42 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, if (off_loc) *off_loc = InvalidOffsetNumber; + do_prune = prstate.nredirected > 0 || + prstate.ndead > 0 || + prstate.nunused > 0; + + /* + * Only incur overhead of checking if we will do an FPI if we might use + * the information. + */ + if (do_prune && pagefrz) + prune_fpi = XLogCheckBufferNeedsBackup(buffer); + + /* Is the whole page freezable? And is there something to freeze */ + whole_page_freezable = presult->all_visible_except_removable && + presult->all_frozen; + + /* + * Freeze the page when heap_prepare_freeze_tuple indicates that at least + * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also + * freeze when pruning generated an FPI, if doing so means that we set the + * page all-frozen afterwards (might not happen until final heap pass). + * XXX: Previously, we knew if pruning emitted an FPI by checking + * pgWalUsage.wal_fpi before and after pruning. Once the freeze and prune + * records are combined, this heuristic couldn't be used anymore. The + * opportunistic freeze heuristic must be improved; however, for now, try + * to approximate it. + */ + + do_freeze = pagefrz && + (pagefrz->freeze_required || + (whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi))); + /* Any error while applying the changes is critical */ START_CRIT_SECTION(); /* Have we found any prunable items? */ - if (prstate.nredirected > 0 || prstate.ndead > 0 || prstate.nunused > 0) + if (do_prune) { /* * Apply the planned item changes, then repair page fragmentation, and @@ -563,20 +605,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, /* Record number of newly-set-LP_DEAD items for caller */ presult->nnewlpdead = prstate.ndead; - /* - * Freeze the page when heap_prepare_freeze_tuple indicates that at least - * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also - * freeze when pruning generated an FPI, if doing so means that we set the - * page all-frozen afterwards (might not happen until final heap pass). - */ - if (pagefrz) - do_freeze = pagefrz->freeze_required || - (presult->all_visible_except_removable && presult->all_frozen && - presult->nfrozen > 0 && - fpi_before != pgWalUsage.wal_fpi); - else - do_freeze = false; - if (do_freeze) { frz_conflict_horizon = heap_frz_conflict_horizon(presult, pagefrz); -- 2.40.1 --rdqtp5puvxqotfdw Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v2-0009-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch" ^ permalink raw reply [nested|flat] 13+ messages in thread
* [PATCH v3 08/17] Make opp freeze heuristic compatible with prune+freeze record @ 2024-01-07 21:11 Melanie Plageman <[email protected]> 0 siblings, 0 replies; 13+ messages in thread From: Melanie Plageman @ 2024-01-07 21:11 UTC (permalink / raw) Once the prune and freeze records are combined, we will no longer be able to use a test of whether or not pruning emitted an FPI to decide whether or not to opportunistically freeze a freezable page. While this heuristic should be improved, for now, approximate the previous logic by keeping track of whether or not a hint bit FPI was emitted during visibility checks (when checksums are on) and combine that with checking XLogCheckBufferNeedsBackup(). If we just finished deciding whether or not to prune and the current buffer seems to need an FPI after modification, it is likely that pruning would have emitted an FPI. --- src/backend/access/heap/pruneheap.c | 58 +++++++++++++++++++++-------- 1 file changed, 43 insertions(+), 15 deletions(-) diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index abf6bdb2d99..f164b7957ed 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -255,6 +255,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, PruneState prstate; HeapTupleData tup; bool do_freeze; + bool do_prune; + bool whole_page_freezable; + bool hint_bit_fpi; + bool prune_fpi = false; int64 fpi_before = pgWalUsage.wal_fpi; TransactionId frz_conflict_horizon = InvalidTransactionId; @@ -424,6 +428,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, } } + /* + * If checksums are enabled, heap_prune_satisfies_vacuum() may have caused + * an FPI to be emitted. Then reset fpi_before for no prune case. + */ + hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi; + fpi_before = pgWalUsage.wal_fpi; + /* * For vacuum, if the whole page will become frozen, we consider * opportunistically freezing tuples. Dead tuples which will be removed by @@ -481,11 +492,42 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, if (off_loc) *off_loc = InvalidOffsetNumber; + do_prune = prstate.nredirected > 0 || + prstate.ndead > 0 || + prstate.nunused > 0; + + /* + * Only incur overhead of checking if we will do an FPI if we might use + * the information. + */ + if (do_prune && pagefrz) + prune_fpi = XLogCheckBufferNeedsBackup(buffer); + + /* Is the whole page freezable? And is there something to freeze */ + whole_page_freezable = presult->all_visible_except_removable && + presult->all_frozen; + + /* + * Freeze the page when heap_prepare_freeze_tuple indicates that at least + * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also + * freeze when pruning generated an FPI, if doing so means that we set the + * page all-frozen afterwards (might not happen until final heap pass). + * XXX: Previously, we knew if pruning emitted an FPI by checking + * pgWalUsage.wal_fpi before and after pruning. Once the freeze and prune + * records are combined, this heuristic couldn't be used anymore. The + * opportunistic freeze heuristic must be improved; however, for now, try + * to approximate it. + */ + + do_freeze = pagefrz && + (pagefrz->freeze_required || + (whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi))); + /* Any error while applying the changes is critical */ START_CRIT_SECTION(); /* Have we found any prunable items? */ - if (prstate.nredirected > 0 || prstate.ndead > 0 || prstate.nunused > 0) + if (do_prune) { /* * Apply the planned item changes, then repair page fragmentation, and @@ -577,20 +619,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, /* Record number of newly-set-LP_DEAD items for caller */ presult->nnewlpdead = prstate.ndead; - /* - * Freeze the page when heap_prepare_freeze_tuple indicates that at least - * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also - * freeze when pruning generated an FPI, if doing so means that we set the - * page all-frozen afterwards (might not happen until final heap pass). - */ - if (pagefrz) - do_freeze = pagefrz->freeze_required || - (presult->all_visible_except_removable && presult->all_frozen && - presult->nfrozen > 0 && - fpi_before != pgWalUsage.wal_fpi); - else - do_freeze = false; - if (do_freeze) { frz_conflict_horizon = heap_frz_conflict_horizon(presult, pagefrz); -- 2.40.1 --racicctn4wry6xe5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v3-0009-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch" ^ permalink raw reply [nested|flat] 13+ messages in thread
* [PATCH v4 10/19] Make opp freeze heuristic compatible with prune+freeze record @ 2024-01-07 21:11 Melanie Plageman <[email protected]> 0 siblings, 0 replies; 13+ messages in thread From: Melanie Plageman @ 2024-01-07 21:11 UTC (permalink / raw) Once the prune and freeze records are combined, we will no longer be able to use a test of whether or not pruning emitted an FPI to decide whether or not to opportunistically freeze a freezable page. While this heuristic should be improved, for now, approximate the previous logic by keeping track of whether or not a hint bit FPI was emitted during visibility checks (when checksums are on) and combine that with checking XLogCheckBufferNeedsBackup(). If we just finished deciding whether or not to prune and the current buffer seems to need an FPI after modification, it is likely that pruning would have emitted an FPI. --- src/backend/access/heap/pruneheap.c | 57 +++++++++++++++++++++-------- 1 file changed, 42 insertions(+), 15 deletions(-) diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 20907ba5408..9edf6bf72d7 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -246,6 +246,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, PruneState prstate; HeapTupleData tup; bool do_freeze; + bool do_prune; + bool whole_page_freezable; + bool hint_bit_fpi; + bool prune_fpi = false; int64 fpi_before = pgWalUsage.wal_fpi; /* @@ -439,6 +443,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, } } + /* + * If checksums are enabled, heap_prune_satisfies_vacuum() may have caused + * an FPI to be emitted. Then reset fpi_before for no prune case. + */ + hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi; + fpi_before = pgWalUsage.wal_fpi; + /* * For vacuum, if the whole page will become frozen, we consider * opportunistically freezing tuples. Dead tuples which will be removed by @@ -483,11 +494,41 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, if (off_loc) *off_loc = InvalidOffsetNumber; + do_prune = prstate.nredirected > 0 || + prstate.ndead > 0 || + prstate.nunused > 0; + + /* + * Only incur overhead of checking if we will do an FPI if we might use + * the information. + */ + if (do_prune && pagefrz) + prune_fpi = XLogCheckBufferNeedsBackup(buffer); + + /* Is the whole page freezable? And is there something to freeze */ + whole_page_freezable = presult->all_visible_except_removable && + presult->all_frozen; + + /* + * Freeze the page when heap_prepare_freeze_tuple indicates that at least + * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also + * freeze when pruning generated an FPI, if doing so means that we set the + * page all-frozen afterwards (might not happen until final heap pass). + * XXX: Previously, we knew if pruning emitted an FPI by checking + * pgWalUsage.wal_fpi before and after pruning. Once the freeze and prune + * records are combined, this heuristic couldn't be used anymore. The + * opportunistic freeze heuristic must be improved; however, for now, try + * to approximate it. + */ + do_freeze = pagefrz && + (pagefrz->freeze_required || + (whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi))); + /* Any error while applying the changes is critical */ START_CRIT_SECTION(); /* Have we found any prunable items? */ - if (prstate.nredirected > 0 || prstate.ndead > 0 || prstate.nunused > 0) + if (do_prune) { /* * Apply the planned item changes, then repair page fragmentation, and @@ -579,20 +620,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, /* Record number of newly-set-LP_DEAD items for caller */ presult->nnewlpdead = prstate.ndead; - /* - * Freeze the page when heap_prepare_freeze_tuple indicates that at least - * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also - * freeze when pruning generated an FPI, if doing so means that we set the - * page all-frozen afterwards (might not happen until final heap pass). - */ - if (pagefrz) - do_freeze = pagefrz->freeze_required || - (presult->all_visible_except_removable && presult->all_frozen && - presult->nfrozen > 0 && - fpi_before != pgWalUsage.wal_fpi); - else - do_freeze = false; - if (do_freeze) { /* -- 2.40.1 --tez7m2a73jtztiij Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v4-0011-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch" ^ permalink raw reply [nested|flat] 13+ messages in thread
* [PATCH v4 10/19] Make opp freeze heuristic compatible with prune+freeze record @ 2024-01-07 21:11 Melanie Plageman <[email protected]> 0 siblings, 0 replies; 13+ messages in thread From: Melanie Plageman @ 2024-01-07 21:11 UTC (permalink / raw) Once the prune and freeze records are combined, we will no longer be able to use a test of whether or not pruning emitted an FPI to decide whether or not to opportunistically freeze a freezable page. While this heuristic should be improved, for now, approximate the previous logic by keeping track of whether or not a hint bit FPI was emitted during visibility checks (when checksums are on) and combine that with checking XLogCheckBufferNeedsBackup(). If we just finished deciding whether or not to prune and the current buffer seems to need an FPI after modification, it is likely that pruning would have emitted an FPI. --- src/backend/access/heap/pruneheap.c | 57 +++++++++++++++++++++-------- 1 file changed, 42 insertions(+), 15 deletions(-) diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 20907ba5408..9edf6bf72d7 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -246,6 +246,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, PruneState prstate; HeapTupleData tup; bool do_freeze; + bool do_prune; + bool whole_page_freezable; + bool hint_bit_fpi; + bool prune_fpi = false; int64 fpi_before = pgWalUsage.wal_fpi; /* @@ -439,6 +443,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, } } + /* + * If checksums are enabled, heap_prune_satisfies_vacuum() may have caused + * an FPI to be emitted. Then reset fpi_before for no prune case. + */ + hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi; + fpi_before = pgWalUsage.wal_fpi; + /* * For vacuum, if the whole page will become frozen, we consider * opportunistically freezing tuples. Dead tuples which will be removed by @@ -483,11 +494,41 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, if (off_loc) *off_loc = InvalidOffsetNumber; + do_prune = prstate.nredirected > 0 || + prstate.ndead > 0 || + prstate.nunused > 0; + + /* + * Only incur overhead of checking if we will do an FPI if we might use + * the information. + */ + if (do_prune && pagefrz) + prune_fpi = XLogCheckBufferNeedsBackup(buffer); + + /* Is the whole page freezable? And is there something to freeze */ + whole_page_freezable = presult->all_visible_except_removable && + presult->all_frozen; + + /* + * Freeze the page when heap_prepare_freeze_tuple indicates that at least + * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also + * freeze when pruning generated an FPI, if doing so means that we set the + * page all-frozen afterwards (might not happen until final heap pass). + * XXX: Previously, we knew if pruning emitted an FPI by checking + * pgWalUsage.wal_fpi before and after pruning. Once the freeze and prune + * records are combined, this heuristic couldn't be used anymore. The + * opportunistic freeze heuristic must be improved; however, for now, try + * to approximate it. + */ + do_freeze = pagefrz && + (pagefrz->freeze_required || + (whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi))); + /* Any error while applying the changes is critical */ START_CRIT_SECTION(); /* Have we found any prunable items? */ - if (prstate.nredirected > 0 || prstate.ndead > 0 || prstate.nunused > 0) + if (do_prune) { /* * Apply the planned item changes, then repair page fragmentation, and @@ -579,20 +620,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, /* Record number of newly-set-LP_DEAD items for caller */ presult->nnewlpdead = prstate.ndead; - /* - * Freeze the page when heap_prepare_freeze_tuple indicates that at least - * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also - * freeze when pruning generated an FPI, if doing so means that we set the - * page all-frozen afterwards (might not happen until final heap pass). - */ - if (pagefrz) - do_freeze = pagefrz->freeze_required || - (presult->all_visible_except_removable && presult->all_frozen && - presult->nfrozen > 0 && - fpi_before != pgWalUsage.wal_fpi); - else - do_freeze = false; - if (do_freeze) { /* -- 2.40.1 --tez7m2a73jtztiij Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v4-0011-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch" ^ permalink raw reply [nested|flat] 13+ messages in thread
* [PATCH v2 08/17] Make opp freeze heuristic compatible with prune+freeze record @ 2024-01-07 21:11 Melanie Plageman <[email protected]> 0 siblings, 0 replies; 13+ messages in thread From: Melanie Plageman @ 2024-01-07 21:11 UTC (permalink / raw) Once the prune and freeze records are combined, we will no longer be able to use a test of whether or not pruning emitted an FPI to decide whether or not to opportunistically freeze a freezable page. While this heuristic should be improved, for now, approximate the previous logic by keeping track of whether or not a hint bit FPI was emitted during visibility checks (when checksums are on) and combine that with checking XLogCheckBufferNeedsBackup(). If we just finished deciding whether or not to prune and the current buffer seems to need an FPI after modification, it is likely that pruning would have emitted an FPI. --- src/backend/access/heap/pruneheap.c | 58 +++++++++++++++++++++-------- 1 file changed, 43 insertions(+), 15 deletions(-) diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 9c709315192..e715fc29a83 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -241,6 +241,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, PruneState prstate; HeapTupleData tup; bool do_freeze; + bool do_prune; + bool whole_page_freezable; + bool hint_bit_fpi; + bool prune_fpi = false; int64 fpi_before = pgWalUsage.wal_fpi; TransactionId frz_conflict_horizon = InvalidTransactionId; @@ -410,6 +414,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, } } + /* + * If checksums are enabled, heap_prune_satisfies_vacuum() may have caused + * an FPI to be emitted. Then reset fpi_before for no prune case. + */ + hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi; + fpi_before = pgWalUsage.wal_fpi; + /* * For vacuum, if the whole page will become frozen, we consider * opportunistically freezing tuples. Dead tuples which will be removed by @@ -467,11 +478,42 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, if (off_loc) *off_loc = InvalidOffsetNumber; + do_prune = prstate.nredirected > 0 || + prstate.ndead > 0 || + prstate.nunused > 0; + + /* + * Only incur overhead of checking if we will do an FPI if we might use + * the information. + */ + if (do_prune && pagefrz) + prune_fpi = XLogCheckBufferNeedsBackup(buffer); + + /* Is the whole page freezable? And is there something to freeze */ + whole_page_freezable = presult->all_visible_except_removable && + presult->all_frozen; + + /* + * Freeze the page when heap_prepare_freeze_tuple indicates that at least + * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also + * freeze when pruning generated an FPI, if doing so means that we set the + * page all-frozen afterwards (might not happen until final heap pass). + * XXX: Previously, we knew if pruning emitted an FPI by checking + * pgWalUsage.wal_fpi before and after pruning. Once the freeze and prune + * records are combined, this heuristic couldn't be used anymore. The + * opportunistic freeze heuristic must be improved; however, for now, try + * to approximate it. + */ + + do_freeze = pagefrz && + (pagefrz->freeze_required || + (whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi))); + /* Any error while applying the changes is critical */ START_CRIT_SECTION(); /* Have we found any prunable items? */ - if (prstate.nredirected > 0 || prstate.ndead > 0 || prstate.nunused > 0) + if (do_prune) { /* * Apply the planned item changes, then repair page fragmentation, and @@ -563,20 +605,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, /* Record number of newly-set-LP_DEAD items for caller */ presult->nnewlpdead = prstate.ndead; - /* - * Freeze the page when heap_prepare_freeze_tuple indicates that at least - * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also - * freeze when pruning generated an FPI, if doing so means that we set the - * page all-frozen afterwards (might not happen until final heap pass). - */ - if (pagefrz) - do_freeze = pagefrz->freeze_required || - (presult->all_visible_except_removable && presult->all_frozen && - presult->nfrozen > 0 && - fpi_before != pgWalUsage.wal_fpi); - else - do_freeze = false; - if (do_freeze) { frz_conflict_horizon = heap_frz_conflict_horizon(presult, pagefrz); -- 2.40.1 --rdqtp5puvxqotfdw Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v2-0009-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch" ^ permalink raw reply [nested|flat] 13+ messages in thread
* [PATCH v3 08/17] Make opp freeze heuristic compatible with prune+freeze record @ 2024-01-07 21:11 Melanie Plageman <[email protected]> 0 siblings, 0 replies; 13+ messages in thread From: Melanie Plageman @ 2024-01-07 21:11 UTC (permalink / raw) Once the prune and freeze records are combined, we will no longer be able to use a test of whether or not pruning emitted an FPI to decide whether or not to opportunistically freeze a freezable page. While this heuristic should be improved, for now, approximate the previous logic by keeping track of whether or not a hint bit FPI was emitted during visibility checks (when checksums are on) and combine that with checking XLogCheckBufferNeedsBackup(). If we just finished deciding whether or not to prune and the current buffer seems to need an FPI after modification, it is likely that pruning would have emitted an FPI. --- src/backend/access/heap/pruneheap.c | 58 +++++++++++++++++++++-------- 1 file changed, 43 insertions(+), 15 deletions(-) diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index abf6bdb2d99..f164b7957ed 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -255,6 +255,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, PruneState prstate; HeapTupleData tup; bool do_freeze; + bool do_prune; + bool whole_page_freezable; + bool hint_bit_fpi; + bool prune_fpi = false; int64 fpi_before = pgWalUsage.wal_fpi; TransactionId frz_conflict_horizon = InvalidTransactionId; @@ -424,6 +428,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, } } + /* + * If checksums are enabled, heap_prune_satisfies_vacuum() may have caused + * an FPI to be emitted. Then reset fpi_before for no prune case. + */ + hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi; + fpi_before = pgWalUsage.wal_fpi; + /* * For vacuum, if the whole page will become frozen, we consider * opportunistically freezing tuples. Dead tuples which will be removed by @@ -481,11 +492,42 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, if (off_loc) *off_loc = InvalidOffsetNumber; + do_prune = prstate.nredirected > 0 || + prstate.ndead > 0 || + prstate.nunused > 0; + + /* + * Only incur overhead of checking if we will do an FPI if we might use + * the information. + */ + if (do_prune && pagefrz) + prune_fpi = XLogCheckBufferNeedsBackup(buffer); + + /* Is the whole page freezable? And is there something to freeze */ + whole_page_freezable = presult->all_visible_except_removable && + presult->all_frozen; + + /* + * Freeze the page when heap_prepare_freeze_tuple indicates that at least + * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also + * freeze when pruning generated an FPI, if doing so means that we set the + * page all-frozen afterwards (might not happen until final heap pass). + * XXX: Previously, we knew if pruning emitted an FPI by checking + * pgWalUsage.wal_fpi before and after pruning. Once the freeze and prune + * records are combined, this heuristic couldn't be used anymore. The + * opportunistic freeze heuristic must be improved; however, for now, try + * to approximate it. + */ + + do_freeze = pagefrz && + (pagefrz->freeze_required || + (whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi))); + /* Any error while applying the changes is critical */ START_CRIT_SECTION(); /* Have we found any prunable items? */ - if (prstate.nredirected > 0 || prstate.ndead > 0 || prstate.nunused > 0) + if (do_prune) { /* * Apply the planned item changes, then repair page fragmentation, and @@ -577,20 +619,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, /* Record number of newly-set-LP_DEAD items for caller */ presult->nnewlpdead = prstate.ndead; - /* - * Freeze the page when heap_prepare_freeze_tuple indicates that at least - * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also - * freeze when pruning generated an FPI, if doing so means that we set the - * page all-frozen afterwards (might not happen until final heap pass). - */ - if (pagefrz) - do_freeze = pagefrz->freeze_required || - (presult->all_visible_except_removable && presult->all_frozen && - presult->nfrozen > 0 && - fpi_before != pgWalUsage.wal_fpi); - else - do_freeze = false; - if (do_freeze) { frz_conflict_horizon = heap_frz_conflict_horizon(presult, pagefrz); -- 2.40.1 --racicctn4wry6xe5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v3-0009-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch" ^ permalink raw reply [nested|flat] 13+ messages in thread
* [PATCH v4 10/19] Make opp freeze heuristic compatible with prune+freeze record @ 2024-01-07 21:11 Melanie Plageman <[email protected]> 0 siblings, 0 replies; 13+ messages in thread From: Melanie Plageman @ 2024-01-07 21:11 UTC (permalink / raw) Once the prune and freeze records are combined, we will no longer be able to use a test of whether or not pruning emitted an FPI to decide whether or not to opportunistically freeze a freezable page. While this heuristic should be improved, for now, approximate the previous logic by keeping track of whether or not a hint bit FPI was emitted during visibility checks (when checksums are on) and combine that with checking XLogCheckBufferNeedsBackup(). If we just finished deciding whether or not to prune and the current buffer seems to need an FPI after modification, it is likely that pruning would have emitted an FPI. --- src/backend/access/heap/pruneheap.c | 57 +++++++++++++++++++++-------- 1 file changed, 42 insertions(+), 15 deletions(-) diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 20907ba5408..9edf6bf72d7 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -246,6 +246,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, PruneState prstate; HeapTupleData tup; bool do_freeze; + bool do_prune; + bool whole_page_freezable; + bool hint_bit_fpi; + bool prune_fpi = false; int64 fpi_before = pgWalUsage.wal_fpi; /* @@ -439,6 +443,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, } } + /* + * If checksums are enabled, heap_prune_satisfies_vacuum() may have caused + * an FPI to be emitted. Then reset fpi_before for no prune case. + */ + hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi; + fpi_before = pgWalUsage.wal_fpi; + /* * For vacuum, if the whole page will become frozen, we consider * opportunistically freezing tuples. Dead tuples which will be removed by @@ -483,11 +494,41 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, if (off_loc) *off_loc = InvalidOffsetNumber; + do_prune = prstate.nredirected > 0 || + prstate.ndead > 0 || + prstate.nunused > 0; + + /* + * Only incur overhead of checking if we will do an FPI if we might use + * the information. + */ + if (do_prune && pagefrz) + prune_fpi = XLogCheckBufferNeedsBackup(buffer); + + /* Is the whole page freezable? And is there something to freeze */ + whole_page_freezable = presult->all_visible_except_removable && + presult->all_frozen; + + /* + * Freeze the page when heap_prepare_freeze_tuple indicates that at least + * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also + * freeze when pruning generated an FPI, if doing so means that we set the + * page all-frozen afterwards (might not happen until final heap pass). + * XXX: Previously, we knew if pruning emitted an FPI by checking + * pgWalUsage.wal_fpi before and after pruning. Once the freeze and prune + * records are combined, this heuristic couldn't be used anymore. The + * opportunistic freeze heuristic must be improved; however, for now, try + * to approximate it. + */ + do_freeze = pagefrz && + (pagefrz->freeze_required || + (whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi))); + /* Any error while applying the changes is critical */ START_CRIT_SECTION(); /* Have we found any prunable items? */ - if (prstate.nredirected > 0 || prstate.ndead > 0 || prstate.nunused > 0) + if (do_prune) { /* * Apply the planned item changes, then repair page fragmentation, and @@ -579,20 +620,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, /* Record number of newly-set-LP_DEAD items for caller */ presult->nnewlpdead = prstate.ndead; - /* - * Freeze the page when heap_prepare_freeze_tuple indicates that at least - * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also - * freeze when pruning generated an FPI, if doing so means that we set the - * page all-frozen afterwards (might not happen until final heap pass). - */ - if (pagefrz) - do_freeze = pagefrz->freeze_required || - (presult->all_visible_except_removable && presult->all_frozen && - presult->nfrozen > 0 && - fpi_before != pgWalUsage.wal_fpi); - else - do_freeze = false; - if (do_freeze) { /* -- 2.40.1 --tez7m2a73jtztiij Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v4-0011-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch" ^ permalink raw reply [nested|flat] 13+ messages in thread
* [PATCH v2 08/17] Make opp freeze heuristic compatible with prune+freeze record @ 2024-01-07 21:11 Melanie Plageman <[email protected]> 0 siblings, 0 replies; 13+ messages in thread From: Melanie Plageman @ 2024-01-07 21:11 UTC (permalink / raw) Once the prune and freeze records are combined, we will no longer be able to use a test of whether or not pruning emitted an FPI to decide whether or not to opportunistically freeze a freezable page. While this heuristic should be improved, for now, approximate the previous logic by keeping track of whether or not a hint bit FPI was emitted during visibility checks (when checksums are on) and combine that with checking XLogCheckBufferNeedsBackup(). If we just finished deciding whether or not to prune and the current buffer seems to need an FPI after modification, it is likely that pruning would have emitted an FPI. --- src/backend/access/heap/pruneheap.c | 58 +++++++++++++++++++++-------- 1 file changed, 43 insertions(+), 15 deletions(-) diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 9c709315192..e715fc29a83 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -241,6 +241,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, PruneState prstate; HeapTupleData tup; bool do_freeze; + bool do_prune; + bool whole_page_freezable; + bool hint_bit_fpi; + bool prune_fpi = false; int64 fpi_before = pgWalUsage.wal_fpi; TransactionId frz_conflict_horizon = InvalidTransactionId; @@ -410,6 +414,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, } } + /* + * If checksums are enabled, heap_prune_satisfies_vacuum() may have caused + * an FPI to be emitted. Then reset fpi_before for no prune case. + */ + hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi; + fpi_before = pgWalUsage.wal_fpi; + /* * For vacuum, if the whole page will become frozen, we consider * opportunistically freezing tuples. Dead tuples which will be removed by @@ -467,11 +478,42 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, if (off_loc) *off_loc = InvalidOffsetNumber; + do_prune = prstate.nredirected > 0 || + prstate.ndead > 0 || + prstate.nunused > 0; + + /* + * Only incur overhead of checking if we will do an FPI if we might use + * the information. + */ + if (do_prune && pagefrz) + prune_fpi = XLogCheckBufferNeedsBackup(buffer); + + /* Is the whole page freezable? And is there something to freeze */ + whole_page_freezable = presult->all_visible_except_removable && + presult->all_frozen; + + /* + * Freeze the page when heap_prepare_freeze_tuple indicates that at least + * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also + * freeze when pruning generated an FPI, if doing so means that we set the + * page all-frozen afterwards (might not happen until final heap pass). + * XXX: Previously, we knew if pruning emitted an FPI by checking + * pgWalUsage.wal_fpi before and after pruning. Once the freeze and prune + * records are combined, this heuristic couldn't be used anymore. The + * opportunistic freeze heuristic must be improved; however, for now, try + * to approximate it. + */ + + do_freeze = pagefrz && + (pagefrz->freeze_required || + (whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi))); + /* Any error while applying the changes is critical */ START_CRIT_SECTION(); /* Have we found any prunable items? */ - if (prstate.nredirected > 0 || prstate.ndead > 0 || prstate.nunused > 0) + if (do_prune) { /* * Apply the planned item changes, then repair page fragmentation, and @@ -563,20 +605,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, /* Record number of newly-set-LP_DEAD items for caller */ presult->nnewlpdead = prstate.ndead; - /* - * Freeze the page when heap_prepare_freeze_tuple indicates that at least - * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also - * freeze when pruning generated an FPI, if doing so means that we set the - * page all-frozen afterwards (might not happen until final heap pass). - */ - if (pagefrz) - do_freeze = pagefrz->freeze_required || - (presult->all_visible_except_removable && presult->all_frozen && - presult->nfrozen > 0 && - fpi_before != pgWalUsage.wal_fpi); - else - do_freeze = false; - if (do_freeze) { frz_conflict_horizon = heap_frz_conflict_horizon(presult, pagefrz); -- 2.40.1 --rdqtp5puvxqotfdw Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v2-0009-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch" ^ permalink raw reply [nested|flat] 13+ messages in thread
* [PATCH v3 08/17] Make opp freeze heuristic compatible with prune+freeze record @ 2024-01-07 21:11 Melanie Plageman <[email protected]> 0 siblings, 0 replies; 13+ messages in thread From: Melanie Plageman @ 2024-01-07 21:11 UTC (permalink / raw) Once the prune and freeze records are combined, we will no longer be able to use a test of whether or not pruning emitted an FPI to decide whether or not to opportunistically freeze a freezable page. While this heuristic should be improved, for now, approximate the previous logic by keeping track of whether or not a hint bit FPI was emitted during visibility checks (when checksums are on) and combine that with checking XLogCheckBufferNeedsBackup(). If we just finished deciding whether or not to prune and the current buffer seems to need an FPI after modification, it is likely that pruning would have emitted an FPI. --- src/backend/access/heap/pruneheap.c | 58 +++++++++++++++++++++-------- 1 file changed, 43 insertions(+), 15 deletions(-) diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index abf6bdb2d99..f164b7957ed 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -255,6 +255,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, PruneState prstate; HeapTupleData tup; bool do_freeze; + bool do_prune; + bool whole_page_freezable; + bool hint_bit_fpi; + bool prune_fpi = false; int64 fpi_before = pgWalUsage.wal_fpi; TransactionId frz_conflict_horizon = InvalidTransactionId; @@ -424,6 +428,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, } } + /* + * If checksums are enabled, heap_prune_satisfies_vacuum() may have caused + * an FPI to be emitted. Then reset fpi_before for no prune case. + */ + hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi; + fpi_before = pgWalUsage.wal_fpi; + /* * For vacuum, if the whole page will become frozen, we consider * opportunistically freezing tuples. Dead tuples which will be removed by @@ -481,11 +492,42 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, if (off_loc) *off_loc = InvalidOffsetNumber; + do_prune = prstate.nredirected > 0 || + prstate.ndead > 0 || + prstate.nunused > 0; + + /* + * Only incur overhead of checking if we will do an FPI if we might use + * the information. + */ + if (do_prune && pagefrz) + prune_fpi = XLogCheckBufferNeedsBackup(buffer); + + /* Is the whole page freezable? And is there something to freeze */ + whole_page_freezable = presult->all_visible_except_removable && + presult->all_frozen; + + /* + * Freeze the page when heap_prepare_freeze_tuple indicates that at least + * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also + * freeze when pruning generated an FPI, if doing so means that we set the + * page all-frozen afterwards (might not happen until final heap pass). + * XXX: Previously, we knew if pruning emitted an FPI by checking + * pgWalUsage.wal_fpi before and after pruning. Once the freeze and prune + * records are combined, this heuristic couldn't be used anymore. The + * opportunistic freeze heuristic must be improved; however, for now, try + * to approximate it. + */ + + do_freeze = pagefrz && + (pagefrz->freeze_required || + (whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi))); + /* Any error while applying the changes is critical */ START_CRIT_SECTION(); /* Have we found any prunable items? */ - if (prstate.nredirected > 0 || prstate.ndead > 0 || prstate.nunused > 0) + if (do_prune) { /* * Apply the planned item changes, then repair page fragmentation, and @@ -577,20 +619,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, /* Record number of newly-set-LP_DEAD items for caller */ presult->nnewlpdead = prstate.ndead; - /* - * Freeze the page when heap_prepare_freeze_tuple indicates that at least - * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also - * freeze when pruning generated an FPI, if doing so means that we set the - * page all-frozen afterwards (might not happen until final heap pass). - */ - if (pagefrz) - do_freeze = pagefrz->freeze_required || - (presult->all_visible_except_removable && presult->all_frozen && - presult->nfrozen > 0 && - fpi_before != pgWalUsage.wal_fpi); - else - do_freeze = false; - if (do_freeze) { frz_conflict_horizon = heap_frz_conflict_horizon(presult, pagefrz); -- 2.40.1 --racicctn4wry6xe5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v3-0009-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch" ^ permalink raw reply [nested|flat] 13+ messages in thread
* [PATCH v7 09/16] Make opp freeze heuristic compatible with prune+freeze record @ 2024-03-26 00:48 Melanie Plageman <[email protected]> 0 siblings, 0 replies; 13+ messages in thread From: Melanie Plageman @ 2024-03-26 00:48 UTC (permalink / raw) Once the prune and freeze records are combined, we will no longer be able to use a test of whether or not pruning emitted an FPI to decide whether or not to opportunistically freeze a freezable page. While this heuristic should be improved, for now, approximate the previous logic by keeping track of whether or not a hint bit FPI was emitted during visibility checks (when checksums are on) and combine that with checking XLogCheckBufferNeedsBackup(). If we just finished deciding whether or not to prune and the current buffer seems to need an FPI after modification, it is likely that pruning would have emitted an FPI. --- src/backend/access/heap/pruneheap.c | 57 +++++++++++++++++++++-------- 1 file changed, 42 insertions(+), 15 deletions(-) diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index e009c7579dd..d38de9b063d 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -247,6 +247,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, TransactionId visibility_cutoff_xid; bool do_freeze; bool all_visible_except_removable; + bool do_prune; + bool whole_page_freezable; + bool hint_bit_fpi; + bool prune_fpi = false; int64 fpi_before = pgWalUsage.wal_fpi; /* @@ -456,6 +460,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, } } + /* + * If checksums are enabled, heap_prune_satisfies_vacuum() may have caused + * an FPI to be emitted. Then reset fpi_before for no prune case. + */ + hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi; + fpi_before = pgWalUsage.wal_fpi; + /* * For vacuum, if the whole page will become frozen, we consider * opportunistically freezing tuples. Dead tuples which will be removed by @@ -500,11 +511,41 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, if (off_loc) *off_loc = InvalidOffsetNumber; + do_prune = prstate.nredirected > 0 || + prstate.ndead > 0 || + prstate.nunused > 0; + + /* + * Only incur overhead of checking if we will do an FPI if we might use + * the information. + */ + if (do_prune && pagefrz) + prune_fpi = XLogCheckBufferNeedsBackup(buffer); + + /* Is the whole page freezable? And is there something to freeze */ + whole_page_freezable = all_visible_except_removable && + presult->all_frozen; + + /* + * Freeze the page when heap_prepare_freeze_tuple indicates that at least + * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also + * freeze when pruning generated an FPI, if doing so means that we set the + * page all-frozen afterwards (might not happen until final heap pass). + * XXX: Previously, we knew if pruning emitted an FPI by checking + * pgWalUsage.wal_fpi before and after pruning. Once the freeze and prune + * records are combined, this heuristic couldn't be used anymore. The + * opportunistic freeze heuristic must be improved; however, for now, try + * to approximate it. + */ + do_freeze = pagefrz && + (pagefrz->freeze_required || + (whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi))); + /* Any error while applying the changes is critical */ START_CRIT_SECTION(); /* Have we found any prunable items? */ - if (prstate.nredirected > 0 || prstate.ndead > 0 || prstate.nunused > 0) + if (do_prune) { /* * Apply the planned item changes, then repair page fragmentation, and @@ -569,20 +610,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, /* Record number of newly-set-LP_DEAD items for caller */ presult->nnewlpdead = prstate.ndead; - /* - * Freeze the page when heap_prepare_freeze_tuple indicates that at least - * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also - * freeze when pruning generated an FPI, if doing so means that we set the - * page all-frozen afterwards (might not happen until final heap pass). - */ - if (pagefrz) - do_freeze = pagefrz->freeze_required || - (all_visible_except_removable && presult->all_frozen && - presult->nfrozen > 0 && - fpi_before != pgWalUsage.wal_fpi); - else - do_freeze = false; - if (do_freeze) { TransactionId frz_conflict_horizon = InvalidTransactionId; -- 2.40.1 --ck6erxojvlx23byk Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v7-0010-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch" ^ permalink raw reply [nested|flat] 13+ messages in thread
* [PATCH v7 09/16] Make opp freeze heuristic compatible with prune+freeze record @ 2024-03-26 00:48 Melanie Plageman <[email protected]> 0 siblings, 0 replies; 13+ messages in thread From: Melanie Plageman @ 2024-03-26 00:48 UTC (permalink / raw) Once the prune and freeze records are combined, we will no longer be able to use a test of whether or not pruning emitted an FPI to decide whether or not to opportunistically freeze a freezable page. While this heuristic should be improved, for now, approximate the previous logic by keeping track of whether or not a hint bit FPI was emitted during visibility checks (when checksums are on) and combine that with checking XLogCheckBufferNeedsBackup(). If we just finished deciding whether or not to prune and the current buffer seems to need an FPI after modification, it is likely that pruning would have emitted an FPI. --- src/backend/access/heap/pruneheap.c | 57 +++++++++++++++++++++-------- 1 file changed, 42 insertions(+), 15 deletions(-) diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index e009c7579dd..d38de9b063d 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -247,6 +247,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, TransactionId visibility_cutoff_xid; bool do_freeze; bool all_visible_except_removable; + bool do_prune; + bool whole_page_freezable; + bool hint_bit_fpi; + bool prune_fpi = false; int64 fpi_before = pgWalUsage.wal_fpi; /* @@ -456,6 +460,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, } } + /* + * If checksums are enabled, heap_prune_satisfies_vacuum() may have caused + * an FPI to be emitted. Then reset fpi_before for no prune case. + */ + hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi; + fpi_before = pgWalUsage.wal_fpi; + /* * For vacuum, if the whole page will become frozen, we consider * opportunistically freezing tuples. Dead tuples which will be removed by @@ -500,11 +511,41 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, if (off_loc) *off_loc = InvalidOffsetNumber; + do_prune = prstate.nredirected > 0 || + prstate.ndead > 0 || + prstate.nunused > 0; + + /* + * Only incur overhead of checking if we will do an FPI if we might use + * the information. + */ + if (do_prune && pagefrz) + prune_fpi = XLogCheckBufferNeedsBackup(buffer); + + /* Is the whole page freezable? And is there something to freeze */ + whole_page_freezable = all_visible_except_removable && + presult->all_frozen; + + /* + * Freeze the page when heap_prepare_freeze_tuple indicates that at least + * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also + * freeze when pruning generated an FPI, if doing so means that we set the + * page all-frozen afterwards (might not happen until final heap pass). + * XXX: Previously, we knew if pruning emitted an FPI by checking + * pgWalUsage.wal_fpi before and after pruning. Once the freeze and prune + * records are combined, this heuristic couldn't be used anymore. The + * opportunistic freeze heuristic must be improved; however, for now, try + * to approximate it. + */ + do_freeze = pagefrz && + (pagefrz->freeze_required || + (whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi))); + /* Any error while applying the changes is critical */ START_CRIT_SECTION(); /* Have we found any prunable items? */ - if (prstate.nredirected > 0 || prstate.ndead > 0 || prstate.nunused > 0) + if (do_prune) { /* * Apply the planned item changes, then repair page fragmentation, and @@ -569,20 +610,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, /* Record number of newly-set-LP_DEAD items for caller */ presult->nnewlpdead = prstate.ndead; - /* - * Freeze the page when heap_prepare_freeze_tuple indicates that at least - * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also - * freeze when pruning generated an FPI, if doing so means that we set the - * page all-frozen afterwards (might not happen until final heap pass). - */ - if (pagefrz) - do_freeze = pagefrz->freeze_required || - (all_visible_except_removable && presult->all_frozen && - presult->nfrozen > 0 && - fpi_before != pgWalUsage.wal_fpi); - else - do_freeze = false; - if (do_freeze) { TransactionId frz_conflict_horizon = InvalidTransactionId; -- 2.40.1 --ck6erxojvlx23byk Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v7-0010-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch" ^ permalink raw reply [nested|flat] 13+ messages in thread
* [PATCH v9 09/21] Make opp freeze heuristic compatible with prune+freeze record @ 2024-03-26 00:48 Melanie Plageman <[email protected]> 0 siblings, 0 replies; 13+ messages in thread From: Melanie Plageman @ 2024-03-26 00:48 UTC (permalink / raw) Once the prune and freeze records are combined, we will no longer be able to use a test of whether or not pruning emitted an FPI to decide whether or not to opportunistically freeze a freezable page. While this heuristic should be improved, for now, approximate the previous logic by keeping track of whether or not a hint bit FPI was emitted during visibility checks (when checksums are on) and combine that with checking XLogCheckBufferNeedsBackup(). If we just finished deciding whether or not to prune and the current buffer seems to need an FPI after modification, it is likely that pruning would have emitted an FPI. --- src/backend/access/heap/pruneheap.c | 57 +++++++++++++++++++++-------- 1 file changed, 42 insertions(+), 15 deletions(-) diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 312695f806c..f0decff35dc 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -247,6 +247,10 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, TransactionId visibility_cutoff_xid; bool do_freeze; bool all_visible_except_removable; + bool do_prune; + bool whole_page_freezable; + bool hint_bit_fpi; + bool prune_fpi = false; int64 fpi_before = pgWalUsage.wal_fpi; /* @@ -456,6 +460,13 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, } } + /* + * If checksums are enabled, heap_prune_satisfies_vacuum() may have caused + * an FPI to be emitted. Then reset fpi_before for no prune case. + */ + hint_bit_fpi = fpi_before != pgWalUsage.wal_fpi; + fpi_before = pgWalUsage.wal_fpi; + /* * For vacuum, if the whole page will become frozen, we consider * opportunistically freezing tuples. Dead tuples which will be removed by @@ -500,11 +511,41 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, if (off_loc) *off_loc = InvalidOffsetNumber; + do_prune = prstate.nredirected > 0 || + prstate.ndead > 0 || + prstate.nunused > 0; + + /* + * Only incur overhead of checking if we will do an FPI if we might use + * the information. + */ + if (do_prune && pagefrz) + prune_fpi = XLogCheckBufferNeedsBackup(buffer); + + /* Is the whole page freezable? And is there something to freeze */ + whole_page_freezable = all_visible_except_removable && + presult->all_frozen; + + /* + * Freeze the page when heap_prepare_freeze_tuple indicates that at least + * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also + * freeze when pruning generated an FPI, if doing so means that we set the + * page all-frozen afterwards (might not happen until final heap pass). + * XXX: Previously, we knew if pruning emitted an FPI by checking + * pgWalUsage.wal_fpi before and after pruning. Once the freeze and prune + * records are combined, this heuristic couldn't be used anymore. The + * opportunistic freeze heuristic must be improved; however, for now, try + * to approximate it. + */ + do_freeze = pagefrz && + (pagefrz->freeze_required || + (whole_page_freezable && presult->nfrozen > 0 && (prune_fpi || hint_bit_fpi))); + /* Any error while applying the changes is critical */ START_CRIT_SECTION(); /* Have we found any prunable items? */ - if (prstate.nredirected > 0 || prstate.ndead > 0 || prstate.nunused > 0) + if (do_prune) { /* * Apply the planned item changes, then repair page fragmentation, and @@ -569,20 +610,6 @@ heap_page_prune_and_freeze(Relation relation, Buffer buffer, /* Record number of newly-set-LP_DEAD items for caller */ presult->nnewlpdead = prstate.ndead; - /* - * Freeze the page when heap_prepare_freeze_tuple indicates that at least - * one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also - * freeze when pruning generated an FPI, if doing so means that we set the - * page all-frozen afterwards (might not happen until final heap pass). - */ - if (pagefrz) - do_freeze = pagefrz->freeze_required || - (all_visible_except_removable && presult->all_frozen && - presult->nfrozen > 0 && - fpi_before != pgWalUsage.wal_fpi); - else - do_freeze = false; - if (do_freeze) { TransactionId frz_conflict_horizon = InvalidTransactionId; -- 2.40.1 --caj67xgx3lukmr5f Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v9-0010-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch" ^ permalink raw reply [nested|flat] 13+ messages in thread
* [PATCH v2 08/15] predicate.h almost doesn't need lock.h @ 2026-03-22 16:59 Álvaro Herrera <[email protected]> 0 siblings, 0 replies; 13+ messages in thread From: Álvaro Herrera @ 2026-03-22 16:59 UTC (permalink / raw) --- src/backend/access/heap/heapam_handler.c | 1 + src/backend/access/index/indexam.c | 1 + src/include/storage/predicate.h | 7 ++++++- 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 253a735b6c1..d40878928e1 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -40,6 +40,7 @@ #include "storage/bufmgr.h" #include "storage/bufpage.h" #include "storage/lmgr.h" +#include "storage/lock.h" #include "storage/predicate.h" #include "storage/procarray.h" #include "storage/smgr.h" diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c index 5eb7e99ad3e..fbfc33159eb 100644 --- a/src/backend/access/index/indexam.c +++ b/src/backend/access/index/indexam.c @@ -53,6 +53,7 @@ #include "nodes/execnodes.h" #include "pgstat.h" #include "storage/lmgr.h" +#include "storage/lock.h" #include "storage/predicate.h" #include "utils/ruleutils.h" #include "utils/snapmgr.h" diff --git a/src/include/storage/predicate.h b/src/include/storage/predicate.h index 607d7d84596..a5ac55b8f7e 100644 --- a/src/include/storage/predicate.h +++ b/src/include/storage/predicate.h @@ -14,11 +14,16 @@ #ifndef PREDICATE_H #define PREDICATE_H +#include "access/transam.h" #include "storage/itemptr.h" -#include "storage/lock.h" #include "utils/relcache.h" #include "utils/snapshot.h" +/* + * forward references in this file + */ +typedef struct VirtualTransactionId VirtualTransactionId; + /* * GUC variables -- 2.47.3 --3mo6hy575thcal37 Content-Type: text/x-diff; charset=utf-8 Content-Disposition: attachment; filename="v2-0009-twophase.h-can-mostly-do-without-lock.h.patch" ^ permalink raw reply [nested|flat] 13+ messages in thread
end of thread, other threads:[~2026-03-22 16:59 UTC | newest] Thread overview: 13+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2024-01-07 21:11 [PATCH v4 10/19] Make opp freeze heuristic compatible with prune+freeze record Melanie Plageman <[email protected]> 2024-01-07 21:11 [PATCH v3 08/17] Make opp freeze heuristic compatible with prune+freeze record Melanie Plageman <[email protected]> 2024-01-07 21:11 [PATCH v2 08/17] Make opp freeze heuristic compatible with prune+freeze record Melanie Plageman <[email protected]> 2024-01-07 21:11 [PATCH v4 10/19] Make opp freeze heuristic compatible with prune+freeze record Melanie Plageman <[email protected]> 2024-01-07 21:11 [PATCH v4 10/19] Make opp freeze heuristic compatible with prune+freeze record Melanie Plageman <[email protected]> 2024-01-07 21:11 [PATCH v2 08/17] Make opp freeze heuristic compatible with prune+freeze record Melanie Plageman <[email protected]> 2024-01-07 21:11 [PATCH v3 08/17] Make opp freeze heuristic compatible with prune+freeze record Melanie Plageman <[email protected]> 2024-01-07 21:11 [PATCH v2 08/17] Make opp freeze heuristic compatible with prune+freeze record Melanie Plageman <[email protected]> 2024-01-07 21:11 [PATCH v3 08/17] Make opp freeze heuristic compatible with prune+freeze record Melanie Plageman <[email protected]> 2024-03-26 00:48 [PATCH v7 09/16] Make opp freeze heuristic compatible with prune+freeze record Melanie Plageman <[email protected]> 2024-03-26 00:48 [PATCH v7 09/16] Make opp freeze heuristic compatible with prune+freeze record Melanie Plageman <[email protected]> 2024-03-26 00:48 [PATCH v9 09/21] Make opp freeze heuristic compatible with prune+freeze record Melanie Plageman <[email protected]> 2026-03-22 16:59 [PATCH v2 08/15] predicate.h almost doesn't need lock.h Álvaro Herrera <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox