public inbox for [email protected]
help / color / mirror / Atom feedFrom: Greg Burd <[email protected]>
To: pgsql-hackers <[email protected]>
Subject: Re: Tepid: selective index updates for heap relations
Date: Wed, 08 Jul 2026 10:05:24 -0400
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
<[email protected]>
<[email protected]>
<[email protected]>
<[email protected]>
Hello,
I've tried to call out below all the issues that I think will need
discussion rather than ask you to identify them so as to make better
use of everyone's precious time. Thanks in advance.
Attached is a rebased patch set (v57, on 16a4b3ef8ee) for HOT-indexed
updates -- extending the HOT optimization so that an UPDATE which
changes an indexed column only maintains the indexes whose attributes
actually changed, rather than falling back to a full non-HOT update.
This is the mechanism the "HOT expression updates" thread and, before
it, WARM and PHOT were all circling.
Rather than lead with benchmarks, I want to put the contentious design
decisions on the table first, because if the core model is a non-starter
on principle I would much rather hear that now than after another round
of implementation.
I have tried to write this the way I would want to review it: every open
question stated as a question, with the options I considered and why the
patch currently chooses what it does, so the debate can be about the
choice and not about excavating what the choice even was.
I owe an obvious debt to Pavan Deolasee (HOT, WARM), to Nathan Bossart's
partial-HOT sketch, and to everyone who pushed back on those threads --
much of what follows is an attempt to answer objections that were raised
years ago and never adequately resolved.
== The one question that gates everything else ==
Q0. Is attribute-bitmap staleness -- as opposed to a value recheck -- an
acceptable foundation at all?
A HOT-indexed update writes the new version as a heap-only tuple and
appends, inline in the line-pointer item past the tuple's natts, a
fixed-size bitmap of which indexed attributes changed at that hop. A
reader arriving through a possibly-stale index entry walks the chain
and unions the bitmaps it crosses; if the union overlaps the scanned
index's attributes, the entry is stale and skipped. There is no
comparison of the leaf's stored key against the live tuple.
This is the decision that plagued WARM, so I want to be explicit about
why the patch does NOT recheck values:
- A value recheck is provably wrong under an ABA cycle. If an
indexed value goes X -> Y -> X, the live tuple's key equals BOTH
the old and the new leaf, so both pass a value recheck and the row
is returned twice. We hit exactly this. The crossed-attribute
bitmap gets it right because it is positional: the fresh entry
points at the version whose key it matched and crosses no later
key-changing hop, while the stale entry's walk does cross one --
even when the values coincide.
- Because the read side never reconstructs or compares a key, it is
access-method agnostic. btree, hash, GIN, GiST and SP-GiST all
work with no per-AM recheck logic (all exercised in the suite,
including the hash ABA case).
The cost of this choice, stated plainly: it adds a per-hop on-disk
bitmap and a chain-walk bitmap-union to the read path, and it weakens
the informal contract that "an index entry accurately reflects the
current indexed value." That contract-weakening is the real thing to
debate. My claim is that the contract is already softer than it looks
(kill_prior_tuple, bottom-up deletion and bitmap scans all tolerate
entries that no longer match), but I would like that challenged
directly.
Options if the list rejects bitmap-staleness:
(a) Value recheck instead -- rejected above (ABA), and it
reintroduces a per-scan leaf-materialize cost this design does
not have.
(b) Do nothing / keep HOT as-is -- the status quo; the
WARM/PHOT/expression threads exist because the write
amplification is real.
(c) A different staleness authority I haven't thought of, and I am
genuinely open to this.
== Is this WARM again? ==
Let's review the four objections that stalled it. If the answer to Q0
is "maybe," these are the specific WARM objections and how the patch
answers them. I would like to know whether each answer actually lands
or whether I am repeating history.
W1 -- Duplicate results from index scans. Answered by Q0: exactly one
entry per index survives the bitmap test; no duplicates, no lost
rows.
W2 -- Only one WARM update per chain (which crippled the benefit).
There is no chain-length cap here. A chain takes as many
HOT-indexed hops as fit on the page; growth is bounded by the
page (an update that no longer fits falls back to non-HOT) and
by prune/collapse reclaiming superseded members. (An early
draft had a fillfactor-derived cap; it measured from the wrong
end of the chain and bounded nothing, so it was removed. If the
list wants an explicit cap for predictability rather than
correctness, that is a knob we could add -- see Q3.)
W3 -- "Converting chains back" and its interaction with VACUUM was the
major sticking point. There is no convert-back step. Staleness
is decided at read time; prune collapses a dead chain prefix
into xid-free forwarding stubs that preserve each surviving
hop's bitmap, and VACUUM reclaims the stubs and re-points the
root redirect once the stale leaves are swept -- collapsing
naturally back to classic HOT. No alternating state machine, no
per-tuple flag to clear. Whether this collapse machinery is
actually simpler than WARM's, or just relocates the complexity,
is a fair question for reviewers of pruneheap.c / vacuumlazy.c.
W4 -- Recheck correctness across all index/scan kinds and opclasses.
Subsumed by the AM-agnostic read path in Q0.
== The on-disk format ==
Q1. Is the on-disk format acceptable to freeze?
Two new LP_NORMAL shapes carry one infomask2 bit
(HEAP_INDEXED_UPDATED, 0x0800, previously free):
- a data-bearing HOT-indexed tuple (natts >= 1) with a trailing
bitmap sized by the tuple's own write-time natts; and
- an xid-free collapse-survivor stub (natts == 0, a signature no
real tuple can produce) that stashes its write-time natts in the
block half of t_ctid and its forward link in the offset half.
Every consumer of LP_NORMAL items must tolerate both. I have audited
the in-tree consumers: visibility-gated paths (seqscan, bitmap,
ANALYZE, index build) are safe because stubs are XMIN_INVALID and the
bitmap is past natts; amcheck and prune/VACUUM are stub-aware;
pg_surgery skips stubs; pageinspect and pgstattuple are read-only and
merely imprecise.
Alternatives considered and rejected, with reasons, so the format
debate is concrete:
- separate relation fork: heavy, and the marker must be co-located
with the tuple for the chain walk;
- a separate adjacent "tombstone" line pointer per hop: doubles
line-pointer pressure; the inline bitmap needs no extra item;
- "redirect-with-data" LP_REDIRECT carrying the bitmap: LP_REDIRECT
has no storage for a payload;
- a new line-pointer flavor: consumes scarce lp_flags space and
touches far more code than reusing LP_NORMAL + one infomask2 bit.
The known-unknown I most want eyes on: is stealing an infomask2 bit
and overloading natts==0 as a stub sentinel acceptable, or does the
project want a cleaner (costlier) representation before this is worth
committing?
Q1a. The bit budget.
This is worth stating on its own because it is irreversible.
t_infomask2 has exactly two spare bits above the 11-bit natts mask:
0x0800 and 0x1000. This patch consumes 0x0800
(HEAP_INDEXED_UPDATED), leaving 0x1000 as the last free t_infomask2
bit in the heap format. I do not take that lightly.
The argument for spending one now: the feature is unimplementable
without a persistent per-tuple signal, and every alternative to a
bit (Q1's rejected list) costs more -- an lp_flags value, a fork,
or a second line pointer. The argument against: a bit is a scarce,
permanent resource and the list may want it reserved for something
the project already knows it wants. If the answer is "not this
bit," I need to hear it before the format freezes, because the
whole on-disk encoding hangs off it.
Q1b. Longer chains.
A chain here can be arbitrarily long within a page (Q3), which is a
real change from classic HOT's practical behavior and from WARM's
hard one-update cap (W2). The read cost is not free: a scan that
arrives through a stale-ish leaf walks the chain accumulating a
bitmap-union until it reaches the live tuple, so worst-case read
work is O(chain length) in bitmap-unions, and a page that
accumulates many preserved HOT-indexed members stays denser and
non-all-visible (Q1c) for longer.
My position: this is bounded in practice -- an update that no
longer fits the page falls back to non-HOT, and prune/collapse
reclaims superseded members and re-points the root redirect, so a
chain does not grow without bound and a hot row's chain is
continually shortened by the same VACUUM that would clean any other
bloat.
But "bounded in practice" is exactly the kind of claim WARM made,
so I want it challenged: is an uncapped chain length acceptable
given the O(length) read walk, or does predictability argue for the
cap discussed in Q3? The read_indexscan benchmark cell exists to
measure the walk cost on a stub-free table; I expect parity there
but will prove it on real hardware.
Q1c. The all-visible / index-only-scan interaction.
This one deserves an explicit debate because it is a visible
operational cost, not just an internal detail. A stale leaf can
resolve, through the chain, to a live tuple whose current key
differs from the leaf's stored key. If the page holding that chain
were marked all-visible, an index-only scan would take the
visibility-map fast path, skip the heap fetch, and return the
leaf's STALE key -- wrong results with no chance to detect it. The
patch closes that from the prune side: any page still carrying
something a stale leaf can resolve through -- a preserved live
HEAP_INDEXED_UPDATED member, an LP_REDIRECT forwarding into one, or
a collapse-survivor stub -- is deliberately kept OUT of the
visibility map (prune forces set_all_visible=false, re-applied in
heap_page_would_be_all_visible), and once forced to the heap the
IOS re-checks the arriving entry with the same bitmap test before
trusting xs_itup.
The consequence to debate: such a page is not all-visible until its
chains fully collapse, so index-only scans over hot,
recently-SIU-updated rows lose the VM fast path and pay a heap
fetch until VACUUM collapses the chains.
A conventionally-updated page reaches all-visible after one VACUUM;
an SIU page with a surviving stub needs the later VACUUM that
reclaims the stub and re-points the redirect.
I argue this is the correct conservative default (correctness over
an IOS micro-optimization on exactly the rows that are being
churned, and it self-heals as the chain collapses), and that IOS on
genuinely-stable all-visible pages is unaffected because such a
page cannot hold a live SIU chain. But it is a real regression for
an IOS-heavy workload on churned rows, and if the list considers
that unacceptable the alternative is a per-entry mechanism that
lets a page go all-visible while still forcing a fetch only for
stale leaves -- more complex, and I did not want to build it before
the model is accepted.
== Eligibility carve-outs: which are permanent vs provisional? ==
Q2. The patch is deliberately conservative about what takes the
HOT-indexed path. Some carve-outs are fundamental; others are "not
proven yet" and may be liftable. I want the list to tell me which
it considers permanent.
- System catalogs (permanent, I think):
Catalogs are reached through access paths -- systable scans,
SnapshotDirty unique checks, seqscans -- I have not proven safe,
so a catalog UPDATE stays classic HOT.
There is a second, mechanical reason this carve-out needs explicit
discussion: catalog tuples are not updated through the executor,
so the modified-indexed-attribute set that the executor computes
for a normal UPDATE (by diffing the old/new TupleTableSlots in
ExecUpdateModifiedIdxAttrs) simply does not exist on the catalog
path.
simple_heap_update -- the entry point CatalogTupleUpdate and
friends use -- therefore has to reconstruct an equivalent bitmap
itself: it fetches the old tuple by TID and diffs it against the
new one to derive modified_idx_attrs, duplicating in heapam.c the
logic the executor already performs. Today that duplicated path
deliberately keeps catalog updates on the classic-HOT track, so
the mirror only has to be correct enough to make the eligibility
decision, not to drive selective maintenance.
The debate: is a second copy of the "which indexed attributes
changed" computation -- one in the executor, one in
simple_heap_update -- acceptable, and is the right long-term
answer to (a) keep them separate and documented as mirrors, (b)
factor the diff into one shared helper both call, or (c) have the
catalog callers pass down the "replaces" array they already have
(most of them do) instead of re-deriving it?
The current code chooses (a) with a comment calling out the
repetition; I lean toward (c) eventually but did not want to churn
every catalog caller before the model is settled.
- Expression indexes (provisional):
The bitmap is attribute-granular and cannot tell whether the
expression's *value* changed, only whether an attribute it
references changed. This is precisely the "HOT expression
updates" thread's problem. I believe it is liftable the same way
the partial-index restriction was (predicate/expression-aware
maintenance), but it is not wired up and not tested, so it is a
carve-out for now.
Is attribute-granular the right initial granularity, or should
expression support be in the first committed version?
- Every indexed attribute changed (permanent, by construction):
Nothing can be skipped, so a plain non-HOT update is cheaper.
"Every" is an exact test -- there is deliberately no percentage
GUC. Q3 asks whether that is the right call (which I feel it is).
- The logical-replication apply path (per-subscription gate,
hot_indexed_on_apply = off / subset_only (default) / always):
A HOT-indexed update of a replica-identity attribute leaves a
stale leaf the apply worker's RI lookup must tolerate, which it
safely does only when the indexed attributes are a subset of the
replica identity.
Is a three-valued per-subscription option the right control
surface, or does replication safety want something
coarser/finer/entirely different?
== Smaller decisions worth a sentence each ==
Q3. No tuning knobs, which is a Good Thing (TM), right?
There is no chain-length cap and no "skip if < N% of indexes change"
GUC. My reasoning is correctness never needs them, and every knob
is a support burden and a planner-surprise. Counter-argument I take
seriously (and which Q1b sharpens): operators may want a
predictability cap even at some efficiency cost, precisely because
the read walk is O(chain length) and the all-visible loss (Q1c)
persists until collapse.
Add a cap, or hold the line and rely on page-fit + prune to bound
it?
Q4. Unique checks.
_bt_check_unique gains a value recheck (opclass comparator) used
ONLY to distinguish a real duplicate from a stale leaf during a
unique insert -- this is the one place a value comparison is
load-bearing, and it routes a genuine hit into the existing xwait
wait-and-recheck.
I believe this is both necessary and sufficient; I would like it
confirmed, since it is the subtlest correctness argument in the set.
Q5. catversion.
The patch adds a pg_subscription column, so it needs a catversion
bump; I have left a placeholder (value matching master, with an XXX
comment) rather than a real bump, so the series does not
rebase-conflict on catversion every time master moves.
== What is deferred, honestly ==
- Expression-index support (Q2). If even the first two commits in
this series land I'll revisit/finish this in the thread about
CF-5556.
- Broader concurrency / crash stress matrices beyond the current
isolation spec + crash-recovery TAP test under
wal_consistency_checking.
- I've done a representative-hardware benchmark sweep using a variety
of instance types in "the cloud" and found performance numers to be
stable and predictable. The A/B numbers show WAL update down ~23%
on a single-indexed-column update and up to ~59% on a wide table
where one of many indexes changes, throughput (TPS) is up ~18-26%,
and others are at parity. The results that matter (WAL and index
bloat on the write workloads; any read-path cost on a tombstone-free
table; net TPS on a mixed workload) are in a commit labled
DO-NOT-MERGE harness in the series so anyone can
reproduce/critique/etc.
== The ask ==
In priority order:
1. Q0 -- is bitmap-staleness acceptable in principle? Everything else
is wasted effort if this is a no.
2. W1-W4 -- do the answers actually address what sank WARM?
3. Q1 / Q1a -- can the on-disk format be frozen, and is spending one
of the two remaining t_infomask2 bits the right call?
4. Q1c -- is losing the index-only-scan all-visible fast path over
not-yet-collapsed chains an acceptable cost, or a blocker?
5. Q1b / Q3 -- is an uncapped, page-bounded chain length acceptable
given the O(length) read walk, or do we want an explicit cap?
6. Q2 -- which carve-outs are permanent; must expression indexes be in
v1; and is the duplicated catalog-path attr-diff the right
structure?
7. Benchmarking, where did I go wrong and/or what else would you like
to see qualified/tested empirically and under what conditions?
Thanks for reading this far, I look forward to constructive debate and
hope the community finds this valuable work.
best.
-greg
PS: Tepid is on the https://github.com/gburd/postgres/tree/tepid branch
Attachments:
[text/x-patch] v57-0001-Add-tests-to-cover-a-variety-of-heap-HOT-update-.patch (45.4K, ../[email protected]/2-v57-0001-Add-tests-to-cover-a-variety-of-heap-HOT-update-.patch)
download | inline diff:
From 96512aea38ae00ce1ed22f6adf7d07814a6d4c47 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Tue, 10 Mar 2026 09:28:15 -0400
Subject: [PATCH v57 1/9] Add tests to cover a variety of heap HOT update
behaviors
Add regression coverage for existing classic Heap-Only Tuple (HOT) update
behavior, committed first so the behavioral changes in the later HOT-indexed
commits are diffable against a known-good baseline. This commit adds
tests so as to codify explicitly the HOT contract for the heap AM.
The new hot_updates regression test exercises:
- Basic HOT vs non-HOT update decisions
- The all-or-none property across multiple indexes
- Partial indexes and predicate handling
- BRIN (summarizing) indexes allowing HOT updates
- TOAST column handling with HOT
- Unique constraint behavior
- Multi-column indexes
- Partitioned table HOT updates
- HOT chain formation and the index-scan walk over a chain
Authored-by: Greg Burd <[email protected]>
---
src/test/regress/expected/hot_updates.out | 745 ++++++++++++++++++++++
src/test/regress/parallel_schedule | 5 +
src/test/regress/sql/hot_updates.sql | 605 ++++++++++++++++++
3 files changed, 1355 insertions(+)
create mode 100644 src/test/regress/expected/hot_updates.out
create mode 100644 src/test/regress/sql/hot_updates.sql
diff --git a/src/test/regress/expected/hot_updates.out b/src/test/regress/expected/hot_updates.out
new file mode 100644
index 00000000000..273fe3310da
--- /dev/null
+++ b/src/test/regress/expected/hot_updates.out
@@ -0,0 +1,745 @@
+--
+-- HOT_UPDATES
+-- Test Heap-Only Tuple (HOT) update decisions
+--
+-- This test systematically verifies that HOT updates are used when appropriate
+-- and avoided when necessary (e.g., when indexed columns are modified).
+--
+-- We use multiple validation methods:
+-- 1. Statistics functions (pg_stat_get_tuples_hot_updated)
+-- 2. pageinspect extension for HOT chain examination
+-- 3. EXPLAIN to verify index usage after updates
+--
+-- Load required extensions
+CREATE EXTENSION IF NOT EXISTS pageinspect;
+-- Function to get HOT update count
+CREATE OR REPLACE FUNCTION get_hot_count(rel_name text)
+RETURNS TABLE (
+ updates BIGINT,
+ hot BIGINT
+) AS $$
+DECLARE
+ rel_oid oid;
+BEGIN
+ rel_oid := rel_name::regclass::oid;
+
+ -- Read both committed and transaction-local stats
+ -- In autocommit mode (default for regression tests), this works correctly
+ -- Note: In explicit transactions (BEGIN/COMMIT), committed stats already
+ -- include flushed updates, so this would double-count. For explicit
+ -- transaction testing, call pg_stat_force_next_flush() before this function.
+ updates := COALESCE(pg_stat_get_tuples_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_updated(rel_oid), 0);
+ hot := COALESCE(pg_stat_get_tuples_hot_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_hot_updated(rel_oid), 0);
+
+ RETURN NEXT;
+END;
+$$ LANGUAGE plpgsql;
+-- Check if a tuple is part of a HOT chain (has a predecessor on same page)
+CREATE OR REPLACE FUNCTION has_hot_chain(rel_name text, target_ctid tid)
+RETURNS boolean AS $$
+DECLARE
+ block_num int;
+ page_item record;
+BEGIN
+ block_num := (target_ctid::text::point)[0]::int;
+
+ -- Look for a different tuple on the same page that points to our target tuple
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp_flags = 1
+ AND t_ctid IS NOT NULL
+ AND t_ctid = target_ctid
+ AND ('(' || block_num::text || ',' || lp::text || ')')::tid != target_ctid
+ LOOP
+ RETURN true;
+ END LOOP;
+
+ RETURN false;
+END;
+$$ LANGUAGE plpgsql;
+-- Print the HOT chain starting from a given tuple
+CREATE OR REPLACE FUNCTION print_hot_chain(rel_name text, start_ctid tid)
+RETURNS TABLE(chain_position int, ctid tid, lp_flags text, t_ctid tid, chain_end boolean) AS
+$$
+#variable_conflict use_column
+DECLARE
+ block_num int;
+ line_ptr int;
+ current_ctid tid := start_ctid;
+ next_ctid tid;
+ position int := 0;
+ max_iterations int := 100;
+ page_item record;
+ found_predecessor boolean := false;
+ flags_name text;
+BEGIN
+ block_num := (start_ctid::text::point)[0]::int;
+
+ -- Find the predecessor (old tuple pointing to our start_ctid)
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp_flags = 1
+ AND t_ctid = start_ctid
+ LOOP
+ current_ctid := ('(' || block_num::text || ',' || page_item.lp::text || ')')::tid;
+ found_predecessor := true;
+ EXIT;
+ END LOOP;
+
+ -- If no predecessor found, start with the given ctid
+ IF NOT found_predecessor THEN
+ current_ctid := start_ctid;
+ END IF;
+
+ -- Follow the chain forward
+ WHILE position < max_iterations LOOP
+ line_ptr := (current_ctid::text::point)[1]::int;
+
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp = line_ptr
+ LOOP
+ -- Map lp_flags to names
+ flags_name := CASE page_item.lp_flags
+ WHEN 0 THEN 'unused (0)'
+ WHEN 1 THEN 'normal (1)'
+ WHEN 2 THEN 'redirect (2)'
+ WHEN 3 THEN 'dead (3)'
+ ELSE 'unknown (' || page_item.lp_flags::text || ')'
+ END;
+
+ RETURN QUERY SELECT
+ position,
+ current_ctid,
+ flags_name,
+ page_item.t_ctid,
+ (page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid)::boolean
+ ;
+
+ IF page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid THEN
+ RETURN;
+ END IF;
+
+ next_ctid := page_item.t_ctid;
+
+ IF (next_ctid::text::point)[0]::int != block_num THEN
+ RETURN;
+ END IF;
+
+ current_ctid := next_ctid;
+ position := position + 1;
+ END LOOP;
+
+ IF position = 0 THEN
+ RETURN;
+ END IF;
+ END LOOP;
+END;
+$$ LANGUAGE plpgsql;
+-- Basic HOT update (update non-indexed column)
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ non_indexed_col text
+) WITH (fillfactor = 50);
+CREATE INDEX hot_test_indexed_idx ON hot_test(indexed_col);
+INSERT INTO hot_test VALUES (1, 100, 'initial');
+INSERT INTO hot_test VALUES (2, 200, 'initial');
+INSERT INTO hot_test VALUES (3, 300, 'initial');
+-- Get baseline
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 0 | 0
+(1 row)
+
+-- Should be HOT updates (only non-indexed column modified)
+UPDATE hot_test SET non_indexed_col = 'updated1' WHERE id = 1;
+UPDATE hot_test SET non_indexed_col = 'updated2' WHERE id = 2;
+UPDATE hot_test SET non_indexed_col = 'updated3' WHERE id = 3;
+-- Verify HOT updates occurred
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 3 | 3
+(1 row)
+
+-- Dump the HOT chain before VACUUMing
+WITH current_tuple AS (
+ SELECT ctid FROM hot_test WHERE id = 1
+)
+SELECT
+ has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position,
+ print_hot_chain.ctid,
+ lp_flags,
+ t_ctid
+FROM current_tuple,
+LATERAL print_hot_chain('hot_test', current_tuple.ctid);
+ has_chain | chain_position | ctid | lp_flags | t_ctid
+-----------+----------------+-------+------------+--------
+ t | 0 | (0,1) | normal (1) | (0,4)
+ t | 1 | (0,4) | normal (1) | (0,4)
+(2 rows)
+
+-- Vacuum the relation, expect the HOT chain to collapse
+VACUUM hot_test;
+-- Show that there is no chain after vacuum
+WITH current_tuple AS (
+ SELECT ctid FROM hot_test WHERE id = 1
+)
+SELECT
+ has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position,
+ print_hot_chain.ctid,
+ lp_flags,
+ t_ctid
+FROM current_tuple,
+LATERAL print_hot_chain('hot_test', current_tuple.ctid);
+ has_chain | chain_position | ctid | lp_flags | t_ctid
+-----------+----------------+-------+------------+--------
+ f | 0 | (0,4) | normal (1) | (0,4)
+(1 row)
+
+-- Non-HOT update (update indexed column)
+UPDATE hot_test SET indexed_col = 150 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 4 | 3
+(1 row)
+
+-- Verify index was updated (new value findable)
+SET enable_seqscan = off;
+EXPLAIN (COSTS OFF) SELECT id, indexed_col FROM hot_test WHERE indexed_col = 150;
+ QUERY PLAN
+---------------------------------------------------
+ Index Scan using hot_test_indexed_idx on hot_test
+ Index Cond: (indexed_col = 150)
+(2 rows)
+
+SELECT id, indexed_col FROM hot_test WHERE indexed_col = 150;
+ id | indexed_col
+----+-------------
+ 1 | 150
+(1 row)
+
+-- Verify old value no longer in index
+EXPLAIN (COSTS OFF) SELECT id FROM hot_test WHERE indexed_col = 100;
+ QUERY PLAN
+---------------------------------------------------
+ Index Scan using hot_test_indexed_idx on hot_test
+ Index Cond: (indexed_col = 100)
+(2 rows)
+
+SELECT id FROM hot_test WHERE indexed_col = 100;
+ id
+----
+(0 rows)
+
+RESET enable_seqscan;
+-- All-or-none property: updating one indexed column requires ALL index updates
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ col_a int,
+ col_b int,
+ col_c int,
+ non_indexed text
+) WITH (fillfactor = 50);
+CREATE INDEX hot_test_a_idx ON hot_test(col_a);
+CREATE INDEX hot_test_b_idx ON hot_test(col_b);
+CREATE INDEX hot_test_c_idx ON hot_test(col_c);
+INSERT INTO hot_test VALUES (1, 10, 20, 30, 'initial');
+-- Update only col_a - should NOT be HOT because an indexed column changed
+-- This means ALL indexes must be updated (all-or-none property)
+UPDATE hot_test SET col_a = 15 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 1 | 0
+(1 row)
+
+-- Now update only non-indexed column - should be HOT
+UPDATE hot_test SET non_indexed = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 2 | 1
+(1 row)
+
+-- Partial index: both old and new outside predicate (conservative = non-HOT)
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ status text,
+ data text
+) WITH (fillfactor = 50);
+-- Partial index only covers status = 'active'
+CREATE INDEX hot_test_active_idx ON hot_test(status) WHERE status = 'active';
+INSERT INTO hot_test VALUES (1, 'active', 'data1');
+INSERT INTO hot_test VALUES (2, 'inactive', 'data2');
+INSERT INTO hot_test VALUES (3, 'deleted', 'data3');
+-- Update non-indexed column on 'active' row (in predicate, status unchanged)
+-- Should be HOT
+UPDATE hot_test SET data = 'updated1' WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 1 | 1
+(1 row)
+
+-- Update non-indexed column on 'inactive' row (outside predicate)
+-- Should be HOT
+UPDATE hot_test SET data = 'updated2' WHERE id = 2;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 2 | 2
+(1 row)
+
+-- Update status from 'inactive' to 'deleted' (both outside predicate)
+-- PostgreSQL is conservative: heap insert happens before predicate check
+-- So this is NON-HOT even though both values are outside predicate
+UPDATE hot_test SET status = 'deleted' WHERE id = 2;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 3 | 2
+(1 row)
+
+-- Verify index still works for 'active' rows
+SELECT id, status FROM hot_test WHERE status = 'active';
+ id | status
+----+--------
+ 1 | active
+(1 row)
+
+-- Only BRIN (summarizing) indexes on non-PK columns
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ ts timestamp,
+ value int,
+ brin_col int
+) WITH (fillfactor = 50);
+CREATE INDEX hot_test_ts_brin ON hot_test USING brin(ts);
+CREATE INDEX hot_test_brin_col_brin ON hot_test USING brin(brin_col);
+INSERT INTO hot_test VALUES (1, '2024-01-01', 100, 1000);
+-- Update both BRIN columns - should still be HOT (only summarizing indexes)
+UPDATE hot_test SET ts = '2024-01-02', brin_col = 2000 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 1 | 1
+(1 row)
+
+-- Update non-indexed column - should also be HOT
+UPDATE hot_test SET value = 200 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 2 | 2
+(1 row)
+
+-- TOAST and HOT: TOASTed columns can participate in HOT
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ large_text text,
+ small_text text
+) WITH (fillfactor = 50);
+CREATE INDEX hot_test_idx ON hot_test(indexed_col);
+-- Insert row with TOASTed column (> 2KB)
+INSERT INTO hot_test VALUES (1, 100, repeat('x', 3000), 'small');
+-- Update non-indexed, non-TOASTed column - should be HOT
+UPDATE hot_test SET small_text = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 1 | 1
+(1 row)
+
+-- Update TOASTed column - should be HOT if indexed column unchanged
+UPDATE hot_test SET large_text = repeat('y', 3000);
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 2 | 2
+(1 row)
+
+-- Update indexed column - should NOT be HOT
+UPDATE hot_test SET indexed_col = 200;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 3 | 2
+(1 row)
+
+-- Unique constraint (unique index) behaves like regular index
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ unique_col int UNIQUE,
+ data text
+) WITH (fillfactor = 50);
+INSERT INTO hot_test VALUES (1, 100, 'data1');
+INSERT INTO hot_test VALUES (2, 200, 'data2');
+-- Update data (non-indexed) - should be HOT
+UPDATE hot_test SET data = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 2 | 2
+(1 row)
+
+-- Verify unique constraint still enforced
+SELECT id, unique_col, data FROM hot_test ORDER BY id;
+ id | unique_col | data
+----+------------+---------
+ 1 | 100 | updated
+ 2 | 200 | updated
+(2 rows)
+
+-- This should fail (unique violation)
+UPDATE hot_test SET unique_col = 100 WHERE id = 2;
+ERROR: duplicate key value violates unique constraint "hot_test_unique_col_key"
+DETAIL: Key (unique_col)=(100) already exists.
+-- Multi-column index: any column change = non-HOT
+DROP TABLE hot_test;
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ col_a int,
+ col_b int,
+ col_c int,
+ data text
+) WITH (fillfactor = 50);
+CREATE INDEX hot_test_ab_idx ON hot_test(col_a, col_b);
+INSERT INTO hot_test VALUES (1, 10, 20, 30, 'data');
+-- Update col_a (part of multi-column index) - should NOT be HOT
+UPDATE hot_test SET col_a = 15;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 1 | 0
+(1 row)
+
+-- Reset
+UPDATE hot_test SET col_a = 10;
+-- Update col_b (part of multi-column index) - should NOT be HOT
+UPDATE hot_test SET col_b = 25;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 3 | 0
+(1 row)
+
+-- Reset
+UPDATE hot_test SET col_b = 20;
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 4 | 0
+(1 row)
+
+-- Update col_c (not indexed) - should be HOT
+UPDATE hot_test SET col_c = 35;
+-- Update data (not indexed) - should be HOT
+UPDATE hot_test SET data = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 6 | 2
+(1 row)
+
+-- Partitioned tables: HOT works within partitions
+DROP TABLE IF EXISTS hot_test_partitioned CASCADE;
+NOTICE: table "hot_test_partitioned" does not exist, skipping
+CREATE TABLE hot_test_partitioned (
+ id int,
+ partition_key int,
+ indexed_col int,
+ data text,
+ PRIMARY KEY (id, partition_key)
+) PARTITION BY RANGE (partition_key);
+CREATE TABLE hot_test_part1 PARTITION OF hot_test_partitioned
+ FOR VALUES FROM (1) TO (100) WITH (fillfactor = 50);
+CREATE TABLE hot_test_part2 PARTITION OF hot_test_partitioned
+ FOR VALUES FROM (100) TO (200) WITH (fillfactor = 50);
+CREATE INDEX hot_test_part_idx ON hot_test_partitioned(indexed_col);
+INSERT INTO hot_test_partitioned VALUES (1, 50, 100, 'initial1');
+INSERT INTO hot_test_partitioned VALUES (2, 150, 200, 'initial2');
+-- Update in partition 1 (non-indexed column) - should be HOT
+UPDATE hot_test_partitioned SET data = 'updated1' WHERE id = 1;
+-- Update in partition 2 (non-indexed column) - should be HOT
+UPDATE hot_test_partitioned SET data = 'updated2' WHERE id = 2;
+SELECT * FROM get_hot_count('hot_test_part1');
+ updates | hot
+---------+-----
+ 1 | 1
+(1 row)
+
+SELECT * FROM get_hot_count('hot_test_part2');
+ updates | hot
+---------+-----
+ 1 | 1
+(1 row)
+
+-- Verify indexes work on partitions
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 100;
+ id
+----
+ 1
+(1 row)
+
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 200;
+ id
+----
+ 2
+(1 row)
+
+-- Update indexed column in partition - should NOT be HOT
+UPDATE hot_test_partitioned SET indexed_col = 150 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test_part1');
+ updates | hot
+---------+-----
+ 2 | 1
+(1 row)
+
+-- Verify index was updated
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 150;
+ id
+----
+ 1
+(1 row)
+
+-- ============================================================================
+-- Trigger modifications: heap_modify_tuple() and HOT
+-- ============================================================================
+-- Test that we correctly detect when triggers modify indexed columns via
+-- heap_modify_tuple(), even when those columns aren't in the UPDATE's SET clause
+CREATE TABLE hot_trigger_test (
+ id int PRIMARY KEY,
+ triggered_col int,
+ data text
+) WITH (fillfactor = 50);
+CREATE INDEX hot_trigger_idx ON hot_trigger_test(triggered_col);
+-- Create a trigger that modifies an indexed column
+CREATE OR REPLACE FUNCTION modify_triggered_col()
+RETURNS TRIGGER AS $$
+BEGIN
+ NEW.triggered_col = NEW.triggered_col + 1;
+ RETURN NEW;
+END;
+$$ LANGUAGE plpgsql;
+CREATE TRIGGER before_update_modify
+ BEFORE UPDATE ON hot_trigger_test
+ FOR EACH ROW
+ EXECUTE FUNCTION modify_triggered_col();
+INSERT INTO hot_trigger_test VALUES (1, 100, 'initial');
+SELECT * FROM get_hot_count('hot_trigger_test');
+ updates | hot
+---------+-----
+ 0 | 0
+(1 row)
+
+-- Update only data column, but trigger modifies indexed column
+-- Should NOT be HOT because trigger modified an indexed column
+UPDATE hot_trigger_test SET data = 'updated' WHERE id = 1;
+-- Verify it was NOT a HOT update (indexed column was modified by trigger)
+SELECT * FROM get_hot_count('hot_trigger_test');
+ updates | hot
+---------+-----
+ 1 | 0
+(1 row)
+
+-- Verify the triggered column was actually modified
+SELECT triggered_col FROM hot_trigger_test WHERE id = 1;
+ triggered_col
+---------------
+ 101
+(1 row)
+
+DROP TABLE hot_trigger_test CASCADE;
+DROP FUNCTION modify_triggered_col();
+-- ============================================================================
+-- JSONB expression indexes and sub-attribute tracking
+-- ============================================================================
+-- Test that updates to non-indexed JSONB paths can be HOT updates
+CREATE TABLE hot_jsonb_test (
+ id int PRIMARY KEY,
+ data jsonb
+) WITH (fillfactor = 50);
+-- Create expression index on a specific JSON path
+CREATE INDEX hot_jsonb_name_idx ON hot_jsonb_test ((data->>'name'));
+INSERT INTO hot_jsonb_test VALUES
+ (1, '{"name":"Alice","age":30,"city":"NYC"}'),
+ (2, '{"name":"Bob","age":25,"city":"LA"}');
+SELECT * FROM get_hot_count('hot_jsonb_test');
+ updates | hot
+---------+-----
+ 0 | 0
+(1 row)
+
+-- Update non-indexed JSON path (age) - should be HOT after instrumentation
+UPDATE hot_jsonb_test SET data = jsonb_set(data, '{age}', '31') WHERE id = 1;
+SELECT * FROM get_hot_count('hot_jsonb_test');
+ updates | hot
+---------+-----
+ 1 | 0
+(1 row)
+
+-- Update indexed JSON path (name) - should NOT be HOT
+UPDATE hot_jsonb_test SET data = jsonb_set(data, '{name}', '"Alice2"') WHERE id = 1;
+SELECT * FROM get_hot_count('hot_jsonb_test');
+ updates | hot
+---------+-----
+ 2 | 0
+(1 row)
+
+-- Verify index works
+SELECT id FROM hot_jsonb_test WHERE data->>'name' = 'Alice2';
+ id
+----
+ 1
+(1 row)
+
+-- Test jsonb_delete on non-indexed path - should be HOT after instrumentation
+UPDATE hot_jsonb_test SET data = data - 'city' WHERE id = 2;
+SELECT * FROM get_hot_count('hot_jsonb_test');
+ updates | hot
+---------+-----
+ 3 | 0
+(1 row)
+
+-- Test jsonb_insert on non-indexed path - should be HOT after instrumentation
+UPDATE hot_jsonb_test SET data = jsonb_insert(data, '{country}', '"USA"') WHERE id = 2;
+SELECT * FROM get_hot_count('hot_jsonb_test');
+ updates | hot
+---------+-----
+ 4 | 0
+(1 row)
+
+DROP TABLE hot_jsonb_test;
+-- ============================================================================
+-- XML expression indexes and sub-attribute tracking
+-- ============================================================================
+-- Test that updates to non-indexed XML paths can be HOT updates
+CREATE TABLE hot_xml_test (
+ id int PRIMARY KEY,
+ doc xml
+) WITH (fillfactor = 50);
+-- Create expression index on a specific XPath
+CREATE INDEX hot_xml_name_idx ON hot_xml_test ((xpath('/person/name/text()', doc)));
+INSERT INTO hot_xml_test VALUES
+ (1, '<person><name>Alice</name><age>30</age></person>'),
+ (2, '<person><name>Bob</name><age>25</age></person>');
+ERROR: could not identify a comparison function for type xml
+SELECT * FROM get_hot_count('hot_xml_test');
+ updates | hot
+---------+-----
+ 0 | 0
+(1 row)
+
+-- Update non-indexed XPath (age) - behavior depends on XML comparison fallback
+-- Full XML value replacement means non-indexed path updates still require index comparison
+UPDATE hot_xml_test SET doc = '<person><name>Alice</name><age>31</age></person>' WHERE id = 1;
+SELECT * FROM get_hot_count('hot_xml_test');
+ updates | hot
+---------+-----
+ 0 | 0
+(1 row)
+
+-- Update indexed XPath (name) - should NOT be HOT
+UPDATE hot_xml_test SET doc = '<person><name>Alice2</name><age>31</age></person>' WHERE id = 1;
+SELECT * FROM get_hot_count('hot_xml_test');
+ updates | hot
+---------+-----
+ 0 | 0
+(1 row)
+
+-- Verify index works
+SELECT id FROM hot_xml_test WHERE xpath('/person/name/text()', doc) = ARRAY['Alice2'::text];
+ERROR: operator does not exist: xml[] = text[]
+LINE 1: ..._xml_test WHERE xpath('/person/name/text()', doc) = ARRAY['A...
+ ^
+DETAIL: No operator of that name accepts the given argument types.
+HINT: You might need to add explicit type casts.
+DROP TABLE hot_xml_test;
+-- ============================================================================
+-- GIN indexes and amcomparedatums for JSONB
+-- ============================================================================
+-- Test that GIN indexes can use amcomparedatums to enable HOT when extracted keys match
+CREATE TABLE hot_gin_test (
+ id int PRIMARY KEY,
+ tags text[],
+ properties jsonb
+) WITH (fillfactor = 50);
+-- GIN index on text array
+CREATE INDEX hot_gin_tags_idx ON hot_gin_test USING gin (tags);
+-- GIN index on JSONB (jsonb_ops - keys and values)
+CREATE INDEX hot_gin_props_idx ON hot_gin_test USING gin (properties);
+INSERT INTO hot_gin_test VALUES
+ (1, ARRAY['tag1', 'tag2'], '{"key1":"val1","key2":"val2"}'),
+ (2, ARRAY['tag3', 'tag4'], '{"key3":"val3","key4":"val4"}');
+SELECT * FROM get_hot_count('hot_gin_test');
+ updates | hot
+---------+-----
+ 0 | 0
+(1 row)
+
+-- Update that changes tag order but not content - after amcomparedatums should be HOT
+-- (GIN extracts same keys, just different order)
+UPDATE hot_gin_test SET tags = ARRAY['tag2', 'tag1'] WHERE id = 1;
+SELECT * FROM get_hot_count('hot_gin_test');
+ updates | hot
+---------+-----
+ 1 | 0
+(1 row)
+
+-- Update JSONB value (not key) - after amcomparedatums may be HOT or non-HOT
+-- depending on GIN operator class (jsonb_ops indexes both keys and values)
+UPDATE hot_gin_test SET properties = '{"key1":"val1_new","key2":"val2"}' WHERE id = 1;
+SELECT * FROM get_hot_count('hot_gin_test');
+ updates | hot
+---------+-----
+ 2 | 0
+(1 row)
+
+-- Add new tag - should NOT be HOT (different extracted keys)
+UPDATE hot_gin_test SET tags = ARRAY['tag2', 'tag1', 'tag5'] WHERE id = 1;
+SELECT * FROM get_hot_count('hot_gin_test');
+ updates | hot
+---------+-----
+ 3 | 0
+(1 row)
+
+-- Verify GIN indexes work
+SELECT id FROM hot_gin_test WHERE tags @> ARRAY['tag5'];
+ id
+----
+ 1
+(1 row)
+
+SELECT id FROM hot_gin_test WHERE properties @> '{"key1":"val1_new"}';
+ id
+----
+ 1
+(1 row)
+
+DROP TABLE hot_gin_test;
+-- ============================================================================
+-- Cleanup
+-- ============================================================================
+DROP TABLE IF EXISTS hot_test;
+DROP TABLE IF EXISTS hot_test_partitioned CASCADE;
+DROP FUNCTION IF EXISTS has_hot_chain(text, tid);
+DROP FUNCTION IF EXISTS print_hot_chain(text, tid);
+DROP FUNCTION IF EXISTS get_hot_count(text);
+DROP EXTENSION pageinspect;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 8fa0a6c47fb..bd95cc24977 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -143,6 +143,11 @@ test: event_trigger_login
# this test also uses event triggers, so likewise run it by itself
test: fast_default
+# ----------
+# HOT updates tests
+# ----------
+test: hot_updates
+
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
test: tablespace
diff --git a/src/test/regress/sql/hot_updates.sql b/src/test/regress/sql/hot_updates.sql
new file mode 100644
index 00000000000..a8894006177
--- /dev/null
+++ b/src/test/regress/sql/hot_updates.sql
@@ -0,0 +1,605 @@
+--
+-- HOT_UPDATES
+-- Test Heap-Only Tuple (HOT) update decisions
+--
+-- This test systematically verifies that HOT updates are used when appropriate
+-- and avoided when necessary (e.g., when indexed columns are modified).
+--
+-- We use multiple validation methods:
+-- 1. Statistics functions (pg_stat_get_tuples_hot_updated)
+-- 2. pageinspect extension for HOT chain examination
+-- 3. EXPLAIN to verify index usage after updates
+--
+
+-- Load required extensions
+CREATE EXTENSION IF NOT EXISTS pageinspect;
+
+-- Function to get HOT update count
+CREATE OR REPLACE FUNCTION get_hot_count(rel_name text)
+RETURNS TABLE (
+ updates BIGINT,
+ hot BIGINT
+) AS $$
+DECLARE
+ rel_oid oid;
+BEGIN
+ rel_oid := rel_name::regclass::oid;
+
+ -- Read both committed and transaction-local stats
+ -- In autocommit mode (default for regression tests), this works correctly
+ -- Note: In explicit transactions (BEGIN/COMMIT), committed stats already
+ -- include flushed updates, so this would double-count. For explicit
+ -- transaction testing, call pg_stat_force_next_flush() before this function.
+ updates := COALESCE(pg_stat_get_tuples_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_updated(rel_oid), 0);
+ hot := COALESCE(pg_stat_get_tuples_hot_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_hot_updated(rel_oid), 0);
+
+ RETURN NEXT;
+END;
+$$ LANGUAGE plpgsql;
+
+-- Check if a tuple is part of a HOT chain (has a predecessor on same page)
+CREATE OR REPLACE FUNCTION has_hot_chain(rel_name text, target_ctid tid)
+RETURNS boolean AS $$
+DECLARE
+ block_num int;
+ page_item record;
+BEGIN
+ block_num := (target_ctid::text::point)[0]::int;
+
+ -- Look for a different tuple on the same page that points to our target tuple
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp_flags = 1
+ AND t_ctid IS NOT NULL
+ AND t_ctid = target_ctid
+ AND ('(' || block_num::text || ',' || lp::text || ')')::tid != target_ctid
+ LOOP
+ RETURN true;
+ END LOOP;
+
+ RETURN false;
+END;
+$$ LANGUAGE plpgsql;
+
+-- Print the HOT chain starting from a given tuple
+CREATE OR REPLACE FUNCTION print_hot_chain(rel_name text, start_ctid tid)
+RETURNS TABLE(chain_position int, ctid tid, lp_flags text, t_ctid tid, chain_end boolean) AS
+$$
+#variable_conflict use_column
+DECLARE
+ block_num int;
+ line_ptr int;
+ current_ctid tid := start_ctid;
+ next_ctid tid;
+ position int := 0;
+ max_iterations int := 100;
+ page_item record;
+ found_predecessor boolean := false;
+ flags_name text;
+BEGIN
+ block_num := (start_ctid::text::point)[0]::int;
+
+ -- Find the predecessor (old tuple pointing to our start_ctid)
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp_flags = 1
+ AND t_ctid = start_ctid
+ LOOP
+ current_ctid := ('(' || block_num::text || ',' || page_item.lp::text || ')')::tid;
+ found_predecessor := true;
+ EXIT;
+ END LOOP;
+
+ -- If no predecessor found, start with the given ctid
+ IF NOT found_predecessor THEN
+ current_ctid := start_ctid;
+ END IF;
+
+ -- Follow the chain forward
+ WHILE position < max_iterations LOOP
+ line_ptr := (current_ctid::text::point)[1]::int;
+
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp = line_ptr
+ LOOP
+ -- Map lp_flags to names
+ flags_name := CASE page_item.lp_flags
+ WHEN 0 THEN 'unused (0)'
+ WHEN 1 THEN 'normal (1)'
+ WHEN 2 THEN 'redirect (2)'
+ WHEN 3 THEN 'dead (3)'
+ ELSE 'unknown (' || page_item.lp_flags::text || ')'
+ END;
+
+ RETURN QUERY SELECT
+ position,
+ current_ctid,
+ flags_name,
+ page_item.t_ctid,
+ (page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid)::boolean
+ ;
+
+ IF page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid THEN
+ RETURN;
+ END IF;
+
+ next_ctid := page_item.t_ctid;
+
+ IF (next_ctid::text::point)[0]::int != block_num THEN
+ RETURN;
+ END IF;
+
+ current_ctid := next_ctid;
+ position := position + 1;
+ END LOOP;
+
+ IF position = 0 THEN
+ RETURN;
+ END IF;
+ END LOOP;
+END;
+$$ LANGUAGE plpgsql;
+
+-- Basic HOT update (update non-indexed column)
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ non_indexed_col text
+) WITH (fillfactor = 50);
+
+CREATE INDEX hot_test_indexed_idx ON hot_test(indexed_col);
+
+INSERT INTO hot_test VALUES (1, 100, 'initial');
+INSERT INTO hot_test VALUES (2, 200, 'initial');
+INSERT INTO hot_test VALUES (3, 300, 'initial');
+
+-- Get baseline
+SELECT * FROM get_hot_count('hot_test');
+
+-- Should be HOT updates (only non-indexed column modified)
+UPDATE hot_test SET non_indexed_col = 'updated1' WHERE id = 1;
+UPDATE hot_test SET non_indexed_col = 'updated2' WHERE id = 2;
+UPDATE hot_test SET non_indexed_col = 'updated3' WHERE id = 3;
+
+-- Verify HOT updates occurred
+SELECT * FROM get_hot_count('hot_test');
+
+-- Dump the HOT chain before VACUUMing
+WITH current_tuple AS (
+ SELECT ctid FROM hot_test WHERE id = 1
+)
+SELECT
+ has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position,
+ print_hot_chain.ctid,
+ lp_flags,
+ t_ctid
+FROM current_tuple,
+LATERAL print_hot_chain('hot_test', current_tuple.ctid);
+
+-- Vacuum the relation, expect the HOT chain to collapse
+VACUUM hot_test;
+
+-- Show that there is no chain after vacuum
+WITH current_tuple AS (
+ SELECT ctid FROM hot_test WHERE id = 1
+)
+SELECT
+ has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position,
+ print_hot_chain.ctid,
+ lp_flags,
+ t_ctid
+FROM current_tuple,
+LATERAL print_hot_chain('hot_test', current_tuple.ctid);
+
+-- Non-HOT update (update indexed column)
+UPDATE hot_test SET indexed_col = 150 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Verify index was updated (new value findable)
+SET enable_seqscan = off;
+EXPLAIN (COSTS OFF) SELECT id, indexed_col FROM hot_test WHERE indexed_col = 150;
+SELECT id, indexed_col FROM hot_test WHERE indexed_col = 150;
+
+-- Verify old value no longer in index
+EXPLAIN (COSTS OFF) SELECT id FROM hot_test WHERE indexed_col = 100;
+SELECT id FROM hot_test WHERE indexed_col = 100;
+RESET enable_seqscan;
+
+-- All-or-none property: updating one indexed column requires ALL index updates
+DROP TABLE hot_test;
+
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ col_a int,
+ col_b int,
+ col_c int,
+ non_indexed text
+) WITH (fillfactor = 50);
+
+CREATE INDEX hot_test_a_idx ON hot_test(col_a);
+CREATE INDEX hot_test_b_idx ON hot_test(col_b);
+CREATE INDEX hot_test_c_idx ON hot_test(col_c);
+
+INSERT INTO hot_test VALUES (1, 10, 20, 30, 'initial');
+
+-- Update only col_a - should NOT be HOT because an indexed column changed
+-- This means ALL indexes must be updated (all-or-none property)
+UPDATE hot_test SET col_a = 15 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Now update only non-indexed column - should be HOT
+UPDATE hot_test SET non_indexed = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+
+-- Partial index: both old and new outside predicate (conservative = non-HOT)
+DROP TABLE hot_test;
+
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ status text,
+ data text
+) WITH (fillfactor = 50);
+
+-- Partial index only covers status = 'active'
+CREATE INDEX hot_test_active_idx ON hot_test(status) WHERE status = 'active';
+
+INSERT INTO hot_test VALUES (1, 'active', 'data1');
+INSERT INTO hot_test VALUES (2, 'inactive', 'data2');
+INSERT INTO hot_test VALUES (3, 'deleted', 'data3');
+
+-- Update non-indexed column on 'active' row (in predicate, status unchanged)
+-- Should be HOT
+UPDATE hot_test SET data = 'updated1' WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Update non-indexed column on 'inactive' row (outside predicate)
+-- Should be HOT
+UPDATE hot_test SET data = 'updated2' WHERE id = 2;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Update status from 'inactive' to 'deleted' (both outside predicate)
+-- PostgreSQL is conservative: heap insert happens before predicate check
+-- So this is NON-HOT even though both values are outside predicate
+UPDATE hot_test SET status = 'deleted' WHERE id = 2;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Verify index still works for 'active' rows
+SELECT id, status FROM hot_test WHERE status = 'active';
+
+-- Only BRIN (summarizing) indexes on non-PK columns
+DROP TABLE hot_test;
+
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ ts timestamp,
+ value int,
+ brin_col int
+) WITH (fillfactor = 50);
+
+CREATE INDEX hot_test_ts_brin ON hot_test USING brin(ts);
+CREATE INDEX hot_test_brin_col_brin ON hot_test USING brin(brin_col);
+
+INSERT INTO hot_test VALUES (1, '2024-01-01', 100, 1000);
+
+-- Update both BRIN columns - should still be HOT (only summarizing indexes)
+UPDATE hot_test SET ts = '2024-01-02', brin_col = 2000 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Update non-indexed column - should also be HOT
+UPDATE hot_test SET value = 200 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test');
+
+-- TOAST and HOT: TOASTed columns can participate in HOT
+DROP TABLE hot_test;
+
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ indexed_col int,
+ large_text text,
+ small_text text
+) WITH (fillfactor = 50);
+
+CREATE INDEX hot_test_idx ON hot_test(indexed_col);
+
+-- Insert row with TOASTed column (> 2KB)
+INSERT INTO hot_test VALUES (1, 100, repeat('x', 3000), 'small');
+
+-- Update non-indexed, non-TOASTed column - should be HOT
+UPDATE hot_test SET small_text = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+
+-- Update TOASTed column - should be HOT if indexed column unchanged
+UPDATE hot_test SET large_text = repeat('y', 3000);
+SELECT * FROM get_hot_count('hot_test');
+
+-- Update indexed column - should NOT be HOT
+UPDATE hot_test SET indexed_col = 200;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Unique constraint (unique index) behaves like regular index
+DROP TABLE hot_test;
+
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ unique_col int UNIQUE,
+ data text
+) WITH (fillfactor = 50);
+
+INSERT INTO hot_test VALUES (1, 100, 'data1');
+INSERT INTO hot_test VALUES (2, 200, 'data2');
+
+-- Update data (non-indexed) - should be HOT
+UPDATE hot_test SET data = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+
+-- Verify unique constraint still enforced
+SELECT id, unique_col, data FROM hot_test ORDER BY id;
+
+-- This should fail (unique violation)
+UPDATE hot_test SET unique_col = 100 WHERE id = 2;
+
+-- Multi-column index: any column change = non-HOT
+DROP TABLE hot_test;
+
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ col_a int,
+ col_b int,
+ col_c int,
+ data text
+) WITH (fillfactor = 50);
+
+CREATE INDEX hot_test_ab_idx ON hot_test(col_a, col_b);
+
+INSERT INTO hot_test VALUES (1, 10, 20, 30, 'data');
+
+-- Update col_a (part of multi-column index) - should NOT be HOT
+UPDATE hot_test SET col_a = 15;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Reset
+UPDATE hot_test SET col_a = 10;
+
+-- Update col_b (part of multi-column index) - should NOT be HOT
+UPDATE hot_test SET col_b = 25;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Reset
+UPDATE hot_test SET col_b = 20;
+SELECT * FROM get_hot_count('hot_test');
+
+-- Update col_c (not indexed) - should be HOT
+UPDATE hot_test SET col_c = 35;
+
+-- Update data (not indexed) - should be HOT
+UPDATE hot_test SET data = 'updated';
+SELECT * FROM get_hot_count('hot_test');
+
+-- Partitioned tables: HOT works within partitions
+DROP TABLE IF EXISTS hot_test_partitioned CASCADE;
+
+CREATE TABLE hot_test_partitioned (
+ id int,
+ partition_key int,
+ indexed_col int,
+ data text,
+ PRIMARY KEY (id, partition_key)
+) PARTITION BY RANGE (partition_key);
+
+CREATE TABLE hot_test_part1 PARTITION OF hot_test_partitioned
+ FOR VALUES FROM (1) TO (100) WITH (fillfactor = 50);
+CREATE TABLE hot_test_part2 PARTITION OF hot_test_partitioned
+ FOR VALUES FROM (100) TO (200) WITH (fillfactor = 50);
+
+CREATE INDEX hot_test_part_idx ON hot_test_partitioned(indexed_col);
+
+INSERT INTO hot_test_partitioned VALUES (1, 50, 100, 'initial1');
+INSERT INTO hot_test_partitioned VALUES (2, 150, 200, 'initial2');
+
+-- Update in partition 1 (non-indexed column) - should be HOT
+UPDATE hot_test_partitioned SET data = 'updated1' WHERE id = 1;
+
+-- Update in partition 2 (non-indexed column) - should be HOT
+UPDATE hot_test_partitioned SET data = 'updated2' WHERE id = 2;
+
+SELECT * FROM get_hot_count('hot_test_part1');
+SELECT * FROM get_hot_count('hot_test_part2');
+
+-- Verify indexes work on partitions
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 100;
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 200;
+
+-- Update indexed column in partition - should NOT be HOT
+UPDATE hot_test_partitioned SET indexed_col = 150 WHERE id = 1;
+SELECT * FROM get_hot_count('hot_test_part1');
+
+-- Verify index was updated
+SELECT id FROM hot_test_partitioned WHERE indexed_col = 150;
+
+-- ============================================================================
+-- Trigger modifications: heap_modify_tuple() and HOT
+-- ============================================================================
+-- Test that we correctly detect when triggers modify indexed columns via
+-- heap_modify_tuple(), even when those columns aren't in the UPDATE's SET clause
+
+CREATE TABLE hot_trigger_test (
+ id int PRIMARY KEY,
+ triggered_col int,
+ data text
+) WITH (fillfactor = 50);
+
+CREATE INDEX hot_trigger_idx ON hot_trigger_test(triggered_col);
+
+-- Create a trigger that modifies an indexed column
+CREATE OR REPLACE FUNCTION modify_triggered_col()
+RETURNS TRIGGER AS $$
+BEGIN
+ NEW.triggered_col = NEW.triggered_col + 1;
+ RETURN NEW;
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE TRIGGER before_update_modify
+ BEFORE UPDATE ON hot_trigger_test
+ FOR EACH ROW
+ EXECUTE FUNCTION modify_triggered_col();
+
+INSERT INTO hot_trigger_test VALUES (1, 100, 'initial');
+
+SELECT * FROM get_hot_count('hot_trigger_test');
+
+-- Update only data column, but trigger modifies indexed column
+-- Should NOT be HOT because trigger modified an indexed column
+UPDATE hot_trigger_test SET data = 'updated' WHERE id = 1;
+
+-- Verify it was NOT a HOT update (indexed column was modified by trigger)
+SELECT * FROM get_hot_count('hot_trigger_test');
+
+-- Verify the triggered column was actually modified
+SELECT triggered_col FROM hot_trigger_test WHERE id = 1;
+
+DROP TABLE hot_trigger_test CASCADE;
+DROP FUNCTION modify_triggered_col();
+
+-- ============================================================================
+-- JSONB expression indexes and sub-attribute tracking
+-- ============================================================================
+-- Test that updates to non-indexed JSONB paths can be HOT updates
+
+CREATE TABLE hot_jsonb_test (
+ id int PRIMARY KEY,
+ data jsonb
+) WITH (fillfactor = 50);
+
+-- Create expression index on a specific JSON path
+CREATE INDEX hot_jsonb_name_idx ON hot_jsonb_test ((data->>'name'));
+
+INSERT INTO hot_jsonb_test VALUES
+ (1, '{"name":"Alice","age":30,"city":"NYC"}'),
+ (2, '{"name":"Bob","age":25,"city":"LA"}');
+
+SELECT * FROM get_hot_count('hot_jsonb_test');
+
+-- Update non-indexed JSON path (age) - should be HOT after instrumentation
+UPDATE hot_jsonb_test SET data = jsonb_set(data, '{age}', '31') WHERE id = 1;
+
+SELECT * FROM get_hot_count('hot_jsonb_test');
+
+-- Update indexed JSON path (name) - should NOT be HOT
+UPDATE hot_jsonb_test SET data = jsonb_set(data, '{name}', '"Alice2"') WHERE id = 1;
+
+SELECT * FROM get_hot_count('hot_jsonb_test');
+
+-- Verify index works
+SELECT id FROM hot_jsonb_test WHERE data->>'name' = 'Alice2';
+
+-- Test jsonb_delete on non-indexed path - should be HOT after instrumentation
+UPDATE hot_jsonb_test SET data = data - 'city' WHERE id = 2;
+
+SELECT * FROM get_hot_count('hot_jsonb_test');
+
+-- Test jsonb_insert on non-indexed path - should be HOT after instrumentation
+UPDATE hot_jsonb_test SET data = jsonb_insert(data, '{country}', '"USA"') WHERE id = 2;
+
+SELECT * FROM get_hot_count('hot_jsonb_test');
+
+DROP TABLE hot_jsonb_test;
+
+-- ============================================================================
+-- XML expression indexes and sub-attribute tracking
+-- ============================================================================
+-- Test that updates to non-indexed XML paths can be HOT updates
+
+CREATE TABLE hot_xml_test (
+ id int PRIMARY KEY,
+ doc xml
+) WITH (fillfactor = 50);
+
+-- Create expression index on a specific XPath
+CREATE INDEX hot_xml_name_idx ON hot_xml_test ((xpath('/person/name/text()', doc)));
+
+INSERT INTO hot_xml_test VALUES
+ (1, '<person><name>Alice</name><age>30</age></person>'),
+ (2, '<person><name>Bob</name><age>25</age></person>');
+
+SELECT * FROM get_hot_count('hot_xml_test');
+
+-- Update non-indexed XPath (age) - behavior depends on XML comparison fallback
+-- Full XML value replacement means non-indexed path updates still require index comparison
+UPDATE hot_xml_test SET doc = '<person><name>Alice</name><age>31</age></person>' WHERE id = 1;
+
+SELECT * FROM get_hot_count('hot_xml_test');
+
+-- Update indexed XPath (name) - should NOT be HOT
+UPDATE hot_xml_test SET doc = '<person><name>Alice2</name><age>31</age></person>' WHERE id = 1;
+
+SELECT * FROM get_hot_count('hot_xml_test');
+
+-- Verify index works
+SELECT id FROM hot_xml_test WHERE xpath('/person/name/text()', doc) = ARRAY['Alice2'::text];
+
+DROP TABLE hot_xml_test;
+
+-- ============================================================================
+-- GIN indexes and amcomparedatums for JSONB
+-- ============================================================================
+-- Test that GIN indexes can use amcomparedatums to enable HOT when extracted keys match
+
+CREATE TABLE hot_gin_test (
+ id int PRIMARY KEY,
+ tags text[],
+ properties jsonb
+) WITH (fillfactor = 50);
+
+-- GIN index on text array
+CREATE INDEX hot_gin_tags_idx ON hot_gin_test USING gin (tags);
+
+-- GIN index on JSONB (jsonb_ops - keys and values)
+CREATE INDEX hot_gin_props_idx ON hot_gin_test USING gin (properties);
+
+INSERT INTO hot_gin_test VALUES
+ (1, ARRAY['tag1', 'tag2'], '{"key1":"val1","key2":"val2"}'),
+ (2, ARRAY['tag3', 'tag4'], '{"key3":"val3","key4":"val4"}');
+
+SELECT * FROM get_hot_count('hot_gin_test');
+
+-- Update that changes tag order but not content - after amcomparedatums should be HOT
+-- (GIN extracts same keys, just different order)
+UPDATE hot_gin_test SET tags = ARRAY['tag2', 'tag1'] WHERE id = 1;
+
+SELECT * FROM get_hot_count('hot_gin_test');
+
+-- Update JSONB value (not key) - after amcomparedatums may be HOT or non-HOT
+-- depending on GIN operator class (jsonb_ops indexes both keys and values)
+UPDATE hot_gin_test SET properties = '{"key1":"val1_new","key2":"val2"}' WHERE id = 1;
+
+SELECT * FROM get_hot_count('hot_gin_test');
+
+-- Add new tag - should NOT be HOT (different extracted keys)
+UPDATE hot_gin_test SET tags = ARRAY['tag2', 'tag1', 'tag5'] WHERE id = 1;
+
+SELECT * FROM get_hot_count('hot_gin_test');
+
+-- Verify GIN indexes work
+SELECT id FROM hot_gin_test WHERE tags @> ARRAY['tag5'];
+SELECT id FROM hot_gin_test WHERE properties @> '{"key1":"val1_new"}';
+
+DROP TABLE hot_gin_test;
+
+-- ============================================================================
+-- Cleanup
+-- ============================================================================
+DROP TABLE IF EXISTS hot_test;
+DROP TABLE IF EXISTS hot_test_partitioned CASCADE;
+DROP FUNCTION IF EXISTS has_hot_chain(text, tid);
+DROP FUNCTION IF EXISTS print_hot_chain(text, tid);
+DROP FUNCTION IF EXISTS get_hot_count(text);
+DROP EXTENSION pageinspect;
--
2.50.1
[text/x-patch] v57-0002-Identify-modified-indexed-attributes-in-the-exec.patch (61.4K, ../[email protected]/3-v57-0002-Identify-modified-indexed-attributes-in-the-exec.patch)
download | inline diff:
From 23460c9db7d4c84d9bd11f1e950c6bdd33f52489 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Tue, 10 Mar 2026 08:17:31 -0400
Subject: [PATCH v57 2/9] Identify modified indexed attributes in the executor
on UPDATE
Refactor executor update logic to determine which indexed columns have
actually changed during an UPDATE operation rather than leaving this up
to HeapDetermineColumnsInfo() in heap_update(). Finding this set of
attributes is not heap-specific, but more general to all table AMs and
having this information in the executor could inform other decisions
about when index inserts are required and when they are not regardless
of the table AM's MVCC implementation strategy.
The heap-only tuple decision (HOT) in heap functions as it always has;
what moves to the executor is only the determination of the "modified
indexed attributes" (modified_idx_attrs).
ExecUpdateModifiedIdxAttrs() replaces HeapDetermineColumnsInfo() and is
called before table_tuple_update() crucially without the need for an
exclusive buffer lock on the page that holds the tuple being updated.
This reduces the time the buffer lock is held later within
heapam_tuple_update() and heap_update().
Besides identifying the set of modified indexed attributes
HeapDetermineColumnsInfo() was also partially responsible for the
decision about what to WAL log for the replica identity key. That logic
moves into heap_update() and into the replacement helper
HeapUpdateModifiedIdxAttrs(), so simple_heap_update() and
heapam_tuple_update() share the same logic since both call into
heap_update().
Updates stemming from logical replication also use the new
ExecUpdateModifiedIdxAttrs() in ExecSimpleRelationUpdate().
ExecUpdateModifiedIdxAttrs() uses ExecCompareSlotAttrs() to identify
which attributes have changed and then intersects that with the set of
indexed attributes to identify the modified indexed set, the
modified_idx_attrs.
This patch introduces a few helper functions to reduce code duplication
and increase readability: HeapUpdateHotAllowable() and
HeapUpdateDetermineLockmode(), used in both heap_update() and
simple_heap_update().
heap_update() is now called with lockmode pre-determined and a boolean
indicating whether the update may be HOT, both const. If during
heap_update() the new tuple fits on the same page and that boolean is
true, the update is HOT. So although the functions and timing of the
HOT decision code have changed, none of the logic governing when HOT is
allowed has changed.
Development of this feature exposed nondeterministic behavior in three
existing tests, which have been adjusted to avoid inconsistent results
due to tuple ordering during heap page scans.
Authored-by: Greg Burd <[email protected]>
Discussion: https://commitfest.postgresql.org/patch/5556/
Discussion: https://www.postgresql.org/message-id/flat/78574B24-BE0A-42C5-8075-3FA9FA63B8FC%40amazon.com
---
src/backend/access/heap/heapam.c | 451 ++++++++++++------
src/backend/access/heap/heapam_handler.c | 34 +-
src/backend/access/table/tableam.c | 5 +-
src/backend/commands/repack.c | 14 +-
src/backend/executor/execReplication.c | 9 +-
src/backend/executor/execTuples.c | 72 +++
src/backend/executor/nodeModifyTable.c | 86 +++-
src/backend/utils/cache/relcache.c | 44 +-
src/include/access/heapam.h | 13 +-
src/include/access/tableam.h | 8 +-
src/include/executor/executor.h | 8 +
src/include/utils/rel.h | 2 +-
src/include/utils/relcache.h | 2 +-
.../expected/syscache-update-pruned.out | 12 +-
.../specs/syscache-update-pruned.spec | 6 +-
.../regress/expected/generated_virtual.out | 2 +-
src/test/regress/expected/triggers.out | 16 +-
src/test/regress/expected/tsearch.out | 3 +-
src/test/regress/expected/updatable_views.out | 4 +-
src/test/regress/sql/generated_virtual.sql | 2 +-
src/test/regress/sql/triggers.sql | 4 +-
src/test/regress/sql/tsearch.sql | 3 +-
src/test/regress/sql/updatable_views.sql | 2 +-
23 files changed, 576 insertions(+), 226 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index abfd8e8970a..5b059a5acef 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -37,6 +37,8 @@
#include "access/multixact.h"
#include "access/subtrans.h"
#include "access/syncscan.h"
+#include "access/sysattr.h"
+#include "access/tableam.h"
#include "access/valid.h"
#include "access/visibilitymap.h"
#include "access/xloginsert.h"
@@ -44,8 +46,11 @@
#include "catalog/pg_database_d.h"
#include "commands/vacuum.h"
#include "executor/instrument_node.h"
+#include "executor/tuptable.h"
+#include "nodes/lockoptions.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
+#include "storage/buf.h"
#include "storage/lmgr.h"
#include "storage/predicate.h"
#include "storage/proc.h"
@@ -53,6 +58,7 @@
#include "utils/datum.h"
#include "utils/injection_point.h"
#include "utils/inval.h"
+#include "utils/relcache.h"
#include "utils/spccache.h"
#include "utils/syscache.h"
@@ -70,11 +76,8 @@ static void check_lock_if_inplace_updateable_rel(Relation relation,
HeapTuple newtup);
static void check_inplace_rel_lock(HeapTuple oldtup);
#endif
-static Bitmapset *HeapDetermineColumnsInfo(Relation relation,
- Bitmapset *interesting_cols,
- Bitmapset *external_cols,
- HeapTuple oldtup, HeapTuple newtup,
- bool *has_external);
+static Bitmapset *HeapUpdateModifiedIdxAttrs(Relation relation,
+ HeapTuple oldtup, HeapTuple newtup);
static bool heap_acquire_tuplock(Relation relation, const ItemPointerData *tid,
LockTupleMode mode, LockWaitPolicy wait_policy,
bool *have_tuple_lock);
@@ -3190,7 +3193,7 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
* heap_update - replace a tuple
*
* See table_tuple_update() for an explanation of the parameters, except that
- * this routine directly takes a tuple rather than a slot.
+ * this routine directly takes a heap tuple rather than a slot.
*
* In the failure cases, the routine fills *tmfd with the tuple's t_ctid,
* t_xmax (resolving a possible MultiXact, if necessary), and t_cmax (the last
@@ -3200,17 +3203,13 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
TM_Result
heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
CommandId cid, uint32 options pg_attribute_unused(), Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+ TM_FailureData *tmfd, const LockTupleMode lockmode,
+ const Bitmapset *modified_idx_attrs, const bool hot_allowed)
{
TM_Result result;
TransactionId xid = GetCurrentTransactionId();
- Bitmapset *hot_attrs;
- Bitmapset *sum_attrs;
- Bitmapset *key_attrs;
- Bitmapset *id_attrs;
- Bitmapset *interesting_attrs;
- Bitmapset *modified_attrs;
+ Bitmapset *idx_attrs,
+ *id_attrs;
ItemId lp;
HeapTupleData oldtup;
HeapTuple heaptup;
@@ -3231,13 +3230,12 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
bool have_tuple_lock = false;
bool iscombo;
bool use_hot_update = false;
- bool summarized_update = false;
bool key_intact;
bool all_visible_cleared = false;
bool all_visible_cleared_new = false;
bool checked_lockers;
bool locker_remains;
- bool id_has_external = false;
+ bool rep_id_key_required = false;
TransactionId xmax_new_tuple,
xmax_old_tuple;
uint16 infomask_old_tuple,
@@ -3268,33 +3266,18 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
#endif
/*
- * Fetch the list of attributes to be checked for various operations.
- *
- * For HOT considerations, this is wasted effort if we fail to update or
- * have to put the new tuple on a different page. But we must compute the
- * list before obtaining buffer lock --- in the worst case, if we are
- * doing an update on one of the relevant system catalogs, we could
- * deadlock if we try to fetch the list later. In any case, the relcache
- * caches the data so this is usually pretty cheap.
- *
- * We also need columns used by the replica identity and columns that are
- * considered the "key" of rows in the table.
+ * Fetch the attributes used across all indexes on this relation as well
+ * as the replica identity and columns.
*
- * Note that we get copies of each bitmap, so we need not worry about
- * relcache flush happening midway through.
- */
- hot_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_HOT_BLOCKING);
- sum_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_SUMMARIZED);
- key_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_KEY);
- id_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_IDENTITY_KEY);
- interesting_attrs = NULL;
- interesting_attrs = bms_add_members(interesting_attrs, hot_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, sum_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, key_attrs);
- interesting_attrs = bms_add_members(interesting_attrs, id_attrs);
+ * Note: We must compute the list before obtaining buffer lock. In the
+ * worst case, if we are doing an update on one of the relevant system
+ * catalogs, we could deadlock if we try to fetch the list later. Keep in
+ * mind that relcache returns copies of each bitmap, so we need not worry
+ * about relcache flush happening midway through, but we do need to free
+ * them.
+ */
+ idx_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_INDEXED);
+ id_attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_IDENTITY_KEY);
block = ItemPointerGetBlockNumber(otid);
INJECTION_POINT("heap_update-before-pin", NULL);
@@ -3348,20 +3331,17 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
tmfd->ctid = *otid;
tmfd->xmax = InvalidTransactionId;
tmfd->cmax = InvalidCommandId;
- *update_indexes = TU_None;
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
bms_free(id_attrs);
- /* modified_attrs not yet initialized */
- bms_free(interesting_attrs);
+ bms_free(idx_attrs);
+ /* modified_idx_attrs is owned by the caller, don't free it */
+
return TM_Deleted;
}
/*
- * Fill in enough data in oldtup for HeapDetermineColumnsInfo to work
- * properly.
+ * Fill in enough data in oldtup to determine replica identity attribute
+ * requirements.
*/
oldtup.t_tableOid = RelationGetRelid(relation);
oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
@@ -3372,16 +3352,59 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
newtup->t_tableOid = RelationGetRelid(relation);
/*
- * Determine columns modified by the update. Additionally, identify
- * whether any of the unmodified replica identity key attributes in the
- * old tuple is externally stored or not. This is required because for
- * such attributes the flattened value won't be WAL logged as part of the
- * new tuple so we must include it as part of the old_key_tuple. See
- * ExtractReplicaIdentity.
+ * ExtractReplicaIdentity() needs to know if a modified indexed attribute
+ * is used as a replica identity or if any of the replica identity
+ * attributes are referenced in an index, unmodified, and are stored
+ * externally in the old tuple being replaced. In those cases it may be
+ * necessary to WAL log them so they are available to replicas.
*/
- modified_attrs = HeapDetermineColumnsInfo(relation, interesting_attrs,
- id_attrs, &oldtup,
- newtup, &id_has_external);
+ rep_id_key_required = bms_overlap(modified_idx_attrs, id_attrs);
+ if (!rep_id_key_required)
+ {
+ Bitmapset *attrs;
+ TupleDesc tupdesc = RelationGetDescr(relation);
+ int attidx = -1;
+
+ /*
+ * Reduce the set under review to only the unmodified indexed replica
+ * identity key attributes. idx_attrs is copied (by bms_difference())
+ * not modified here.
+ */
+ attrs = bms_difference(idx_attrs, modified_idx_attrs);
+ attrs = bms_int_members(attrs, id_attrs);
+
+ while ((attidx = bms_next_member(attrs, attidx)) >= 0)
+ {
+ /*
+ * attidx is zero-based, attrnum is the normal attribute number
+ */
+ AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
+ Datum value;
+ bool isnull;
+
+ /*
+ * System attributes are not added into INDEX_ATTR_BITMAP_INDEXED
+ * bitmap by relcache.
+ */
+ Assert(attrnum > 0);
+
+ value = heap_getattr(&oldtup, attrnum, tupdesc, &isnull);
+
+ /* No need to check attributes that can't be stored externally */
+ if (isnull ||
+ TupleDescCompactAttr(tupdesc, attrnum - 1)->attlen != -1)
+ continue;
+
+ /* Check if the old tuple's attribute is stored externally */
+ if (VARATT_IS_EXTERNAL((struct varlena *) DatumGetPointer(value)))
+ {
+ rep_id_key_required = true;
+ break;
+ }
+ }
+
+ bms_free(attrs);
+ }
/*
* If we're not updating any "key" column, we can grab a weaker lock type.
@@ -3394,9 +3417,8 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
* is updates that don't manipulate key columns, not those that
* serendipitously arrive at the same key values.
*/
- if (!bms_overlap(modified_attrs, key_attrs))
+ if (lockmode == LockTupleNoKeyExclusive)
{
- *lockmode = LockTupleNoKeyExclusive;
mxact_status = MultiXactStatusNoKeyUpdate;
key_intact = true;
@@ -3413,7 +3435,7 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
}
else
{
- *lockmode = LockTupleExclusive;
+ Assert(lockmode == LockTupleExclusive);
mxact_status = MultiXactStatusUpdate;
key_intact = false;
}
@@ -3492,7 +3514,7 @@ l2:
bool current_is_member = false;
if (DoesMultiXactIdConflict((MultiXactId) xwait, infomask,
- *lockmode, ¤t_is_member))
+ lockmode, ¤t_is_member))
{
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
@@ -3501,7 +3523,7 @@ l2:
* requesting a lock and already have one; avoids deadlock).
*/
if (!current_is_member)
- heap_acquire_tuplock(relation, &(oldtup.t_self), *lockmode,
+ heap_acquire_tuplock(relation, &(oldtup.t_self), lockmode,
LockWaitBlock, &have_tuple_lock);
/* wait for multixact */
@@ -3586,7 +3608,7 @@ l2:
* lock.
*/
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
- heap_acquire_tuplock(relation, &(oldtup.t_self), *lockmode,
+ heap_acquire_tuplock(relation, &(oldtup.t_self), lockmode,
LockWaitBlock, &have_tuple_lock);
XactLockTableWait(xwait, relation, &oldtup.t_self,
XLTW_Update);
@@ -3646,17 +3668,14 @@ l2:
tmfd->cmax = InvalidCommandId;
UnlockReleaseBuffer(buffer);
if (have_tuple_lock)
- UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
+ UnlockTupleTuplock(relation, &(oldtup.t_self), lockmode);
if (vmbuffer != InvalidBuffer)
ReleaseBuffer(vmbuffer);
- *update_indexes = TU_None;
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
bms_free(id_attrs);
- bms_free(modified_attrs);
- bms_free(interesting_attrs);
+ bms_free(idx_attrs);
+ /* modified_idx_attrs is owned by the caller, don't free it */
+
return result;
}
@@ -3686,7 +3705,7 @@ l2:
compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup.t_data),
oldtup.t_data->t_infomask,
oldtup.t_data->t_infomask2,
- xid, *lockmode, true,
+ xid, lockmode, true,
&xmax_old_tuple, &infomask_old_tuple,
&infomask2_old_tuple);
@@ -3803,7 +3822,7 @@ l2:
compute_new_xmax_infomask(HeapTupleHeaderGetRawXmax(oldtup.t_data),
oldtup.t_data->t_infomask,
oldtup.t_data->t_infomask2,
- xid, *lockmode, false,
+ xid, lockmode, false,
&xmax_lock_old_tuple, &infomask_lock_old_tuple,
&infomask2_lock_old_tuple);
@@ -3976,20 +3995,8 @@ l2:
* to do a HOT update. Check if any of the index columns have been
* changed.
*/
- if (!bms_overlap(modified_attrs, hot_attrs))
- {
+ if (hot_allowed)
use_hot_update = true;
-
- /*
- * If none of the columns that are used in hot-blocking indexes
- * were updated, we can apply HOT, but we do still need to check
- * if we need to update the summarizing indexes, and update those
- * indexes if the columns were updated, or we may fail to detect
- * e.g. value bound changes in BRIN minmax indexes.
- */
- if (bms_overlap(modified_attrs, sum_attrs))
- summarized_update = true;
- }
}
else
{
@@ -4005,8 +4012,7 @@ l2:
* columns are modified or it has external data.
*/
old_key_tuple = ExtractReplicaIdentity(relation, &oldtup,
- bms_overlap(modified_attrs, id_attrs) ||
- id_has_external,
+ rep_id_key_required,
&old_key_copied);
/* NO EREPORT(ERROR) from here till changes are logged */
@@ -4136,7 +4142,7 @@ l2:
* Release the lmgr tuple lock, if we had it.
*/
if (have_tuple_lock)
- UnlockTupleTuplock(relation, &(oldtup.t_self), *lockmode);
+ UnlockTupleTuplock(relation, &(oldtup.t_self), lockmode);
pgstat_count_heap_update(relation, use_hot_update, newbuf != buffer);
@@ -4150,31 +4156,12 @@ l2:
heap_freetuple(heaptup);
}
- /*
- * If it is a HOT update, the update may still need to update summarized
- * indexes, lest we fail to update those summaries and get incorrect
- * results (for example, minmax bounds of the block may change with this
- * update).
- */
- if (use_hot_update)
- {
- if (summarized_update)
- *update_indexes = TU_Summarizing;
- else
- *update_indexes = TU_None;
- }
- else
- *update_indexes = TU_All;
-
if (old_key_tuple != NULL && old_key_copied)
heap_freetuple(old_key_tuple);
- bms_free(hot_attrs);
- bms_free(sum_attrs);
- bms_free(key_attrs);
bms_free(id_attrs);
- bms_free(modified_attrs);
- bms_free(interesting_attrs);
+ bms_free(idx_attrs);
+ /* modified_idx_attrs is owned by the caller, don't free it */
return TM_Ok;
}
@@ -4347,28 +4334,115 @@ heap_attr_equals(TupleDesc tupdesc, int attrnum, Datum value1, Datum value2,
}
/*
- * Check which columns are being updated.
- *
- * Given an updated tuple, determine (and return into the output bitmapset),
- * from those listed as interesting, the set of columns that changed.
- *
- * has_external indicates if any of the unmodified attributes (from those
- * listed as interesting) of the old tuple is a member of external_cols and is
- * stored externally.
+ * HOT updates are possible when either: a) there are no modified indexed
+ * attributes, or b) the modified attributes are all on summarizing indexes.
+ * Later, in heap_update(), we can choose to perform a HOT update if there is
+ * space on the page for the new tuple and the following code has determined
+ * that HOT is allowed.
+ */
+bool
+HeapUpdateHotAllowable(Relation relation, const Bitmapset *modified_idx_attrs,
+ bool *summarized_only)
+{
+ bool hot_allowed;
+
+ /*
+ * Let's be optimistic and start off by assuming the best case, no indexes
+ * need updating and HOT is allowable.
+ */
+ hot_allowed = true;
+ *summarized_only = false;
+
+ /*
+ * Check for case (a); when there are no modified index attributes HOT is
+ * allowed.
+ */
+ if (bms_is_empty(modified_idx_attrs))
+ hot_allowed = true;
+ else
+ {
+ Bitmapset *sum_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_SUMMARIZED);
+
+ /*
+ * At least one index attribute was modified, but is this case (b)
+ * where all the modified index attributes are only used by
+ * summarizing indexes? If it is, then we need to update those
+ * indexes, but this update can still be considered heap-only (HOT)
+ * and avoid updating any non-summarizing indexes on the relation.
+ */
+ if (bms_is_subset(modified_idx_attrs, sum_attrs))
+ {
+ hot_allowed = true;
+ *summarized_only = true;
+ }
+ else
+ {
+ /*
+ * Now we know a) one or more indexed attributes were modified
+ * (changed value, not just referenced within the UPDATE) and that
+ * b) at least one of those attributes is used by a
+ * non-summarizing index. HOT is not allowed.
+ */
+ hot_allowed = false;
+ }
+
+ bms_free(sum_attrs);
+ }
+
+ return hot_allowed;
+}
+
+/*
+ * If we're not updating any attributes used when forming the index keys we can
+ * grab a weaker lock type. This allows for more concurrency when we are
+ * running simultaneously with foreign key checks.
+ */
+LockTupleMode
+HeapUpdateDetermineLockmode(Relation relation, const Bitmapset *modified_idx_attrs)
+{
+ LockTupleMode lockmode = LockTupleExclusive;
+
+ Bitmapset *key_attrs = RelationGetIndexAttrBitmap(relation,
+ INDEX_ATTR_BITMAP_KEY);
+
+ if (!bms_overlap(modified_idx_attrs, key_attrs))
+ lockmode = LockTupleNoKeyExclusive;
+
+ bms_free(key_attrs);
+
+ return lockmode;
+}
+
+/*
+ * Return a Bitmapset that contains the set of modified (changed) indexed
+ * attributes between oldtup and newtup.
*/
static Bitmapset *
-HeapDetermineColumnsInfo(Relation relation,
- Bitmapset *interesting_cols,
- Bitmapset *external_cols,
- HeapTuple oldtup, HeapTuple newtup,
- bool *has_external)
+HeapUpdateModifiedIdxAttrs(Relation relation, HeapTuple oldtup, HeapTuple newtup)
{
int attidx;
- Bitmapset *modified = NULL;
+ Bitmapset *attrs,
+ *modified_idx_attrs = NULL;
TupleDesc tupdesc = RelationGetDescr(relation);
+ /* Get the set of all attributes across all indexes for this relation */
+ attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_INDEXED);
+
+ /* No indexed attributes, we're done */
+ if (bms_is_empty(attrs))
+ return NULL;
+
+ /*
+ * This heap update function is used outside the executor and so unlike
+ * heapam_tuple_update() where there is ResultRelInfo and EState to
+ * provide the concise set of attributes that might have been modified
+ * (via ExecGetAllUpdatedCols()) we simply check all indexed attributes to
+ * find the subset that changed value. That's the "modified indexed
+ * attributes" or "modified_idx_attrs".
+ */
attidx = -1;
- while ((attidx = bms_next_member(interesting_cols, attidx)) >= 0)
+ while ((attidx = bms_next_member(attrs, attidx)) >= 0)
{
/* attidx is zero-based, attrnum is the normal attribute number */
AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
@@ -4384,7 +4458,7 @@ HeapDetermineColumnsInfo(Relation relation,
*/
if (attrnum == 0)
{
- modified = bms_add_member(modified, attidx);
+ modified_idx_attrs = bms_add_member(modified_idx_attrs, attidx);
continue;
}
@@ -4397,7 +4471,7 @@ HeapDetermineColumnsInfo(Relation relation,
{
if (attrnum != TableOidAttributeNumber)
{
- modified = bms_add_member(modified, attidx);
+ modified_idx_attrs = bms_add_member(modified_idx_attrs, attidx);
continue;
}
}
@@ -4413,29 +4487,12 @@ HeapDetermineColumnsInfo(Relation relation,
if (!heap_attr_equals(tupdesc, attrnum, value1,
value2, isnull1, isnull2))
- {
- modified = bms_add_member(modified, attidx);
- continue;
- }
-
- /*
- * No need to check attributes that can't be stored externally. Note
- * that system attributes can't be stored externally.
- */
- if (attrnum < 0 || isnull1 ||
- TupleDescCompactAttr(tupdesc, attrnum - 1)->attlen != -1)
- continue;
-
- /*
- * Check if the old tuple's attribute is stored externally and is a
- * member of external_cols.
- */
- if (VARATT_IS_EXTERNAL((varlena *) DatumGetPointer(value1)) &&
- bms_is_member(attidx, external_cols))
- *has_external = true;
+ modified_idx_attrs = bms_add_member(modified_idx_attrs, attidx);
}
- return modified;
+ bms_free(attrs);
+
+ return modified_idx_attrs;
}
/*
@@ -4453,12 +4510,94 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
TM_Result result;
TM_FailureData tmfd;
LockTupleMode lockmode;
+ TupleTableSlot *slot;
+ BufferHeapTupleTableSlot *bslot;
+ HeapTuple oldtup;
+ bool shouldFree = true;
+ Bitmapset *modified_idx_attrs;
+ bool hot_allowed,
+ summarized_only;
+ Buffer buffer;
- result = heap_update(relation, otid, tup,
- GetCurrentCommandId(true), 0,
- InvalidSnapshot,
- true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
+ Assert(ItemPointerIsValid(otid));
+
+ /*
+ * To update a heap tuple we need to find the set of modified indexed
+ * attributes ("modified_idx_attrs") and use that to determine if a HOT
+ * update is allowable or not. When updating heap tuples via execution of
+ * UPDATE statements this set is constructed before calling into the table
+ * AM's update function by ExecUpdateModifiedIdxAttrs() which compares the
+ * old/new TupleTableSlots.
+ *
+ * Here things are a bit different, we have the old TID and the new tuple,
+ * not two TupleTableSlots, but we still need to construct a similar
+ * bitmap so as to be able to know if HOT updates are allowed or not.
+ *
+ * To do that we first have to fetch the old tuple itself, but because
+ * heapam_fetch_row_version() is static, we replicate in part that code
+ * here.
+ *
+ * This is a bit repetitive because heap_update() will again find and form
+ * the old HeapTuple from the old TID and in most cases the callers
+ * (ignoring extensions, are always catalog tuple updates) already had the
+ * set of changed attributes (the "replaces" array), but for now this
+ * minor repetition of work is necessary.
+ */
+ INJECTION_POINT("simple_heap_update-before-pin", NULL);
+
+ slot = MakeTupleTableSlot(RelationGetDescr(relation), &TTSOpsBufferHeapTuple, 0);
+ bslot = (BufferHeapTupleTableSlot *) slot;
+
+ /*
+ * Set the TID in the slot and then fetch the old tuple so we can examine
+ * it
+ */
+ bslot->base.tupdata.t_self = *otid;
+ if (!heap_fetch(relation, SnapshotAny, &bslot->base.tupdata, &buffer, false))
+ {
+ /*
+ * heap_update() checks for !ItemIdIsNormal(lp) and will return false
+ * in those cases.
+ */
+ Assert(RelationSupportsSysCache(RelationGetRelid(relation)));
+
+ *update_indexes = TU_None;
+
+ /* modified_idx_attrs not yet initialized */
+ ExecDropSingleTupleTableSlot(slot);
+
+ elog(ERROR, "tuple concurrently deleted");
+
+ return;
+ }
+
+ Assert(buffer != InvalidBuffer);
+
+ /* Store in slot, transferring existing pin */
+ ExecStorePinnedBufferHeapTuple(&bslot->base.tupdata, slot, buffer);
+ oldtup = ExecFetchSlotHeapTuple(slot, false, &shouldFree);
+
+ modified_idx_attrs = HeapUpdateModifiedIdxAttrs(relation, oldtup, tup);
+ lockmode = HeapUpdateDetermineLockmode(relation, modified_idx_attrs);
+ hot_allowed = HeapUpdateHotAllowable(relation, modified_idx_attrs, &summarized_only);
+
+ result = heap_update(relation, otid, tup, GetCurrentCommandId(true), 0,
+ InvalidSnapshot, true /* wait for commit */ ,
+ &tmfd, lockmode, modified_idx_attrs, hot_allowed);
+
+ if (shouldFree)
+ heap_freetuple(oldtup);
+
+ ExecDropSingleTupleTableSlot(slot);
+
+ /*
+ * Decide whether new index entries are needed for the tuple
+ *
+ * If the update is not HOT, we must update all indexes. If the update is
+ * HOT, it could be that we updated summarized columns, so we either
+ * update only summarized indexes, or none at all.
+ */
+ *update_indexes = TU_None;
switch (result)
{
case TM_SelfModified:
@@ -4468,6 +4607,10 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
case TM_Ok:
/* done successfully */
+ if (!HeapTupleIsHeapOnly(tup))
+ *update_indexes = TU_All;
+ else if (summarized_only)
+ *update_indexes = TU_Summarizing;
break;
case TM_Updated:
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bf87430cf01..75b4d40aafc 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -27,7 +27,6 @@
#include "access/syncscan.h"
#include "access/tableam.h"
#include "access/tsmapi.h"
-#include "access/visibilitymap.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/index.h"
@@ -218,20 +217,26 @@ static TM_Result
heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, uint32 options,
Snapshot snapshot, Snapshot crosscheck,
- bool wait, TM_FailureData *tmfd,
- LockTupleMode *lockmode, TU_UpdateIndexes *update_indexes)
+ bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
+ const Bitmapset *modified_idx_attrs, TU_UpdateIndexes *update_indexes)
{
bool shouldFree = true;
HeapTuple tuple = ExecFetchSlotHeapTuple(slot, true, &shouldFree);
+ bool hot_allowed;
+ bool summarized_only;
TM_Result result;
+ Assert(ItemPointerIsValid(otid));
+
+ hot_allowed = HeapUpdateHotAllowable(relation, modified_idx_attrs, &summarized_only);
+ *lockmode = HeapUpdateDetermineLockmode(relation, modified_idx_attrs);
+
/* Update the tuple with table oid */
slot->tts_tableOid = RelationGetRelid(relation);
tuple->t_tableOid = slot->tts_tableOid;
- result = heap_update(relation, otid, tuple, cid, options,
- crosscheck, wait,
- tmfd, lockmode, update_indexes);
+ result = heap_update(relation, otid, tuple, cid, options, crosscheck, wait,
+ tmfd, *lockmode, modified_idx_attrs, hot_allowed);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
/*
@@ -244,16 +249,17 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
* HOT, it could be that we updated summarized columns, so we either
* update only summarized indexes, or none at all.
*/
- if (result != TM_Ok)
+ *update_indexes = TU_None;
+ if (result == TM_Ok)
{
- Assert(*update_indexes == TU_None);
- *update_indexes = TU_None;
+ if (HeapTupleIsHeapOnly(tuple))
+ {
+ if (summarized_only)
+ *update_indexes = TU_Summarizing;
+ }
+ else
+ *update_indexes = TU_All;
}
- else if (!HeapTupleIsHeapOnly(tuple))
- Assert(*update_indexes == TU_All);
- else
- Assert((*update_indexes == TU_Summarizing) ||
- (*update_indexes == TU_None));
if (shouldFree)
pfree(tuple);
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index 68ff0966f1c..12c2674cbd7 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -361,6 +361,7 @@ void
simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot,
Snapshot snapshot,
+ const Bitmapset *modified_idx_attrs,
TU_UpdateIndexes *update_indexes)
{
TM_Result result;
@@ -371,7 +372,9 @@ simple_table_tuple_update(Relation rel, ItemPointer otid,
GetCurrentCommandId(true),
0, snapshot, InvalidSnapshot,
true /* wait for commit */ ,
- &tmfd, &lockmode, update_indexes);
+ &tmfd, &lockmode,
+ modified_idx_attrs,
+ update_indexes);
switch (result)
{
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index faa07d1a118..9761bdac9d0 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -2689,8 +2689,18 @@ apply_concurrent_update(Relation rel, TupleTableSlot *spilled_tuple,
LockTupleMode lockmode;
TM_FailureData tmfd;
TU_UpdateIndexes update_indexes;
+ Bitmapset *modified_idx_attrs;
TM_Result res;
+ /*
+ * Compute the set of modified indexed attributes by comparing the old
+ * (ondisk) and new (spilled) tuples; heap_update needs it for a correct
+ * HOT decision (a NULL set would look like "no indexed column changed").
+ */
+ modified_idx_attrs = ExecUpdateModifiedIdxAttrs(chgcxt->cc_rri,
+ ondisk_tuple,
+ spilled_tuple);
+
/*
* Carry out the update, skipping logical decoding for it.
*/
@@ -2700,7 +2710,7 @@ apply_concurrent_update(Relation rel, TupleTableSlot *spilled_tuple,
InvalidSnapshot,
InvalidSnapshot,
false,
- &tmfd, &lockmode, &update_indexes);
+ &tmfd, &lockmode, modified_idx_attrs, &update_indexes);
if (res != TM_Ok)
ereport(ERROR,
errcode(ERRCODE_T_R_SERIALIZATION_FAILURE),
@@ -2721,6 +2731,8 @@ apply_concurrent_update(Relation rel, TupleTableSlot *spilled_tuple,
}
pgstat_progress_incr_param(PROGRESS_REPACK_HEAP_TUPLES_UPDATED, 1);
+
+ bms_free(modified_idx_attrs);
}
static void
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index b2ca5cbf117..6262f71bd93 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -33,6 +33,7 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
#include "utils/typcache.h"
@@ -910,6 +911,7 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
bool skip_tuple = false;
Relation rel = resultRelInfo->ri_RelationDesc;
ItemPointer tid = &(searchslot->tts_tid);
+ Bitmapset *modified_idx_attrs;
/*
* We support only non-system tables, with
@@ -948,8 +950,13 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
if (rel->rd_rel->relispartition)
ExecPartitionCheck(resultRelInfo, slot, estate, true);
+ modified_idx_attrs = ExecUpdateModifiedIdxAttrs(resultRelInfo,
+ searchslot, slot);
+
simple_table_tuple_update(rel, tid, slot, estate->es_snapshot,
- &update_indexes);
+ modified_idx_attrs, &update_indexes);
+ bms_free(modified_idx_attrs);
+
conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 7f4ebf95432..29f9e6ce67c 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -66,6 +66,7 @@
#include "nodes/nodeFuncs.h"
#include "storage/bufmgr.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/expandeddatum.h"
#include "utils/lsyscache.h"
#include "utils/typcache.h"
@@ -2012,6 +2013,77 @@ ExecFetchSlotHeapTupleDatum(TupleTableSlot *slot)
return ret;
}
+/*
+ * ExecCompareSlotAttrs
+ *
+ * Compare the subset of attributes in attrs between TupleTableSlots to detect
+ * which attributes have changed.
+ *
+ * The input Bitmapset attrs is modified in place (recycled when possible via
+ * bms_del_member, which may pfree it and return NULL) and may be freed;
+ * callers must use only the returned pointer, not their original attrs
+ * value. Returns the Bitmapset of attribute indices (using the
+ * FirstLowInvalidHeapAttributeNumber convention) that differ between the two
+ * slots.
+ */
+Bitmapset *
+ExecCompareSlotAttrs(Bitmapset *attrs, TupleDesc tupdesc,
+ TupleTableSlot *s1, TupleTableSlot *s2)
+{
+ int attidx = -1;
+
+ while ((attidx = bms_next_member(attrs, attidx)) >= 0)
+ {
+ /* attidx is zero-based, attrnum is the normal attribute number */
+ AttrNumber attrnum = attidx + FirstLowInvalidHeapAttributeNumber;
+ Datum value1,
+ value2;
+ bool null1,
+ null2;
+ CompactAttribute *att;
+
+ /*
+ * If it's a whole-tuple reference, say "not equal". It's not really
+ * worth supporting this case, since it could only succeed after a
+ * no-op update, which is hardly a case worth optimizing for.
+ */
+ if (attrnum == 0)
+ continue;
+
+ /*
+ * Likewise, automatically say "not equal" for any system attribute
+ * other than tableOID; we cannot expect these to be consistent in a
+ * HOT chain, or even to be set correctly yet in the new tuple.
+ */
+ if (attrnum < 0)
+ {
+ if (attrnum == TableOidAttributeNumber)
+ attrs = bms_del_member(attrs, attidx);
+ continue;
+ }
+
+ att = TupleDescCompactAttr(tupdesc, attrnum - 1);
+ value1 = slot_getattr(s1, attrnum, &null1);
+ value2 = slot_getattr(s2, attrnum, &null2);
+
+ /* A change to/from NULL, so not equal */
+ if (null1 != null2)
+ continue;
+
+ /* Both NULL, no change/unmodified */
+ if (null2)
+ {
+ attrs = bms_del_member(attrs, attidx);
+ continue;
+ }
+
+ if (datum_image_eq(value1, value2, att->attbyval, att->attlen))
+ attrs = bms_del_member(attrs, attidx);
+ }
+
+ return attrs;
+}
+
/* ----------------------------------------------------------------
* convenience initialization routines
* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index c333d7139fa..eb6e0ef2ad9 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -18,6 +18,7 @@
* ExecModifyTable - retrieve the next tuple from the node
* ExecEndModifyTable - shut down the ModifyTable node
* ExecReScanModifyTable - rescan the ModifyTable node
+ * ExecUpdateModifiedIdxAttrs - find set of updated indexed columns
*
* NOTES
* The ModifyTable node receives input from its outerPlan, which is
@@ -56,6 +57,7 @@
#include "access/htup_details.h"
#include "access/tableam.h"
#include "access/tupconvert.h"
+#include "access/tupdesc.h"
#include "access/xact.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
@@ -202,6 +204,63 @@ static void fireASTriggers(ModifyTableState *node);
static void ExecInitForPortionOf(ModifyTableState *mtstate, EState *estate,
ResultRelInfo *resultRelInfo);
+/*
+ * ExecUpdateModifiedIdxAttrs
+ *
+ * Find the set of attributes referenced by this relation and used in this
+ * UPDATE that now differ in value. This is done by reviewing slot datum that
+ * are in the UPDATE statment and are known to be referenced by at least one
+ * index in some way. This set is called the "modified indexed attributes" or
+ * "modified_idx_attrs". An overlap of a single index's attributes and this
+ * modified_idx_attrs set signals that the attributes in the new_tts used to
+ * form the index datum have changed.
+ *
+ * Return a Bitmapset that contains the set of modified (changed) indexed
+ * attributes between oldtup and newtup.
+ *
+ * Note: There is a similar function called HeapUpdateModifiedIdxAttrs() that operates
+ * on the old TID and new HeapTuple rather than the old/new TupleTableSlots as
+ * this function does. These two functions should mirror one another until
+ * someday when catalog tuple updates track their changes avoiding the need to
+ * re-discover them in simple_heap_update().
+ */
+Bitmapset *
+ExecUpdateModifiedIdxAttrs(ResultRelInfo *resultRelInfo,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts)
+{
+ Relation relation = resultRelInfo->ri_RelationDesc;
+ TupleDesc tupdesc = RelationGetDescr(relation);
+ Bitmapset *attrs;
+
+ /* If no indexes, we're done */
+ if (resultRelInfo->ri_NumIndices == 0)
+ return NULL;
+
+ /*
+ * Get the set of all attributes across all indexes for this relation from
+ * the relcache, it returns us a copy of the bitmap so we can modify it.
+ *
+ * Note: We intentionally scan all indexed columns when looking for
+ * changes rather than reduce that set by intersecting it with
+ * ExecGetAllUpdatedCols(). Desipte the name it provides the set of
+ * targeted attributes in the SQL used for the UPDATE and any triggers,
+ * but that doesn't include any attributes updated using
+ * heap_modifiy_tuple(). There is one test in tsearch.sql that does just
+ * that, modifies an indexed attribute that isn't specified in the SQL and
+ * so isn't present in that bitmapset.
+ */
+ attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_INDEXED);
+
+ /*
+ * When there are indexed attributes mentioned in the UPDATE then we need
+ * to find the subset that changed value. That's the
+ * "modified_idx_attrs".
+ */
+ attrs = ExecCompareSlotAttrs(attrs, tupdesc, old_tts, new_tts);
+
+ return attrs;
+}
/*
* Verify that the tuples to be produced by INSERT match the
@@ -2446,14 +2505,17 @@ ExecUpdatePrepareSlot(ResultRelInfo *resultRelInfo,
*/
static TM_Result
ExecUpdateAct(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
- ItemPointer tupleid, HeapTuple oldtuple, TupleTableSlot *slot,
- bool canSetTag, UpdateContext *updateCxt)
+ ItemPointer tupleid, HeapTuple oldtuple, TupleTableSlot *oldSlot,
+ TupleTableSlot *slot, bool canSetTag, UpdateContext *updateCxt)
{
EState *estate = context->estate;
Relation resultRelationDesc = resultRelInfo->ri_RelationDesc;
bool partition_constraint_failed;
TM_Result result;
+ /* The set of modified indexed attributes that trigger new index entries */
+ Bitmapset *modified_idx_attrs = NULL;
+
updateCxt->crossPartUpdate = false;
/*
@@ -2570,7 +2632,16 @@ lreplace:
ExecConstraints(resultRelInfo, slot, estate);
/*
- * replace the heap tuple
+ * Next up we need to find out the set of indexed attributes that have
+ * changed in value and should trigger a new index tuple. We could start
+ * with the set of updated columns via ExecGetUpdatedCols(), but if we do
+ * we will overlook attributes directly modified by heap_modify_tuple()
+ * which are not known to ExecGetUpdatedCols().
+ */
+ modified_idx_attrs = ExecUpdateModifiedIdxAttrs(resultRelInfo, oldSlot, slot);
+
+ /*
+ * Call into the table AM to update the heap tuple.
*
* Note: if es_crosscheck_snapshot isn't InvalidSnapshot, we check that
* the row to be updated is visible to that snapshot, and throw a
@@ -2585,6 +2656,7 @@ lreplace:
estate->es_crosscheck_snapshot,
true /* wait for commit */ ,
&context->tmfd, &updateCxt->lockmode,
+ modified_idx_attrs,
&updateCxt->updateIndexes);
return result;
@@ -2812,8 +2884,8 @@ ExecUpdate(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
*/
redo_act:
lockedtid = *tupleid;
- result = ExecUpdateAct(context, resultRelInfo, tupleid, oldtuple, slot,
- canSetTag, &updateCxt);
+ result = ExecUpdateAct(context, resultRelInfo, tupleid, oldtuple, oldSlot,
+ slot, canSetTag, &updateCxt);
/*
* If ExecUpdateAct reports that a cross-partition update was done,
@@ -3663,8 +3735,8 @@ lmerge_matched:
Assert(oldtuple == NULL);
result = ExecUpdateAct(context, resultRelInfo, tupleid,
- NULL, newslot, canSetTag,
- &updateCxt);
+ NULL, resultRelInfo->ri_oldTupleSlot,
+ newslot, canSetTag, &updateCxt);
/*
* As in ExecUpdate(), if ExecUpdateAct() reports that a
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index fb4e042be8a..055f757107f 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -2482,7 +2482,7 @@ RelationDestroyRelation(Relation relation, bool remember_tupdesc)
bms_free(relation->rd_keyattr);
bms_free(relation->rd_pkattr);
bms_free(relation->rd_idattr);
- bms_free(relation->rd_hotblockingattr);
+ bms_free(relation->rd_indexedattr);
bms_free(relation->rd_summarizedattr);
if (relation->rd_pubdesc)
pfree(relation->rd_pubdesc);
@@ -5291,8 +5291,8 @@ RelationGetIndexPredicate(Relation relation)
* (beware: even if PK is deferrable!)
* INDEX_ATTR_BITMAP_IDENTITY_KEY Columns in the table's replica identity
* index (empty if FULL)
- * INDEX_ATTR_BITMAP_HOT_BLOCKING Columns that block updates from being HOT
- * INDEX_ATTR_BITMAP_SUMMARIZED Columns included in summarizing indexes
+ * INDEX_ATTR_BITMAP_INDEXED Columns referenced by indexes
+ * INDEX_ATTR_BITMAP_SUMMARIZED Columns only included in summarizing indexes
*
* Attribute numbers are offset by FirstLowInvalidHeapAttributeNumber so that
* we can include system attributes (e.g., OID) in the bitmap representation.
@@ -5315,8 +5315,8 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
Bitmapset *uindexattrs; /* columns in unique indexes */
Bitmapset *pkindexattrs; /* columns in the primary index */
Bitmapset *idindexattrs; /* columns in the replica identity */
- Bitmapset *hotblockingattrs; /* columns with HOT blocking indexes */
- Bitmapset *summarizedattrs; /* columns with summarizing indexes */
+ Bitmapset *indexedattrs; /* columns referenced by indexes */
+ Bitmapset *summarizedattrs; /* columns only in summarizing indexes */
List *indexoidlist;
List *newindexoidlist;
Oid relpkindex;
@@ -5335,8 +5335,8 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
return bms_copy(relation->rd_pkattr);
case INDEX_ATTR_BITMAP_IDENTITY_KEY:
return bms_copy(relation->rd_idattr);
- case INDEX_ATTR_BITMAP_HOT_BLOCKING:
- return bms_copy(relation->rd_hotblockingattr);
+ case INDEX_ATTR_BITMAP_INDEXED:
+ return bms_copy(relation->rd_indexedattr);
case INDEX_ATTR_BITMAP_SUMMARIZED:
return bms_copy(relation->rd_summarizedattr);
default:
@@ -5381,7 +5381,7 @@ restart:
uindexattrs = NULL;
pkindexattrs = NULL;
idindexattrs = NULL;
- hotblockingattrs = NULL;
+ indexedattrs = NULL;
summarizedattrs = NULL;
foreach(l, indexoidlist)
{
@@ -5441,7 +5441,7 @@ restart:
if (indexDesc->rd_indam->amsummarizing)
attrs = &summarizedattrs;
else
- attrs = &hotblockingattrs;
+ attrs = &indexedattrs;
/* Collect simple attribute references */
for (i = 0; i < indexDesc->rd_index->indnatts; i++)
@@ -5450,9 +5450,9 @@ restart:
/*
* Since we have covering indexes with non-key columns, we must
- * handle them accurately here. non-key columns must be added into
- * hotblockingattrs or summarizedattrs, since they are in index,
- * and update shouldn't miss them.
+ * handle them accurately here. Non-key columns must be added into
+ * indexedattrs or summarizedattrs, since they are in index, and
+ * update shouldn't miss them.
*
* Summarizing indexes do not block HOT, but do need to be updated
* when the column value changes, thus require a separate
@@ -5513,12 +5513,20 @@ restart:
bms_free(uindexattrs);
bms_free(pkindexattrs);
bms_free(idindexattrs);
- bms_free(hotblockingattrs);
+ bms_free(indexedattrs);
bms_free(summarizedattrs);
goto restart;
}
+ /*
+ * Record what attributes are only referenced by summarizing indexes. Then
+ * add that into the other indexed attributes to track all referenced
+ * attributes.
+ */
+ summarizedattrs = bms_del_members(summarizedattrs, indexedattrs);
+ indexedattrs = bms_add_members(indexedattrs, summarizedattrs);
+
/* Don't leak the old values of these bitmaps, if any */
relation->rd_attrsvalid = false;
bms_free(relation->rd_keyattr);
@@ -5527,8 +5535,8 @@ restart:
relation->rd_pkattr = NULL;
bms_free(relation->rd_idattr);
relation->rd_idattr = NULL;
- bms_free(relation->rd_hotblockingattr);
- relation->rd_hotblockingattr = NULL;
+ bms_free(relation->rd_indexedattr);
+ relation->rd_indexedattr = NULL;
bms_free(relation->rd_summarizedattr);
relation->rd_summarizedattr = NULL;
@@ -5543,7 +5551,7 @@ restart:
relation->rd_keyattr = bms_copy(uindexattrs);
relation->rd_pkattr = bms_copy(pkindexattrs);
relation->rd_idattr = bms_copy(idindexattrs);
- relation->rd_hotblockingattr = bms_copy(hotblockingattrs);
+ relation->rd_indexedattr = bms_copy(indexedattrs);
relation->rd_summarizedattr = bms_copy(summarizedattrs);
relation->rd_attrsvalid = true;
MemoryContextSwitchTo(oldcxt);
@@ -5557,8 +5565,8 @@ restart:
return pkindexattrs;
case INDEX_ATTR_BITMAP_IDENTITY_KEY:
return idindexattrs;
- case INDEX_ATTR_BITMAP_HOT_BLOCKING:
- return hotblockingattrs;
+ case INDEX_ATTR_BITMAP_INDEXED:
+ return indexedattrs;
case INDEX_ATTR_BITMAP_SUMMARIZED:
return summarizedattrs;
default:
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 5176478c295..2dbfad92113 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -385,11 +385,10 @@ extern TM_Result heap_delete(Relation relation, const ItemPointerData *tid,
extern void heap_finish_speculative(Relation relation, const ItemPointerData *tid);
extern void heap_abort_speculative(Relation relation, const ItemPointerData *tid);
extern TM_Result heap_update(Relation relation, const ItemPointerData *otid,
- HeapTuple newtup,
- CommandId cid, uint32 options,
+ HeapTuple newtup, CommandId cid, uint32 options,
Snapshot crosscheck, bool wait,
- TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes);
+ TM_FailureData *tmfd, const LockTupleMode lockmode,
+ const Bitmapset *modified_idx_attrs, const bool hot_allowed);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
bool follow_updates,
@@ -464,6 +463,12 @@ extern void log_heap_prune_and_freeze(Relation relation, Buffer buffer,
OffsetNumber *dead, int ndead,
OffsetNumber *unused, int nunused);
+/* in heap/heapam.c */
+extern bool HeapUpdateHotAllowable(Relation relation, const Bitmapset *modified_idx_attrs,
+ bool *summarized_only);
+extern LockTupleMode HeapUpdateDetermineLockmode(Relation relation,
+ const Bitmapset *modified_idx_attrs);
+
/* in heap/vacuumlazy.c */
extern void heap_vacuum_rel(Relation rel,
const VacuumParams *params, BufferAccessStrategy bstrategy);
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index f2c36696bca..a9778b3528d 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -586,6 +586,7 @@ typedef struct TableAmRoutine
bool wait,
TM_FailureData *tmfd,
LockTupleMode *lockmode,
+ const Bitmapset *modified_idx_attrs,
TU_UpdateIndexes *update_indexes);
/* see table_tuple_lock() for reference about parameters */
@@ -1599,12 +1600,12 @@ table_tuple_update(Relation rel, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, uint32 options,
Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
- TU_UpdateIndexes *update_indexes)
+ const Bitmapset *modified_idx_attrs, TU_UpdateIndexes *update_indexes)
{
return rel->rd_tableam->tuple_update(rel, otid, slot,
cid, options, snapshot, crosscheck,
- wait, tmfd,
- lockmode, update_indexes);
+ wait, tmfd, lockmode,
+ modified_idx_attrs, update_indexes);
}
/*
@@ -2089,6 +2090,7 @@ extern void simple_table_tuple_delete(Relation rel, ItemPointer tid,
Snapshot snapshot);
extern void simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot, Snapshot snapshot,
+ const Bitmapset *modified_idx_attrs,
TU_UpdateIndexes *update_indexes);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 1798e6027d4..16661bc66d9 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -18,6 +18,7 @@
#include "datatype/timestamp.h"
#include "executor/execdesc.h"
#include "fmgr.h"
+#include "nodes/execnodes.h"
#include "nodes/lockoptions.h"
#include "nodes/parsenodes.h"
#include "utils/memutils.h"
@@ -615,6 +616,10 @@ extern TupleDesc ExecCleanTypeFromTL(List *targetList);
extern TupleDesc ExecTypeFromExprList(List *exprList);
extern void ExecTypeSetColNames(TupleDesc typeInfo, List *namesList);
extern void UpdateChangedParamSet(PlanState *node, Bitmapset *newchg);
+extern Bitmapset *ExecCompareSlotAttrs(Bitmapset *attrs,
+ TupleDesc tupdesc,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts);
typedef struct TupOutputState
{
@@ -814,5 +819,8 @@ extern ResultRelInfo *ExecLookupResultRelByOid(ModifyTableState *node,
Oid resultoid,
bool missing_ok,
bool update_cache);
+extern Bitmapset *ExecUpdateModifiedIdxAttrs(ResultRelInfo *relinfo,
+ TupleTableSlot *old_tts,
+ TupleTableSlot *new_tts);
#endif /* EXECUTOR_H */
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 89c159b133f..17dbdbe645d 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -162,7 +162,7 @@ typedef struct RelationData
Bitmapset *rd_keyattr; /* cols that can be ref'd by foreign keys */
Bitmapset *rd_pkattr; /* cols included in primary key */
Bitmapset *rd_idattr; /* included in replica identity index */
- Bitmapset *rd_hotblockingattr; /* cols blocking HOT update */
+ Bitmapset *rd_indexedattr; /* all cols referenced by indexes */
Bitmapset *rd_summarizedattr; /* cols indexed by summarizing indexes */
PublicationDesc *rd_pubdesc; /* publication descriptor, or NULL */
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 89c27aa1529..89788091576 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -70,7 +70,7 @@ typedef enum IndexAttrBitmapKind
INDEX_ATTR_BITMAP_KEY,
INDEX_ATTR_BITMAP_PRIMARY_KEY,
INDEX_ATTR_BITMAP_IDENTITY_KEY,
- INDEX_ATTR_BITMAP_HOT_BLOCKING,
+ INDEX_ATTR_BITMAP_INDEXED,
INDEX_ATTR_BITMAP_SUMMARIZED,
} IndexAttrBitmapKind;
diff --git a/src/test/modules/injection_points/expected/syscache-update-pruned.out b/src/test/modules/injection_points/expected/syscache-update-pruned.out
index a6a4e8db996..07ef67a1eb4 100644
--- a/src/test/modules/injection_points/expected/syscache-update-pruned.out
+++ b/src/test/modules/injection_points/expected/syscache-update-pruned.out
@@ -16,8 +16,8 @@ step wakeinval4:
step at2: <... completed>
step wakeinval4: <... completed>
step wakegrant4:
- SELECT FROM injection_points_detach('heap_update-before-pin');
- SELECT FROM injection_points_wakeup('heap_update-before-pin');
+ SELECT FROM injection_points_detach('simple_heap_update-before-pin');
+ SELECT FROM injection_points_wakeup('simple_heap_update-before-pin');
<waiting ...>
step grant1: <... completed>
ERROR: tuple concurrently deleted
@@ -42,8 +42,8 @@ step mkrels4:
SELECT FROM vactest.mkrels('intruder', 1, 100); -- repopulate LP_UNUSED
step wakegrant4:
- SELECT FROM injection_points_detach('heap_update-before-pin');
- SELECT FROM injection_points_wakeup('heap_update-before-pin');
+ SELECT FROM injection_points_detach('simple_heap_update-before-pin');
+ SELECT FROM injection_points_wakeup('simple_heap_update-before-pin');
<waiting ...>
step grant1: <... completed>
ERROR: duplicate key value violates unique constraint "pg_class_oid_index"
@@ -71,8 +71,8 @@ step at2: <... completed>
step wakeinval4: <... completed>
step at4: ALTER TABLE vactest.child50 INHERIT vactest.orig50;
step wakegrant4:
- SELECT FROM injection_points_detach('heap_update-before-pin');
- SELECT FROM injection_points_wakeup('heap_update-before-pin');
+ SELECT FROM injection_points_detach('simple_heap_update-before-pin');
+ SELECT FROM injection_points_wakeup('simple_heap_update-before-pin');
<waiting ...>
step grant1: <... completed>
step wakegrant4: <... completed>
diff --git a/src/test/modules/injection_points/specs/syscache-update-pruned.spec b/src/test/modules/injection_points/specs/syscache-update-pruned.spec
index e3a4295bd12..fef9ac895a1 100644
--- a/src/test/modules/injection_points/specs/syscache-update-pruned.spec
+++ b/src/test/modules/injection_points/specs/syscache-update-pruned.spec
@@ -103,7 +103,7 @@ session s1
setup {
SET debug_discard_caches = 0;
SELECT FROM injection_points_set_local();
- SELECT FROM injection_points_attach('heap_update-before-pin', 'wait');
+ SELECT FROM injection_points_attach('simple_heap_update-before-pin', 'wait');
}
step cachefill1 { SELECT FROM vactest.reloid_catcache_set('vactest.orig50'); }
step grant1 { GRANT SELECT ON vactest.orig50 TO PUBLIC; }
@@ -140,8 +140,8 @@ step mkrels4 {
SELECT FROM vactest.mkrels('intruder', 1, 100); -- repopulate LP_UNUSED
}
step wakegrant4 {
- SELECT FROM injection_points_detach('heap_update-before-pin');
- SELECT FROM injection_points_wakeup('heap_update-before-pin');
+ SELECT FROM injection_points_detach('simple_heap_update-before-pin');
+ SELECT FROM injection_points_wakeup('simple_heap_update-before-pin');
}
step at4 { ALTER TABLE vactest.child50 INHERIT vactest.orig50; }
step wakeinval4 {
diff --git a/src/test/regress/expected/generated_virtual.out b/src/test/regress/expected/generated_virtual.out
index 01ee29fee10..a7a2463e54d 100644
--- a/src/test/regress/expected/generated_virtual.out
+++ b/src/test/regress/expected/generated_virtual.out
@@ -287,7 +287,7 @@ DETAIL: Column "b" is a generated column.
INSERT INTO gtest1v VALUES (8, DEFAULT), (9, DEFAULT); -- error
ERROR: cannot insert a non-DEFAULT value into column "b"
DETAIL: Column "b" is a generated column.
-SELECT * FROM gtest1v;
+SELECT * FROM gtest1v ORDER BY a;
a | b
---+----
3 | 6
diff --git a/src/test/regress/expected/triggers.out b/src/test/regress/expected/triggers.out
index 8fcb33ac81a..00ebe305875 100644
--- a/src/test/regress/expected/triggers.out
+++ b/src/test/regress/expected/triggers.out
@@ -959,16 +959,24 @@ NOTICE: main_view BEFORE UPDATE STATEMENT (before_view_upd_stmt)
NOTICE: main_view AFTER UPDATE STATEMENT (after_view_upd_stmt)
UPDATE 0
-- Delete from view using trigger
-DELETE FROM main_view WHERE a IN (20,21);
+DELETE FROM main_view WHERE a = 20 AND b = 31;
NOTICE: main_view BEFORE DELETE STATEMENT (before_view_del_stmt)
NOTICE: main_view INSTEAD OF DELETE ROW (instead_of_del)
-NOTICE: OLD: (21,10)
-NOTICE: main_view INSTEAD OF DELETE ROW (instead_of_del)
NOTICE: OLD: (20,31)
+NOTICE: main_view AFTER DELETE STATEMENT (after_view_del_stmt)
+DELETE 1
+DELETE FROM main_view WHERE a = 21 AND b = 10;
+NOTICE: main_view BEFORE DELETE STATEMENT (before_view_del_stmt)
+NOTICE: main_view INSTEAD OF DELETE ROW (instead_of_del)
+NOTICE: OLD: (21,10)
+NOTICE: main_view AFTER DELETE STATEMENT (after_view_del_stmt)
+DELETE 1
+DELETE FROM main_view WHERE a = 21 AND b = 32;
+NOTICE: main_view BEFORE DELETE STATEMENT (before_view_del_stmt)
NOTICE: main_view INSTEAD OF DELETE ROW (instead_of_del)
NOTICE: OLD: (21,32)
NOTICE: main_view AFTER DELETE STATEMENT (after_view_del_stmt)
-DELETE 3
+DELETE 1
DELETE FROM main_view WHERE a = 31 RETURNING a, b;
NOTICE: main_view BEFORE DELETE STATEMENT (before_view_del_stmt)
NOTICE: main_view INSTEAD OF DELETE ROW (instead_of_del)
diff --git a/src/test/regress/expected/tsearch.out b/src/test/regress/expected/tsearch.out
index 5b7c2123f37..6dc193f02d6 100644
--- a/src/test/regress/expected/tsearch.out
+++ b/src/test/regress/expected/tsearch.out
@@ -2493,7 +2493,8 @@ SELECT to_tsquery('SKIES & My | booKs');
'sky' | 'book'
(1 row)
---trigger
+-- tsvector_update_trigger() uses heap_modify_tuple() to set column 'a'
+-- without going through the executor's SET-clause tracking.
CREATE TRIGGER tsvectorupdate
BEFORE UPDATE OR INSERT ON test_tsvector
FOR EACH ROW EXECUTE PROCEDURE tsvector_update_trigger(a, 'pg_catalog.english', t);
diff --git a/src/test/regress/expected/updatable_views.out b/src/test/regress/expected/updatable_views.out
index 7b00c742776..cd6e71ac490 100644
--- a/src/test/regress/expected/updatable_views.out
+++ b/src/test/regress/expected/updatable_views.out
@@ -372,15 +372,15 @@ INSERT INTO rw_view16 (a, b) VALUES (3, 'Row 3'); -- should be OK
UPDATE rw_view16 SET a=3, aa=-3 WHERE a=3; -- should fail
ERROR: multiple assignments to same column "a"
UPDATE rw_view16 SET aa=-3 WHERE a=3; -- should be OK
-SELECT * FROM base_tbl;
+SELECT * FROM base_tbl ORDER BY a;
a | b
----+--------
+ -3 | Row 3
-2 | Row -2
-1 | Row -1
0 | Row 0
1 | Row 1
2 | Row 2
- -3 | Row 3
(6 rows)
DELETE FROM rw_view16 WHERE a=-3; -- should be OK
diff --git a/src/test/regress/sql/generated_virtual.sql b/src/test/regress/sql/generated_virtual.sql
index 0cb14eb0e36..bc91742a492 100644
--- a/src/test/regress/sql/generated_virtual.sql
+++ b/src/test/regress/sql/generated_virtual.sql
@@ -127,7 +127,7 @@ ALTER VIEW gtest1v ALTER COLUMN b SET DEFAULT 100;
INSERT INTO gtest1v VALUES (8, DEFAULT); -- error
INSERT INTO gtest1v VALUES (8, DEFAULT), (9, DEFAULT); -- error
-SELECT * FROM gtest1v;
+SELECT * FROM gtest1v ORDER BY a;
DELETE FROM gtest1v WHERE a >= 5;
DROP VIEW gtest1v;
diff --git a/src/test/regress/sql/triggers.sql b/src/test/regress/sql/triggers.sql
index 2285e90110e..19c2572201f 100644
--- a/src/test/regress/sql/triggers.sql
+++ b/src/test/regress/sql/triggers.sql
@@ -660,7 +660,9 @@ UPDATE main_view SET b = 32 WHERE a = 21 AND b = 31 RETURNING a, b;
UPDATE main_view SET b = 0 WHERE false;
-- Delete from view using trigger
-DELETE FROM main_view WHERE a IN (20,21);
+DELETE FROM main_view WHERE a = 20 AND b = 31;
+DELETE FROM main_view WHERE a = 21 AND b = 10;
+DELETE FROM main_view WHERE a = 21 AND b = 32;
DELETE FROM main_view WHERE a = 31 RETURNING a, b;
\set QUIET true
diff --git a/src/test/regress/sql/tsearch.sql b/src/test/regress/sql/tsearch.sql
index 8b3d700f57c..094181e7764 100644
--- a/src/test/regress/sql/tsearch.sql
+++ b/src/test/regress/sql/tsearch.sql
@@ -760,7 +760,8 @@ SELECT to_tsvector('SKIES My booKs');
SELECT plainto_tsquery('SKIES My booKs');
SELECT to_tsquery('SKIES & My | booKs');
---trigger
+-- tsvector_update_trigger() uses heap_modify_tuple() to set column 'a'
+-- without going through the executor's SET-clause tracking.
CREATE TRIGGER tsvectorupdate
BEFORE UPDATE OR INSERT ON test_tsvector
FOR EACH ROW EXECUTE PROCEDURE tsvector_update_trigger(a, 'pg_catalog.english', t);
diff --git a/src/test/regress/sql/updatable_views.sql b/src/test/regress/sql/updatable_views.sql
index 4a60126ec90..0170040c098 100644
--- a/src/test/regress/sql/updatable_views.sql
+++ b/src/test/regress/sql/updatable_views.sql
@@ -125,7 +125,7 @@ INSERT INTO rw_view16 VALUES (3, 'Row 3', 3); -- should fail
INSERT INTO rw_view16 (a, b) VALUES (3, 'Row 3'); -- should be OK
UPDATE rw_view16 SET a=3, aa=-3 WHERE a=3; -- should fail
UPDATE rw_view16 SET aa=-3 WHERE a=3; -- should be OK
-SELECT * FROM base_tbl;
+SELECT * FROM base_tbl ORDER BY a;
DELETE FROM rw_view16 WHERE a=-3; -- should be OK
-- Read-only views
INSERT INTO ro_view17 VALUES (3, 'ROW 3');
--
2.50.1
[text/x-patch] v57-0003-Add-the-HOT-indexed-on-disk-format-inline-attr-b.patch (12.8K, ../[email protected]/4-v57-0003-Add-the-HOT-indexed-on-disk-format-inline-attr-b.patch)
download | inline diff:
From 7c791f4c67420f3f83ba50485116952280480713 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Wed, 17 Jun 2026 17:14:44 -0400
Subject: [PATCH v57 3/9] Add the HOT-indexed on-disk format: inline attr
bitmap and stubs
Define the on-disk representation a HOT-indexed update and its later
prune/collapse produce, ahead of the code that reads or writes it:
- HEAP_INDEXED_UPDATED (htup_details.h), the t_infomask2 bit marking a
heap-only tuple whose producing UPDATE also changed an indexed column; and
- access/hot_indexed.h, the inline fixed-size modified-attrs bitmap stored in
the tail of such a tuple, plus the xid-free "collapse-survivor stub" format
(HEAP_INDEXED_UPDATED with natts == 0, a forward link, and the segment's
bitmap) and the accessors both share.
README.HOT-INDEXED introduces the design and the relaxed classic-HOT
invariant; later commits document the eligibility, write, read, and
prune/collapse machinery in their own sections.
Co-authored-by: Greg Burd <[email protected]>
Co-authored-by: Nathan Bossart <[email protected]>
---
src/backend/access/heap/README.HOT-INDEXED | 56 ++++++
src/include/access/hot_indexed.h | 198 +++++++++++++++++++++
src/include/access/htup_details.h | 8 +-
3 files changed, 261 insertions(+), 1 deletion(-)
create mode 100644 src/backend/access/heap/README.HOT-INDEXED
create mode 100644 src/include/access/hot_indexed.h
diff --git a/src/backend/access/heap/README.HOT-INDEXED b/src/backend/access/heap/README.HOT-INDEXED
new file mode 100644
index 00000000000..4b701e42586
--- /dev/null
+++ b/src/backend/access/heap/README.HOT-INDEXED
@@ -0,0 +1,56 @@
+HOT-indexed updates (Selective Index Update, "HOT/SIU")
+=======================================================
+
+Classic HOT (see README.HOT) keeps an UPDATE off the indexes only when no
+indexed column changed: the new tuple is a heap-only tuple appended to the
+chain, and every index entry continues to resolve, via the same-page t_ctid
+chain, to a tuple whose indexed values still equal the entry's key.
+
+HOT-indexed (Selective Index Update, SIU) relaxes that to: an UPDATE may
+change indexed columns and still be a heap-only tuple on the same page,
+provided new index entries are inserted only into the indexes whose key
+attributes actually changed. The indexes whose attributes did not change are
+left untouched, which is where the write-amplification saving comes from.
+
+The price is that a chain may now contain tuples with different index keys, so
+an index entry for an old key can chain-resolve to a live tuple whose current
+key differs. Such an entry is STALE. The read side detects and drops stale
+entries by testing the heap's crossed-attribute bitmap against the index's
+key columns.
+
+This file documents the eligibility rules, the write path, the on-chain
+representation, the read-side staleness test, prune/collapse, vacuum reclamation,
+recovery, the logical-replication apply gating, and the statistics.
+
+
+The classic-HOT invariant we are relaxing
+------------------------------------------
+
+Classic HOT relies on two properties:
+
+ (a) every live index entry resolves, via a same-page t_ctid chain, to the
+ chain's root line pointer; and
+
+ (b) the indexed values of every tuple on the chain equal the key of every
+ index entry that reaches it.
+
+HOT-indexed keeps (a) but breaks (b): after a HOT-indexed update the chain
+holds tuples with different keys, and a new index entry is planted that points
+at the *specific* heap-only tuple whose key it matches -- not necessarily the
+root.
+
+
+The HOT-indexed invariant (the new contract)
+--------------------------------------------
+
+ An index entry points at the heap-only tuple version whose indexed key it
+ matched at insertion time. A chain walk that reaches a live tuple by
+ crossing a HOT-indexed hop *after* the entry's own target may be stale: if
+ the union of those crossed hops' modified attributes overlaps the index's
+ key columns, the entry is stale and the tuple is dropped from that scan.
+ The row is
+ re-supplied by the fresh entry that the same update inserted for the new
+ key.
+
+This is what makes dropping a stale entry safe: the live row is always
+reachable through exactly one non-stale entry per index.
diff --git a/src/include/access/hot_indexed.h b/src/include/access/hot_indexed.h
new file mode 100644
index 00000000000..295f589535e
--- /dev/null
+++ b/src/include/access/hot_indexed.h
@@ -0,0 +1,198 @@
+/*-------------------------------------------------------------------------
+ *
+ * hot_indexed.h
+ * Inline-trailing modified-attributes bitmap for HOT-indexed (HOT/SIU)
+ * tuples.
+ *
+ * A heap tuple produced by a HOT-indexed UPDATE has HEAP_INDEXED_UPDATED set
+ * in t_infomask2 and carries, appended after its normal attribute data, a
+ * fixed-size bitmap recording which heap attributes changed at this chain hop
+ * (relative to the prior chain member). Bit (attnum - 1) corresponds to user
+ * attribute attnum.
+ *
+ * The bitmap is ceil(natts / 8) bytes, where natts is the tuple's own
+ * attribute count at the time it was written (HeapTupleHeaderGetNatts for a
+ * live tuple; preserved separately for a stub, see below). Its length is not
+ * stored as such; it occupies the final HotIndexedBitmapBytes(natts) bytes of
+ * the line pointer's item and is located via ItemIdGetLength(), past the
+ * attribute data: routines that deform the tuple stop at natts and never see
+ * it.
+ *
+ * Because ADD COLUMN raises the relation's natts without rewriting existing
+ * tuples, a chain can hold tuples whose bitmaps were sized for different
+ * (smaller) natts than the relation has now. Consumers therefore size and
+ * locate a tuple's bitmap from that tuple's OWN write-time natts
+ * (HotIndexedTupleBitmapNatts), never from the relation's current natts.
+ * Bit positions are attribute based and identical across sizes, so a smaller
+ * bitmap simply ORs into the low bytes of a larger accumulator.
+ *
+ * Portions Copyright (c) 1996-2026, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/access/hot_indexed.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef HOT_INDEXED_H
+#define HOT_INDEXED_H
+
+#include "access/htup_details.h"
+
+/*
+ * Number of bytes in the trailing modified-attrs bitmap for a relation with
+ * natts user attributes (one bit per attribute, attnum - 1).
+ */
+static inline Size
+HotIndexedBitmapBytes(int natts)
+{
+ Assert(natts >= 0);
+ return (Size) ((natts + 7) / 8);
+}
+
+/*
+ * Read-only pointer to the trailing modified-attrs bitmap of a HOT-indexed
+ * tuple. item_len is ItemIdGetLength() of the tuple's line pointer; natts is
+ * the relation's number of attributes. The caller must have verified that
+ * the tuple has HEAP_INDEXED_UPDATED set.
+ */
+static inline const uint8 *
+HotIndexedGetModifiedBitmap(const HeapTupleHeaderData *htup,
+ Size item_len, int natts)
+{
+ return (const uint8 *) ((const char *) htup +
+ item_len - HotIndexedBitmapBytes(natts));
+}
+
+/* Writable variant, for OR-ing the union onto a surviving redirect target. */
+static inline uint8 *
+HotIndexedGetModifiedBitmapRW(HeapTupleHeaderData *htup,
+ Size item_len, int natts)
+{
+ return (uint8 *) ((char *) htup +
+ item_len - HotIndexedBitmapBytes(natts));
+}
+
+/* True if user attribute attnum (1-based) is set in the bitmap. */
+static inline bool
+HotIndexedAttrIsModified(const uint8 *bitmap, AttrNumber attnum)
+{
+ int bit = attnum - 1;
+
+ Assert(attnum >= 1);
+ return (bitmap[bit / 8] & (1 << (bit % 8))) != 0;
+}
+
+/* Set user attribute attnum (1-based) in the bitmap. */
+static inline void
+HotIndexedSetAttrModified(uint8 *bitmap, AttrNumber attnum)
+{
+ int bit = attnum - 1;
+
+ Assert(attnum >= 1);
+ bitmap[bit / 8] |= (uint8) (1 << (bit % 8));
+}
+
+/* OR the first nbytes of src into dst. dst must be at least nbytes long; it
+ * may be longer (sized for a larger natts) -- bit positions are attribute
+ * based and identical across sizes, so OR-ing only src's bytes is correct. */
+static inline void
+HotIndexedBitmapUnion(uint8 *dst, const uint8 *src, int src_natts)
+{
+ Size nbytes = HotIndexedBitmapBytes(src_natts);
+
+ for (Size i = 0; i < nbytes; i++)
+ dst[i] |= src[i];
+}
+
+/*
+ * Is every bit set in sub also set in super? sub is sized for sub_natts;
+ * super must be at least that long. Used by prune to decide a collapse
+ * survivor is reclaimable: when the attributes a dead member changed are all
+ * changed again by later hops, every index entry pointing at that member is
+ * superseded (stale), so it carries no live entry.
+ */
+static inline bool
+HotIndexedBitmapIsSubset(const uint8 *sub, const uint8 *super, int sub_natts)
+{
+ Size nbytes = HotIndexedBitmapBytes(sub_natts);
+
+ for (Size i = 0; i < nbytes; i++)
+ if ((sub[i] & ~super[i]) != 0)
+ return false;
+ return true;
+}
+
+/*
+ * Stub line pointers (collapse survivors).
+ *
+ * When prune collapses a dead HOT-indexed chain it cannot keep the dead key
+ * tuples as live data-bearing tuples (their XIDs would hold back
+ * relfrozenxid). Each preserved dead key tuple is instead converted to an
+ * xid-free "stub": an LP_NORMAL item whose HeapTupleHeader is frozen and
+ * marked not-a-real-tuple (HEAP_INDEXED_UPDATED set, HeapTupleHeaderGetNatts
+ * == 0), whose t_ctid.offnum forwards to the next key tuple on the page, and
+ * whose payload is the modified-attrs bitmap for the segment it represents --
+ * located, like a live tuple's inline bitmap, in the final
+ * HotIndexedBitmapBytes(natts) bytes of the item, so the same accessors work
+ * for both. Because the natts field is overwritten with the 0 sentinel, the
+ * stub's write-time natts (needed to size/locate that bitmap) is preserved in
+ * the otherwise-unused block-number half of t_ctid; HotIndexedStubGetForward
+ * reads only the offset half.
+ *
+ * A stub is signature-skipped (not visibility-skipped) by every consumer that
+ * walks LP_NORMAL items.
+ */
+static inline bool
+HotIndexedHeaderIsStub(const HeapTupleHeaderData *tup)
+{
+ return (tup->t_infomask2 & HEAP_INDEXED_UPDATED) != 0 &&
+ HeapTupleHeaderGetNatts(tup) == 0;
+}
+
+/* Offset of the next key tuple this stub forwards to (same page). */
+static inline OffsetNumber
+HotIndexedStubGetForward(const HeapTupleHeaderData *tup)
+{
+ return ItemPointerGetOffsetNumberNoCheck(&tup->t_ctid);
+}
+
+/*
+ * Bitmap sizing across the relation's lifetime.
+ *
+ * The trailing bitmap is ceil(natts / 8) bytes, where natts is the relation's
+ * attribute count *at the time the tuple was written*. ADD COLUMN raises the
+ * relation's natts without rewriting existing tuples, so a chain can hold
+ * tuples whose bitmaps were sized for different (smaller) natts than the
+ * relation has now. Every consumer must therefore size and locate a tuple's
+ * bitmap from that tuple's own write-time natts, never from the relation's
+ * current natts.
+ *
+ * For a live HOT-indexed tuple the write-time natts is HeapTupleHeaderGetNatts
+ * (a freshly formed UPDATE tuple stores the full relation natts, which is what
+ * heap_form_hot_indexed_tuple sized the bitmap with). A stub overwrites natts
+ * with 0 as its sentinel, so it preserves its write-time natts in the unused
+ * block-number half of t_ctid instead (the offset half holds the forward
+ * link).
+ */
+static inline void
+HotIndexedStubSetBitmapNatts(HeapTupleHeaderData *tup, int natts)
+{
+ BlockIdSet(&tup->t_ctid.ip_blkid, (BlockNumber) natts);
+}
+
+static inline int
+HotIndexedStubGetBitmapNatts(const HeapTupleHeaderData *tup)
+{
+ return (int) BlockIdGetBlockNumber(&tup->t_ctid.ip_blkid);
+}
+
+/* Write-time natts of any HOT-indexed item (live tuple or stub). */
+static inline int
+HotIndexedTupleBitmapNatts(const HeapTupleHeaderData *tup)
+{
+ if (HotIndexedHeaderIsStub(tup))
+ return HotIndexedStubGetBitmapNatts(tup);
+ return HeapTupleHeaderGetNatts(tup);
+}
+
+#endif /* HOT_INDEXED_H */
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 77a6c48fd71..7eb7f86f5ed 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -289,7 +289,13 @@ HEAP_XMAX_IS_KEYSHR_LOCKED(uint16 infomask)
* information stored in t_infomask2:
*/
#define HEAP_NATTS_MASK 0x07FF /* 11 bits for number of attributes */
-/* bits 0x1800 are available */
+#define HEAP_INDEXED_UPDATED 0x0800 /* HOT tuple produced by an UPDATE
+ * that also changed an indexed
+ * attribute (HOT/SIU); index scans
+ * that reach it via a chain recheck
+ * the arriving leaf key against the
+ * live tuple. */
+/* bit 0x1000 is available */
#define HEAP_KEYS_UPDATED 0x2000 /* tuple was updated and key cols
* modified, or tuple deleted */
#define HEAP_HOT_UPDATED 0x4000 /* tuple was HOT-updated */
--
2.50.1
[text/x-patch] v57-0004-Add-HOT-indexed-updates-selective-index-maintena.patch (229.8K, ../[email protected]/5-v57-0004-Add-HOT-indexed-updates-selective-index-maintena.patch)
download | inline diff:
From 31091ffca4dcebed4e9e114fbb41cf01a6ef3888 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Wed, 17 Jun 2026 17:35:17 -0400
Subject: [PATCH v57 4/9] Add HOT-indexed updates: selective index maintenance
and reads
Implement the HOT-indexed (Selective Index Update) feature on the foundation
laid by the executor's modified-attribute identification.
Eligibility: HeapUpdateHotAllowable returns a HeapUpdateIndexMode --
HEAP_UPDATE_ALL_INDEXES (not HOT; every index needs an entry), HEAP_UPDATE_HOT
(classic HOT; no index needs an entry), or HEAP_SELECTIVE_INDEX_UPDATE (HOT
chain, only the changed indexes maintained) -- computed from modified_idx_attrs
and the per-relation indexed-attribute set (RelationGetIndexedAttrs). An
UPDATE that changes a non-summarizing indexed attribute is
HEAP_SELECTIVE_INDEX_UPDATE unless it is forced to HEAP_UPDATE_ALL_INDEXES by
one of: every indexed attribute changed (nothing to skip), an attribute
referenced by an expression index changed (expression-aware maintenance is not
implemented yet), a system catalog, or the logical-replication apply gate (see
the apply-gating commit). Partial indexes, exclusion constraints, partitioned
tables, and non-btree access methods are all eligible -- the read path is
access-method agnostic and the predicate column is part of the index's
attribute set, so no carve-out is needed for them.
Write path: the table-AM update contract carries modified attributes IN/OUT as
a Bitmapset (on output the AM adds the whole-row sentinel,
TableTupleUpdateAllIndexes, to signal "every index needs an entry"), and
heap_update, for HEAP_SELECTIVE_INDEX_UPDATE, keeps the new version on the HOT
chain while ExecInsertIndexTuples maintains only the indexes whose attributes
changed. The new heap-only tuple records, in an inline bitmap in its tail, the
attributes that changed at its hop. Only the stored tuple carries the bitmap
and the HEAP_INDEXED_UPDATED flag; the caller's in-memory copy is left unmarked
so the flag never promises a trailing bitmap that is not present.
Read path: a chain walk to the live tuple unions the modified-attribute
bitmaps of every hop it crosses. The index-access layer treats that
crossed-attribute bitmap as the staleness authority: if it overlaps the
arriving index's key columns the entry is stale and is dropped, and the row is
re-supplied by the fresh entry the same update planted. The read path is
access-method agnostic and needs no value recheck or leaf key: it is correct
even when a key is cycled away and back, because the value-restoring update
planted a fresh entry whose walk crosses no later key-changing hop.
Unique checks are the one place that does compare values: _bt_check_unique
fetches the conflicting tuple under SnapshotDirty and, on a crossed-hop
arrival, compares the live tuple's current key against the arriving leaf with
the index's own ordering procedure (_bt_heap_keys_equal_leaf, BTORDER_PROC
under each column's collation). Using the opclass comparator -- not a bitwise
image comparison -- distinguishes a stale ancestor leaf from a genuinely live
duplicate (equal under the opclass even if not bitwise-identical) and, in the
in-flight window of a restoring update, routes the stale-ancestor hit into
_bt_doinsert's xwait so the duplicate is still caught. The comparison reads
plain key columns straight from the heap slot; it never evaluates an indexed
expression, because an UPDATE touching an expression-index attribute is
ineligible for HOT-indexed, so an expression index is never the one receiving
the fresh entry whose insert runs this check.
Co-authored-by: Greg Burd <[email protected]>
Co-authored-by: Nathan Bossart <[email protected]>
---
src/backend/access/heap/README.HOT | 34 +
src/backend/access/heap/README.HOT-INDEXED | 306 ++++++++
src/backend/access/heap/heapam.c | 471 ++++++++---
src/backend/access/heap/heapam_handler.c | 187 +++--
src/backend/access/heap/heapam_indexscan.c | 192 ++++-
src/backend/access/index/genam.c | 3 +
src/backend/access/index/indexam.c | 94 ++-
src/backend/access/nbtree/nbtinsert.c | 171 +++-
src/backend/access/nbtree/nbtree.c | 12 +-
src/backend/access/table/tableam.c | 24 +-
src/backend/catalog/indexing.c | 66 +-
src/backend/catalog/toasting.c | 2 -
src/backend/commands/repack.c | 26 +-
src/backend/executor/execIndexing.c | 329 ++++----
src/backend/executor/execReplication.c | 38 +-
src/backend/executor/nodeIndexonlyscan.c | 32 +
src/backend/executor/nodeIndexscan.c | 14 +
src/backend/executor/nodeModifyTable.c | 73 +-
src/backend/nodes/makefuncs.c | 2 -
src/backend/utils/activity/pgstat_relation.c | 21 +-
src/backend/utils/cache/relcache.c | 241 +++++-
src/include/access/amapi.h | 1 -
src/include/access/heapam.h | 50 +-
src/include/access/relscan.h | 60 ++
src/include/access/tableam.h | 71 +-
src/include/executor/executor.h | 5 +-
src/include/nodes/execnodes.h | 14 +-
src/include/pgstat.h | 31 +-
src/include/utils/rel.h | 11 +
src/include/utils/relcache.h | 17 +
src/test/regress/expected/hot_updates.out | 771 +++++++------------
src/test/regress/sql/hot_updates.sql | 603 +++++----------
src/tools/pgindent/typedefs.list | 2 +-
33 files changed, 2594 insertions(+), 1380 deletions(-)
diff --git a/src/backend/access/heap/README.HOT b/src/backend/access/heap/README.HOT
index 74e407f375a..7123656173c 100644
--- a/src/backend/access/heap/README.HOT
+++ b/src/backend/access/heap/README.HOT
@@ -156,6 +156,40 @@ all summarizing indexes. (Realistically, we only need to propagate the
update to the indexes that contain the updated values, but that is yet to
be implemented.)
+
+Per-Index Update Tracking
+-------------------------
+
+After the table AM performs the update, the executor determines which
+indexes need new entries using per-index tracking.
+
+The table AM communicates whether a HOT update occurred via the
+update_all_indexes boolean output of table_tuple_update(), together with the
+modified-attrs Bitmapset the caller passed in (attribute numbers encoded with
+FirstLowInvalidHeapAttributeNumber). When update_all_indexes is true the
+update was non-HOT and every index requires a new entry (the tuple has a new
+TID). When false the update was HOT: the caller consults modified_attrs with
+each index's own attributes to insert entries only into the indexes whose key
+attributes changed (a HOT-indexed update) or only the summarizing indexes (a
+classic HOT update that changed a summarized column), and skips the rest.
+
+The executor then calls ExecSetIndexUnchanged() to populate the per-index
+ii_IndexUnchanged flag on each IndexInfo. This flag indicates whether each
+index's key values are unchanged by the update. For non-HOT updates
+the flag is cleared on every index, so each gets a fresh entry at the
+new TID; the flag is never a skip on its own, just a hint to the
+index AM's aminsert for optimizations such as bottom-up deletion of
+logically equivalent duplicate entries.
+
+ExecInsertIndexTuples consults ii_IndexUnchanged to decide whether to
+skip a non-summarizing index during an UPDATE: if the index is marked
+unchanged, the HOT chain root's existing entry still points at the
+tuple, so no new entry is needed. For non-HOT updates the TID
+changed and ExecSetIndexUnchanged marks every index as changed,
+forcing each to receive a new entry. Summarizing indexes always get
+the opportunity to update their block-level summaries.
+
+
Abort Cases
-----------
diff --git a/src/backend/access/heap/README.HOT-INDEXED b/src/backend/access/heap/README.HOT-INDEXED
index 4b701e42586..5d4a2c7d66c 100644
--- a/src/backend/access/heap/README.HOT-INDEXED
+++ b/src/backend/access/heap/README.HOT-INDEXED
@@ -54,3 +54,309 @@ The HOT-indexed invariant (the new contract)
This is what makes dropping a stale entry safe: the live row is always
reachable through exactly one non-stale entry per index.
+
+
+Eligibility: HeapUpdateHotAllowable
+-----------------------------------
+
+The executor computes modified_idx_attrs (the indexed attributes this UPDATE
+changed, attribute numbers offset by FirstLowInvalidHeapAttributeNumber) and
+passes it to heap_update via table_tuple_update. HeapUpdateHotAllowable
+classifies the update:
+
+ HEAP_UPDATE_ALL_INDEXES
+ HOT is not permitted; the new tuple goes on a fresh TID and every
+ index gets a new entry.
+ HEAP_HEAP_ONLY_UPDATE
+ no non-summarizing indexed attribute changed, so no index needs a
+ new entry (classic HOT).
+ HEAP_SELECTIVE_INDEX_UPDATE
+ at least one non-summarizing index's attribute changed, but the
+ update may stay on the HOT chain and maintain only the changed
+ indexes selectively.
+
+A non-summarizing indexed attribute changing yields HEAP_SELECTIVE_INDEX_UPDATE
+unless one of these forces HEAP_UPDATE_ALL_INDEXES:
+
+ 1. The logical-replication apply path, gated per subscription (see "Logical
+ replication" below).
+ 2. An UPDATE touching an attribute referenced by an expression index
+ (selective maintenance of expression indexes is not implemented yet).
+ 3. An UPDATE that changes *every* indexed attribute: there is no index to
+ skip, so a plain non-HOT update is cheaper.
+
+System catalogs stay classic-HOT only: a catalog UPDATE that changes a
+non-summarizing indexed attribute falls back to HEAP_UPDATE_ALL_INDEXES, because
+catalog reads go through many paths not all proven safe against stale chain
+entries. This is the pre-HOT-indexed behaviour for such updates.
+
+INDEX_ATTR_BITMAP_INDEXED (cached in rd_indexedattr) is the set of columns
+referenced by non-summarizing indexes plus, folded in, the columns referenced
+only by summarizing indexes, so that a change to a summarizing-only column is
+seen by the modified-attribute comparison (its index is maintained via the
+classic-HOT summarizing path). Read-side staleness is filtered by the
+crossed-attribute bitmap, which is access-method agnostic, so a change to a
+column covered by any index is HOT-indexed regardless of the index's access
+method. Summarizing indexes (e.g. BRIN) keep no per-row leaf that can go
+stale and are maintained via the summarizing path.
+
+
+The write path
+--------------
+
+For HEAP_SELECTIVE_INDEX_UPDATE, heap_update:
+
+ - stores the new tuple as a heap-only tuple on the same page, linked into
+ the chain via t_ctid, exactly like classic HOT; and
+ - sets HEAP_INDEXED_UPDATED (t_infomask2 bit 0x0800) on the new tuple to
+ mark that the chain now carries differing keys.
+
+There is no separate on-page meta-item: the bit on the heap-only tuple is the
+entire on-disk footprint. As for classic HOT, if the new tuple does not fit
+on the page the update falls back to a non-HOT (new-page) update.
+
+The inline modified-attrs bitmap is ceil(natts/8) bytes, sized by the tuple's
+OWN attribute count at write time (HeapTupleHeaderGetNatts), not the relation's
+current natts. ADD COLUMN raises the relation's natts without rewriting
+existing tuples, so one chain can hold hops whose bitmaps were sized for
+different (smaller) natts; every consumer locates and sizes a hop's bitmap
+from that hop's own write-time natts (HotIndexedTupleBitmapNatts in
+access/hot_indexed.h). A collapse-survivor stub overwrites natts with its 0
+sentinel, so it preserves its write-time natts in the unused block-number half
+of t_ctid (the offset half is the forward link). Bit positions are attribute
+based and identical across sizes, so a smaller bitmap simply ORs into the low
+bytes of a larger crossed-attribute accumulator. DROP COLUMN keeps the attnum
+slot (it never renumbers), so existing bitmaps stay aligned.
+
+After the update, table_tuple_update reports update_all_indexes = false (the
+tuple is heap-only). The executor then maintains indexes selectively:
+ExecSetIndexUnchanged marks each index whose key attributes did not change as
+unchanged, and ExecInsertIndexTuples inserts a fresh entry only into the
+indexes that did change. Each such entry points at the new tuple's own TID.
+
+
+The chain and the two kinds of leaf entry
+------------------------------------------
+
+After a HOT-indexed update there are, for a changed index, two kinds of leaf
+entry reaching the chain:
+
+ - the pre-update entry for the OLD key, still pointing at an older chain
+ member (now stale once the walk crosses the HOT-indexed hop); and
+ - the fresh entry for the NEW key, pointing at the new heap-only tuple.
+
+Index build and REINDEX index a live HOT-indexed tuple under its OWN TID (not
+the chain root), so the freshly built entry has no hop after it and is never
+treated as stale.
+
+
+Read-side correctness: the crossed-attribute bitmap
+---------------------------------------------------
+
+heap_hot_search_buffer walks the chain from the entry's target to the live
+visible tuple. Each hop it crosses after the entry's own target -- a live
+HOT-indexed member, a collapse-survivor stub, or a collapsed (redirected)
+prefix -- contributes that hop's inline modified-attrs bitmap to a running
+union, IndexFetchTableData.xs_hot_indexed_crossed, and sets
+*hot_indexed_recheck to flag that the walk crossed at least one such hop.
+
+The index-access layer (index_fetch_heap) tests that union against the
+arriving index's key columns. Any overlap means a crossed hop changed one of
+this index's inputs, so the entry's stored key no longer matches the live
+tuple: IndexScanDesc.xs_hot_indexed_stale is set, and IndexScan,
+IndexOnlyScan, CLUSTER, and the logical-replication replica-identity lookups
+drop the tuple. If the union is disjoint from the index's key columns, none
+of the index's inputs changed across the chain, so the entry is current and
+the row is returned.
+
+The union is complete: every crossed live hop and stub contributes its
+bitmap, and chain collapse only ever reclaims a member whose attributes are a
+subset of the surviving later hops (see "Prune and chain collapse"), so a
+reader crossing the survivors still sees every collapsed hop's attributes.
+Disjointness therefore reliably means the entry is current.
+
+This needs no value comparison and no leaf key, so it serves equality, range,
+and inequality scans uniformly, works for any access method whose columns are
+eligible for HOT-indexed updates, and is correct even when a key is cycled
+away and back (X -> Y -> X): the update that restored the value planted a
+fresh entry pointing at its own live tuple, whose walk crosses no later
+key-changing hop, so that entry uniquely returns the row while the stale
+ancestor entry -- whose walk does cross the changing hops -- is dropped.
+
+The read mechanism never reconstructs or compares an index key, so it needs no
+per-access-method support. (nbtree keeps an internal leaf-key comparison,
+_bt_heap_keys_equal_leaf, used only by _bt_check_unique to tell a stale chain
+entry from a live duplicate during a unique insert; it is not part of the read
+path.)
+
+Unique checks. _bt_check_unique fetches the conflicting tuple under
+SnapshotDirty and, when the chain walk crossed a HOT-indexed hop, compares the
+live tuple's current key against the arriving leaf with the index's own
+ordering procedure (_bt_heap_keys_equal_leaf, using BTORDER_PROC under each
+column's collation). This recheck is reached only for an index receiving a
+fresh entry during a HOT-indexed update; HeapUpdateHotAllowable disqualifies
+any UPDATE that touches an expression-index attribute, so the index here never
+has an expression key column (every key column is a plain attribute), and the
+comparison reads attribute values straight from the heap slot -- no expression
+evaluation or executor state is needed. Using the opclass comparator -- not a
+bitwise image comparison -- means a key
+that was cycled away and back (X -> Y -> X) does not raise a spurious
+duplicate against its own stale leaf, while a genuinely live duplicate (equal
+under the opclass even if not bitwise-identical, e.g. numeric 1.0 vs 1.00) is
+still detected. (Appendix A motivates this recheck in detail.)
+
+
+Appendices
+----------
+
+Appendix A: Why the unique-check path needs a value comparison at all
+---------------------------------------------------------------------
+
+This is the one place HOT-indexed does compare a key value, even though the
+read path deliberately avoids one. The rest of this appendix explains why the
+comparison is needed, why it must use the opclass comparator rather than a
+bitwise one, and why the ABA case is what forces the issue.
+
+1. The setup: why a unique insert can even reach a stale leaf
+
+Under classic HOT, an index has exactly one leaf entry per logical row, and
+every leaf entry's key matches the live tuple it chain-resolves to. So
+_bt_check_unique can trust: "if I find a leaf whose key equals my new key, and
+it resolves to a live tuple, that's a genuine duplicate."
+
+HOT/SIU breaks that one-to-one correspondence. A HOT-indexed UPDATE that
+changes column a from X to Y:
+ - inserts a fresh leaf entry (Y -> new tuple) into idx_a, and
+ - leaves the old leaf entry (X -> old chain root) in place.
+
+That old (X -> root) entry is now stale: it still chain-resolves to a live
+tuple, but that live tuple's current a is Y, not X. The read path handles this
+with the crossed-attribute bitmap (no value comparison needed): if the walk
+from the entry's target to the live tuple crosses a hop that changed a, the
+entry is stale and dropped.
+
+When you now INSERT a row with a = X, _bt_check_unique scans idx_a for key X
+and finds that stale (X -> root) leaf. It must decide: is this a real
+conflict?
+
+2. Why the read-path bitmap is not sufficient here
+
+The read path's logic is: "this entry crossed a hop that changed a => stale =>
+drop it, the fresh entry will supply the row." For scans that's correct and
+complete, because every live row has exactly one non-stale entry that
+re-supplies it.
+
+But a unique check is asking a different question. It is not "should I return
+this row?" -- it is "does the live tuple this entry resolves to conflict with
+the key I'm inserting?" The bitmap can only tell you "an indexed attribute
+changed somewhere on the chain." It cannot tell you what the live value is
+now, and that is exactly what you need to know to detect a duplicate.
+
+This is the crux of the ABA problem. Consider:
+
+ INSERT (a=10) LP[1] a=10 (root)
+ UPDATE a=11 (HOT-indexed) LP[2] a=11 bitmap {a}, leaf (11)->LP[2]
+ UPDATE a=10 (HOT-indexed) LP[3] a=10 bitmap {a}, leaf (10)->LP[3], live
+
+idx_a now has leaves (10)->LP[1] [stale ancestor], (11)->LP[2] [stale], and
+(10)->LP[3] [fresh, live].
+
+Now INSERT (a=10), a genuine duplicate of the live row. _bt_check_unique scans
+for key 10 and finds the (10)->LP[1] stale ancestor entry. The chain walk from
+LP[1] to the live tuple LP[3] crosses hops that changed a (10->11, then
+11->10), so the bitmap says "stale." If the unique check trusted the bitmap
+alone it would skip (10)->LP[1] as stale and miss the real duplicate. The
+bitmap is fooled because a changed (so the bit is set) even though it changed
+back to the same value: "an attribute changed on the chain" is not "the live
+value differs from this leaf's key." Under ABA they diverge.
+
+The sharper case is concurrency. While the restoring UPDATE (a: 11 -> 10) is
+in flight, it has written its new heap tuple but not yet inserted the fresh
+(10)->LP[3] leaf. A concurrent INSERT (a=10) running its _bt_check_unique scan
+in that window sees only the stale (10)->LP[1] ancestor. The value recheck
+below makes that hit resolve to xwait on the in-flight updater (via
+_bt_doinsert's wait-and-recheck), so the inserter re-checks after the updater
+commits and finds the conflict. A bitmap-only verdict would skip the ancestor
+before reaching the xwait logic and admit a duplicate -- which is why the
+recheck is a correctness requirement, not merely an optimization.
+
+3. Why a value comparison fixes it, and why it must be the opclass comparator
+
+So the unique path needs to look at the actual live value, not just "did
+something change." _bt_check_unique fetches the conflicting tuple under
+SnapshotDirty and, when hi_recheck says a HOT-indexed hop was crossed, calls
+_bt_heap_keys_equal_leaf to compare the live tuple's current key against the
+arriving leaf's stored key:
+
+ - live key equals the leaf key -> genuine duplicate (or an in-flight conflict
+ reached as xwait) -- correct: ABA back to X is a real conflict with a new X.
+ - live key differs -> the leaf is truly stale -> skip it (the fresh entry
+ handles the real row).
+
+Which equality? Two candidates:
+
+Bitwise/image comparison (datum_image_eq) compares raw bytes. That is wrong
+for unique checking in the dangerous direction. Uniqueness in PostgreSQL is
+defined by the index opclass's equality operator, not byte identity, and many
+types have values equal under the opclass but byte-distinct:
+ - numeric: 1.0 and 1.00 are opclass-equal, different on-disk bytes.
+ - float8: -0.0 and +0.0 are equal, different bit patterns.
+ - text/citext under a nondeterministic collation: canonically-equivalent
+ strings that are not byte-identical.
+
+A bitwise comparison would conclude "not equal => stale => skip" for a live
+1.00 versus an inserted 1.0 and miss a genuine violation -- a correctness hole
+as bad as the ABA one.
+
+So _bt_heap_keys_equal_leaf uses the index's own BTORDER_PROC (btree support
+function 1) under each key column's collation, the same machinery _bt_compare
+and _bt_mkscankey use to define equality for the index. A zero result means
+"equal as the index defines equality," which is precisely the unique-violation
+condition, and the verdict agrees with the index's own notion of uniqueness in
+both directions.
+
+4. Why no expression evaluation is needed
+
+_bt_heap_keys_equal_leaf reads each key column straight from the heap slot
+(slot_getattr) and compares it to the leaf datum; it does not evaluate indexed
+expressions and needs no executor state. That is sufficient because the
+recheck is only ever reached for an index receiving a fresh entry during a
+HOT-indexed update, and HeapUpdateHotAllowable disqualifies any UPDATE that
+touches an attribute referenced by an expression index
+(INDEX_ATTR_BITMAP_EXPRESSION captures every such attribute). So a HOT-indexed
+chain never has a crossed hop affecting an expression index, the index reaching
+the recheck never has an expression key column (every indkey is a real
+attribute number), and there is nothing to evaluate. If selective maintenance
+of expression indexes is implemented in the future, this is where an
+expression-evaluating comparison (e.g. FormIndexDatum) would be reintroduced.
+
+5. Why the asymmetry (bitmap on read, value recheck on unique) is intentional
+
+It looks like two different answers to the same question, but the questions
+differ:
+
+ - Read/scan path: "should this row be returned?" A stale entry is redundant
+ (the fresh entry supplies the row), so the conservative bitmap verdict is
+ sufficient -- worst case under ABA you drop a redundant entry and the fresh
+ one still returns the row. No value comparison, so reads stay
+ access-method-agnostic and cheap.
+ - Unique-check path: "is this a conflict?" A wrong "stale" verdict here does
+ not just drop a redundant entry; it silently admits a duplicate, corrupting
+ the constraint. It cannot tolerate the bitmap's false "stale" under ABA and
+ must consult the live value (or wait on an in-flight updater) via the
+ opclass comparator.
+
+The bitmap is a filter (a necessary condition: "could be stale"); the opclass
+recheck is the authority (the sufficient condition: "is the live key actually
+different, or is a conflicting update in flight"). The unique path layers the
+authority on top of the filter precisely because its error mode is
+unforgiving.
+
+In one sentence: the unique check compares the live tuple's current key to the
+arriving leaf with the index's own equality (not bytes) because the
+crossed-attribute bitmap can only say "something changed" -- true under an
+X->Y->X cycle even though the value is back to X -- and only an opclass-correct
+value comparison (which also routes an in-flight restoring update to xwait) can
+both recognize the cycled-back value as a genuine duplicate and catch
+duplicates that are opclass-equal but not byte-identical, either of which a
+bitmap or a bitwise comparison would get wrong.
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 5b059a5acef..3bfac8c0b18 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -34,6 +34,7 @@
#include "access/heapam.h"
#include "access/heaptoast.h"
#include "access/hio.h"
+#include "access/hot_indexed.h"
#include "access/multixact.h"
#include "access/subtrans.h"
#include "access/syncscan.h"
@@ -44,13 +45,14 @@
#include "access/xloginsert.h"
#include "catalog/pg_database.h"
#include "catalog/pg_database_d.h"
+#include "catalog/pg_subscription.h"
#include "commands/vacuum.h"
#include "executor/instrument_node.h"
#include "executor/tuptable.h"
#include "nodes/lockoptions.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
-#include "storage/buf.h"
+#include "replication/logicalworker.h"
#include "storage/lmgr.h"
#include "storage/predicate.h"
#include "storage/proc.h"
@@ -78,6 +80,8 @@ static void check_inplace_rel_lock(HeapTuple oldtup);
#endif
static Bitmapset *HeapUpdateModifiedIdxAttrs(Relation relation,
HeapTuple oldtup, HeapTuple newtup);
+static HeapTuple heap_form_hot_indexed_tuple(HeapTuple tup, int relnatts,
+ const Bitmapset *modified_idx_attrs);
static bool heap_acquire_tuplock(Relation relation, const ItemPointerData *tid,
LockTupleMode mode, LockWaitPolicy wait_policy,
bool *have_tuple_lock);
@@ -2109,9 +2113,28 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
* If this is the single and first tuple on page, we can reinit the
* page instead of restoring the whole thing. Set flag, and hide
* buffer references from XLogInsert.
+ *
+ * Also require that the page's tuple area contains nothing other than
+ * this tuple. Vacuum's lp_truncate_only second pass
+ * (PRUNE_VACUUM_CLEANUP) does not call PageRepairFragmentation, so a
+ * page can legitimately end up with one LP_UNUSED slot at offset 1
+ * plus orphan tuple bytes left over from the previous lifetime. If
+ * heap_insert reuses that LP_UNUSED slot, primary's page keeps the
+ * orphan bytes while a standby replaying INSERT+INIT zeroes them.
+ * Emitting INSERT+INIT in that case trips wal_consistency_checking.
+ * Falling back to a regular INSERT (with the FPI on first touch after
+ * a checkpoint) keeps replay byte-identical without sacrificing crash
+ * safety.
+ *
+ * NOTE: heap_multi_insert() does not need this extra check: it uses the
+ * simpler starting_with_empty_page (PageGetMaxOffsetNumber(page) == 0)
+ * gate for its own INIT_PAGE decision, because by construction it is
+ * never handed a page with leftover orphan bytes from a prior lifetime.
*/
if (ItemPointerGetOffsetNumber(&(heaptup->t_self)) == FirstOffsetNumber &&
- PageGetMaxOffsetNumber(page) == FirstOffsetNumber)
+ PageGetMaxOffsetNumber(page) == FirstOffsetNumber &&
+ ((PageHeader) page)->pd_upper ==
+ ((PageHeader) page)->pd_special - MAXALIGN(heaptup->t_len))
{
info |= XLOG_HEAP_INIT_PAGE;
bufflags |= REGBUF_WILL_INIT;
@@ -3202,9 +3225,11 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
*/
TM_Result
heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
- CommandId cid, uint32 options pg_attribute_unused(), Snapshot crosscheck, bool wait,
+ CommandId cid, uint32 options pg_attribute_unused(),
+ Snapshot crosscheck, bool wait,
TM_FailureData *tmfd, const LockTupleMode lockmode,
- const Bitmapset *modified_idx_attrs, const bool hot_allowed)
+ const Bitmapset *modified_idx_attrs,
+ HeapUpdateIndexMode hot_mode)
{
TM_Result result;
TransactionId xid = GetCurrentTransactionId();
@@ -3230,6 +3255,9 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
bool have_tuple_lock = false;
bool iscombo;
bool use_hot_update = false;
+ bool hot_indexed = false; /* HOT-indexed update (modified an
+ * indexed attr but stayed HOT) */
+ Size hi_bmbytes = 0; /* trailing modified-attrs bitmap size, if any */
bool key_intact;
bool all_visible_cleared = false;
bool all_visible_cleared_new = false;
@@ -3794,6 +3822,38 @@ l2:
newtupsize = MAXALIGN(newtup->t_len);
+ /*
+ * Keep HOT-indexed (SIU) chains uniform. HeapUpdateHotAllowable returns
+ * HEAP_HEAP_ONLY_UPDATE whenever this update modifies no indexed
+ * attribute. But if the tuple being updated is already a HOT-indexed
+ * chain member (it carries HEAP_INDEXED_UPDATED), emitting a classic-HOT
+ * version would splice a non-HEAP_INDEXED_UPDATED tuple into the chain.
+ * The prune/collapse machinery forwards only HEAP_INDEXED_UPDATED members
+ * through bridges, so such a classic-HOT version, once it dies mid
+ * collapsed-chain, has no handler and trips the "not linked to from any
+ * HOT chain" error. Promote to HEAP_SELECTIVE_INDEX_UPDATE instead: with an
+ * empty modified-attrs set the new version carries HEAP_INDEXED_UPDATED
+ * and an empty inline-trailing bitmap, inserts into no index (nothing
+ * changed), and keeps every chain member uniform. Catalog relations are
+ * classic-HOT only and never carry HEAP_INDEXED_UPDATED, so this never
+ * fires for them.
+ */
+ if (hot_mode == HEAP_HEAP_ONLY_UPDATE &&
+ (oldtup.t_data->t_infomask2 & HEAP_INDEXED_UPDATED) != 0)
+ hot_mode = HEAP_SELECTIVE_INDEX_UPDATE;
+
+ /*
+ * A HOT-indexed update appends a fixed-size inline-trailing
+ * modified-attrs bitmap to the new tuple (see access/hot_indexed.h).
+ * Reserve room for it in the page-fit calculation now, while we still
+ * might take the same-page HOT path; if the update later drops to non-HOT
+ * (the tuple does not fit on the page) it is stored without the bitmap and
+ * the reservation is simply conservative.
+ */
+ if (hot_mode == HEAP_SELECTIVE_INDEX_UPDATE)
+ hi_bmbytes = HotIndexedBitmapBytes(RelationGetNumberOfAttributes(relation));
+ newtupsize = MAXALIGN(newtup->t_len + hi_bmbytes);
+
if (need_toast || newtupsize > pagefree)
{
TransactionId xmax_lock_old_tuple;
@@ -3891,7 +3951,7 @@ l2:
{
/* Note we always use WAL and FSM during updates */
heaptup = heap_toast_insert_or_update(relation, newtup, &oldtup, 0);
- newtupsize = MAXALIGN(heaptup->t_len);
+ newtupsize = MAXALIGN(heaptup->t_len + hi_bmbytes);
}
else
heaptup = newtup;
@@ -3992,10 +4052,10 @@ l2:
{
/*
* Since the new tuple is going into the same page, we might be able
- * to do a HOT update. Check if any of the index columns have been
- * changed.
+ * to do a HOT update. Check if HeapUpdateHotAllowable() has
+ * sanctioned it (HEAP_HEAP_ONLY_UPDATE or HEAP_SELECTIVE_INDEX_UPDATE).
*/
- if (hot_allowed)
+ if (hot_mode != HEAP_UPDATE_ALL_INDEXES)
use_hot_update = true;
}
else
@@ -4004,6 +4064,27 @@ l2:
PageSetFull(page);
}
+ /*
+ * For a same-page HOT-indexed update, replace heaptup with a copy that
+ * carries the inline-trailing modified-attrs bitmap (and
+ * HEAP_INDEXED_UPDATED). Done here, outside the critical section,
+ * because it allocates; the bitmap's size was reserved in newtupsize
+ * above. Only the stored tuple (heaptup) gets the bitmap and the flag;
+ * the caller's newtup must NOT be marked HEAP_INDEXED_UPDATED, because it
+ * has no trailing bitmap -- see the flag handling below.
+ */
+ if (use_hot_update && hot_mode == HEAP_SELECTIVE_INDEX_UPDATE)
+ {
+ HeapTuple ext;
+
+ ext = heap_form_hot_indexed_tuple(heaptup,
+ RelationGetNumberOfAttributes(relation),
+ modified_idx_attrs);
+ if (heaptup != newtup)
+ heap_freetuple(heaptup);
+ heaptup = ext;
+ }
+
/*
* Compute replica identity tuple before entering the critical section so
* we don't PANIC upon a memory allocation failure.
@@ -4040,6 +4121,29 @@ l2:
HeapTupleSetHeapOnly(heaptup);
/* Mark the caller's copy too, in case different from heaptup */
HeapTupleSetHeapOnly(newtup);
+
+ /*
+ * For a HOT-indexed update, the new live tuple carries
+ * HEAP_INDEXED_UPDATED so index scans walking the chain know it is a
+ * HOT-indexed hop carrying an inline-trailing modified-attrs bitmap.
+ *
+ * Set the flag only on heaptup, the version actually stored on the
+ * page: heaptup carries the trailing bitmap, so the flag's promise (a
+ * bitmap occupies the final HotIndexedBitmapBytes(natts) bytes of the
+ * item) holds. The caller's newtup is a separate in-memory tuple
+ * whose t_len does not include the bitmap; marking it
+ * HEAP_INDEXED_UPDATED would assert a trailing bitmap that is not
+ * there, so any later reader using ItemIdGetLength()-relative access
+ * would misread attribute data as the bitmap. We therefore leave
+ * newtup's flag clear. Nothing reads the modified-attrs bitmap off an
+ * in-memory tuple; every consumer reads it from the page via the line
+ * pointer's length.
+ */
+ if (hot_mode == HEAP_SELECTIVE_INDEX_UPDATE)
+ {
+ heaptup->t_data->t_infomask2 |= HEAP_INDEXED_UPDATED;
+ hot_indexed = true;
+ }
}
else
{
@@ -4144,7 +4248,8 @@ l2:
if (have_tuple_lock)
UnlockTupleTuplock(relation, &(oldtup.t_self), lockmode);
- pgstat_count_heap_update(relation, use_hot_update, newbuf != buffer);
+ pgstat_count_heap_update(relation, use_hot_update, hot_indexed,
+ newbuf != buffer);
/*
* If heaptup is a private copy, release it. Don't forget to copy t_self
@@ -4290,7 +4395,7 @@ check_inplace_rel_lock(HeapTuple oldtup)
/*
* Check if the specified attribute's values are the same. Subroutine for
- * HeapDetermineColumnsInfo.
+ * HeapUpdateModifiedIdxAttrs.
*/
static bool
heap_attr_equals(TupleDesc tupdesc, int attrnum, Datum value1, Datum value2,
@@ -4334,63 +4439,103 @@ heap_attr_equals(TupleDesc tupdesc, int attrnum, Datum value1, Datum value2,
}
/*
- * HOT updates are possible when either: a) there are no modified indexed
- * attributes, or b) the modified attributes are all on summarizing indexes.
- * Later, in heap_update(), we can choose to perform a HOT update if there is
- * space on the page for the new tuple and the following code has determined
- * that HOT is allowed.
+ * HeapUpdateHotAllowable --
+ *
+ * Classify an UPDATE for HOT eligibility from the set of indexed attributes
+ * it changed (modified_idx_attrs, computed by the executor):
+ *
+ * HEAP_UPDATE_ALL_INDEXES HOT is not permitted; the new tuple goes on a
+ * fresh TID and every index gets a new entry.
+ * HEAP_HEAP_ONLY_UPDATE Classic HOT: no non-summarizing indexed
+ * attribute changed, so no index needs a new
+ * entry and the new tuple joins the chain via a
+ * t_ctid forward link.
+ * HEAP_SELECTIVE_INDEX_UPDATE HOT with selective index update: at least one
+ * non-summarizing index's attribute changed, but
+ * the new tuple can still join the HOT chain on
+ * the same page; only the indexes whose
+ * attributes changed receive a new entry.
+ *
+ * This routine only classifies the update; heap_update() performs it and may
+ * still fall back to a non-HOT update when the new tuple does not fit on the
+ * page, exactly as for classic HOT.
*/
-bool
-HeapUpdateHotAllowable(Relation relation, const Bitmapset *modified_idx_attrs,
- bool *summarized_only)
+HeapUpdateIndexMode
+HeapUpdateHotAllowable(Relation relation, const Bitmapset *modified_idx_attrs)
{
- bool hot_allowed;
+ const Bitmapset *all_idx_attrs;
/*
- * Let's be optimistic and start off by assuming the best case, no indexes
- * need updating and HOT is allowable.
+ * Case (a): no indexed attribute was modified -> classic HOT.
*/
- hot_allowed = true;
- *summarized_only = false;
+ if (bms_is_empty(modified_idx_attrs))
+ return HEAP_HEAP_ONLY_UPDATE;
/*
- * Check for case (a); when there are no modified index attributes HOT is
- * allowed.
+ * Case (b): at least one indexed attribute changed. If all of them are
+ * used only by summarizing indexes, we can still take the classic HOT
+ * path -- the summarizing index AM gets a new entry via aminsert and no
+ * non-summarizing index needs to change.
*/
- if (bms_is_empty(modified_idx_attrs))
- hot_allowed = true;
- else
- {
- Bitmapset *sum_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_SUMMARIZED);
+ if (bms_is_subset(modified_idx_attrs, RelationGetIndexAttrBitmapNoCopy(relation,
+ INDEX_ATTR_BITMAP_SUMMARIZED)))
+ return HEAP_HEAP_ONLY_UPDATE;
- /*
- * At least one index attribute was modified, but is this case (b)
- * where all the modified index attributes are only used by
- * summarizing indexes? If it is, then we need to update those
- * indexes, but this update can still be considered heap-only (HOT)
- * and avoid updating any non-summarizing indexes on the relation.
- */
- if (bms_is_subset(modified_idx_attrs, sum_attrs))
- {
- hot_allowed = true;
- *summarized_only = true;
- }
- else
- {
- /*
- * Now we know a) one or more indexed attributes were modified
- * (changed value, not just referenced within the UPDATE) and that
- * b) at least one of those attributes is used by a
- * non-summarizing index. HOT is not allowed.
- */
- hot_allowed = false;
- }
+ /*
+ * A non-summarizing indexed attribute changed. HOT-indexed is supported
+ * whenever the relation can tolerate extra index entries in a chain whose
+ * per-chain-member keys may differ. The logical-replication apply path
+ * is gated above by hot_indexed_on_apply. The remaining
+ * HEAP_UPDATE_ALL_INDEXES fallbacks are:
+ *
+ * - An UPDATE that modifies an attribute referenced by an expression
+ * index. Selective maintenance of an expression index requires
+ * evaluating the indexed expression to decide whether its value (hence
+ * its entry) changed; that expression-aware path is not implemented yet,
+ * so such an update falls back to non-HOT. Updates that do not touch any
+ * expression-index attribute stay eligible.
+ *
+ * - An UPDATE that modifies every indexed attribute of the relation.
+ * HOT-indexed only pays off when it can skip maintaining at least one
+ * index whose key did not change; if all indexed attributes changed there
+ * is nothing to skip, so a plain non-HOT update is cheaper (it avoids the
+ * chain-walk and bitmap-overlap overhead).
+ */
+ all_idx_attrs = RelationGetIndexAttrBitmapNoCopy(relation,
+ INDEX_ATTR_BITMAP_INDEXED);
- bms_free(sum_attrs);
- }
+ /*
+ * System catalogs keep classic HOT (an UPDATE touching no non-summarizing
+ * indexed attribute already returned HEAP_HEAP_ONLY_UPDATE above), but do
+ * NOT take the HOT-indexed path: catalog reads go through many code paths
+ * (systable index scans, SnapshotDirty unique checks, seqscans in
+ * orderings the chain-walk dedup does not cover) that are not all proven
+ * safe against stale chain entries. Falling back to a non-HOT update
+ * here is exactly the pre-HOT-indexed behaviour for such catalog updates.
+ */
+ if (IsCatalogRelation(relation))
+ return HEAP_UPDATE_ALL_INDEXES;
- return hot_allowed;
+ /*
+ * Disqualify when the update touches an attribute referenced by an
+ * expression index (see case 1 above). Updates that leave every
+ * expression-index attribute unchanged remain eligible.
+ */
+ if (bms_overlap(modified_idx_attrs,
+ RelationGetIndexAttrBitmapNoCopy(relation,
+ INDEX_ATTR_BITMAP_EXPRESSION)))
+ return HEAP_UPDATE_ALL_INDEXES;
+
+ /*
+ * If every indexed attribute changed, a HOT-selective update could not
+ * skip any index -- each index needs a fresh entry anyway -- so it would
+ * pay the HOT/SIU chain-walk and bitmap-overlap overhead for no saved
+ * index maintenance. Fall back to a plain non-HOT update in that case.
+ */
+ if (bms_is_subset(all_idx_attrs, modified_idx_attrs))
+ return HEAP_UPDATE_ALL_INDEXES;
+
+ return HEAP_SELECTIVE_INDEX_UPDATE;
}
/*
@@ -4402,15 +4547,33 @@ LockTupleMode
HeapUpdateDetermineLockmode(Relation relation, const Bitmapset *modified_idx_attrs)
{
LockTupleMode lockmode = LockTupleExclusive;
+ const Bitmapset *key_attrs;
+
+ /*
+ * Common fast path: when no indexed attribute changed (e.g. pgbench-style
+ * "UPDATE t SET non_idx_col = ..." or the wide_0 "UPDATE t SET id = id"
+ * workload after the executor's fast path in ExecUpdateModifiedIdxAttrs),
+ * modified_idx_attrs is empty and a key column cannot have changed. Skip
+ * the relcache lookup and return the weaker lock immediately. At high
+ * TPS this avoids a per-UPDATE RelationGetIndexAttrBitmap call (and its
+ * bms_copy) on the KEY bitmap.
+ */
+ if (bms_is_empty(modified_idx_attrs))
+ return LockTupleNoKeyExclusive;
- Bitmapset *key_attrs = RelationGetIndexAttrBitmap(relation,
- INDEX_ATTR_BITMAP_KEY);
+ /*
+ * Borrow the cached bitmap rather than copying it; we only test overlap
+ * and never mutate or free key_attrs. HeapUpdateDetermineLockmode runs
+ * without buffer locks but the relcache entry is pinned by the caller's
+ * lock on the relation, and we touch nothing between fetch and the
+ * bms_overlap that could trigger a relcache invalidation.
+ */
+ key_attrs = RelationGetIndexAttrBitmapNoCopy(relation,
+ INDEX_ATTR_BITMAP_KEY);
if (!bms_overlap(modified_idx_attrs, key_attrs))
lockmode = LockTupleNoKeyExclusive;
- bms_free(key_attrs);
-
return lockmode;
}
@@ -4495,6 +4658,71 @@ HeapUpdateModifiedIdxAttrs(Relation relation, HeapTuple oldtup, HeapTuple newtup
return modified_idx_attrs;
}
+/*
+ * heap_form_hot_indexed_tuple
+ *
+ * Return a newly palloc'd copy of tup that carries the fixed-size
+ * inline-trailing modified-attributes bitmap (see access/hot_indexed.h),
+ * with HEAP_INDEXED_UPDATED set. The bitmap records the user attributes in
+ * modified_idx_attrs (the indexed attributes this UPDATE changed, using the
+ * FirstLowInvalidHeapAttributeNumber offset convention); an empty set yields
+ * an all-zero bitmap, which is correct for the chain-uniformity promotion of
+ * a classic-HOT update on an already-HOT-indexed chain.
+ *
+ * The bitmap occupies the final HotIndexedBitmapBytes(natts) bytes of the
+ * tuple, where natts is the tuple's own attribute count
+ * (HeapTupleHeaderGetNatts) -- which a reader recovers from the stored tuple,
+ * so the bitmap stays locatable even after the relation's natts later grows
+ * via ADD COLUMN. For a freshly formed UPDATE tuple this equals the
+ * relation's current natts; we assert that to catch any future divergence.
+ * The bitmap sits past the attribute data, so heap_deform_tuple never sees
+ * it. The caller must have reserved room for the extra bytes in the page-fit
+ * calculation, and must free the returned tuple.
+ */
+static HeapTuple
+heap_form_hot_indexed_tuple(HeapTuple tup, int relnatts,
+ const Bitmapset *modified_idx_attrs)
+{
+ int natts = HeapTupleHeaderGetNatts(tup->t_data);
+ Size bmbytes;
+ Size newlen;
+ HeapTuple newtuple;
+ uint8 *bitmap;
+ int x = -1;
+
+ /*
+ * The bitmap is sized and located by the tuple's own natts; a freshly
+ * formed UPDATE tuple carries the full relation natts. If these ever
+ * diverge the page-fit reservation (made with relnatts) and the actual
+ * bitmap size would disagree.
+ */
+ Assert(natts == relnatts);
+ bmbytes = HotIndexedBitmapBytes(natts);
+ newlen = tup->t_len + bmbytes;
+
+ newtuple = (HeapTuple) palloc0(HEAPTUPLESIZE + newlen);
+ newtuple->t_len = newlen;
+ newtuple->t_self = tup->t_self;
+ newtuple->t_tableOid = tup->t_tableOid;
+ newtuple->t_data = (HeapTupleHeader) ((char *) newtuple + HEAPTUPLESIZE);
+
+ /* copy the original tuple; the trailing bitmap bytes stay zero */
+ memcpy(newtuple->t_data, tup->t_data, tup->t_len);
+ newtuple->t_data->t_infomask2 |= HEAP_INDEXED_UPDATED;
+
+ bitmap = HotIndexedGetModifiedBitmapRW(newtuple->t_data, newlen, natts);
+ while ((x = bms_next_member(modified_idx_attrs, x)) >= 0)
+ {
+ AttrNumber attnum = x + FirstLowInvalidHeapAttributeNumber;
+
+ /* only user attributes can be modified-and-indexed */
+ if (attnum >= 1)
+ HotIndexedSetAttrModified(bitmap, attnum);
+ }
+
+ return newtuple;
+}
+
/*
* simple_heap_update - replace a tuple
*
@@ -4505,7 +4733,7 @@ HeapUpdateModifiedIdxAttrs(Relation relation, HeapTuple oldtup, HeapTuple newtup
*/
void
simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup,
- TU_UpdateIndexes *update_indexes)
+ bool *update_all_indexes)
{
TM_Result result;
TM_FailureData tmfd;
@@ -4514,13 +4742,14 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
BufferHeapTupleTableSlot *bslot;
HeapTuple oldtup;
bool shouldFree = true;
- Bitmapset *modified_idx_attrs;
- bool hot_allowed,
- summarized_only;
+ Bitmapset *local_modified_idx_attrs;
+ HeapUpdateIndexMode hot_mode;
Buffer buffer;
Assert(ItemPointerIsValid(otid));
+ *update_all_indexes = false;
+
/*
* To update a heap tuple we need to find the set of modified indexed
* attributes ("modified_idx_attrs") and use that to determine if a HOT
@@ -4561,8 +4790,6 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
*/
Assert(RelationSupportsSysCache(RelationGetRelid(relation)));
- *update_indexes = TU_None;
-
/* modified_idx_attrs not yet initialized */
ExecDropSingleTupleTableSlot(slot);
@@ -4577,27 +4804,20 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
ExecStorePinnedBufferHeapTuple(&bslot->base.tupdata, slot, buffer);
oldtup = ExecFetchSlotHeapTuple(slot, false, &shouldFree);
- modified_idx_attrs = HeapUpdateModifiedIdxAttrs(relation, oldtup, tup);
- lockmode = HeapUpdateDetermineLockmode(relation, modified_idx_attrs);
- hot_allowed = HeapUpdateHotAllowable(relation, modified_idx_attrs, &summarized_only);
+ local_modified_idx_attrs = HeapUpdateModifiedIdxAttrs(relation, oldtup, tup);
+ lockmode = HeapUpdateDetermineLockmode(relation, local_modified_idx_attrs);
+ hot_mode = HeapUpdateHotAllowable(relation, local_modified_idx_attrs);
- result = heap_update(relation, otid, tup, GetCurrentCommandId(true), 0,
+ result = heap_update(relation, otid, tup, GetCurrentCommandId(true),
+ 0 /* options */ ,
InvalidSnapshot, true /* wait for commit */ ,
- &tmfd, lockmode, modified_idx_attrs, hot_allowed);
+ &tmfd, lockmode, local_modified_idx_attrs, hot_mode);
if (shouldFree)
heap_freetuple(oldtup);
ExecDropSingleTupleTableSlot(slot);
- /*
- * Decide whether new index entries are needed for the tuple
- *
- * If the update is not HOT, we must update all indexes. If the update is
- * HOT, it could be that we updated summarized columns, so we either
- * update only summarized indexes, or none at all.
- */
- *update_indexes = TU_None;
switch (result)
{
case TM_SelfModified:
@@ -4606,11 +4826,14 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
break;
case TM_Ok:
- /* done successfully */
- if (!HeapTupleIsHeapOnly(tup))
- *update_indexes = TU_All;
- else if (summarized_only)
- *update_indexes = TU_Summarizing;
+
+ /*
+ * If the tuple stored by heap_update is heap-only this was a HOT
+ * update and (subject to per-index checks) not every index needs
+ * a new entry; otherwise every index must get one pointing at the
+ * new tuple's TID.
+ */
+ *update_all_indexes = !HeapTupleIsHeapOnly(tup);
break;
case TM_Updated:
@@ -4625,6 +4848,8 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
elog(ERROR, "unrecognized heap_update status: %u", result);
break;
}
+
+ bms_free(local_modified_idx_attrs);
}
@@ -8189,40 +8414,42 @@ index_delete_check_htid(TM_IndexDeleteOp *delstate,
Assert(OffsetNumberIsValid(istatus->idxoffnum));
if (unlikely(indexpagehoffnum > maxoff))
- ereport(ERROR,
- (errcode(ERRCODE_INDEX_CORRUPTED),
- errmsg_internal("heap tid from index tuple (%u,%u) points past end of heap page line pointer array at offset %u of block %u in index \"%s\"",
- ItemPointerGetBlockNumber(htid),
- indexpagehoffnum,
- istatus->idxoffnum, delstate->iblknum,
- RelationGetRelationName(delstate->irel))));
+ {
+ /*
+ * Under HOT-indexed updates, a stale btree entry can outlive heap
+ * pruning/vacuum of the page it targets; if the target offset is past
+ * the current max, treat as vacuumable instead of raising an
+ * index-corruption error.
+ */
+ return;
+ }
iid = PageGetItemId(page, indexpagehoffnum);
if (unlikely(!ItemIdIsUsed(iid)))
- ereport(ERROR,
- (errcode(ERRCODE_INDEX_CORRUPTED),
- errmsg_internal("heap tid from index tuple (%u,%u) points to unused heap page item at offset %u of block %u in index \"%s\"",
- ItemPointerGetBlockNumber(htid),
- indexpagehoffnum,
- istatus->idxoffnum, delstate->iblknum,
- RelationGetRelationName(delstate->irel))));
-
- if (ItemIdHasStorage(iid))
{
- HeapTupleHeader htup;
-
- Assert(ItemIdIsNormal(iid));
- htup = (HeapTupleHeader) PageGetItem(page, iid);
-
- if (unlikely(HeapTupleHeaderIsHeapOnly(htup)))
- ereport(ERROR,
- (errcode(ERRCODE_INDEX_CORRUPTED),
- errmsg_internal("heap tid from index tuple (%u,%u) points to heap-only tuple at offset %u of block %u in index \"%s\"",
- ItemPointerGetBlockNumber(htid),
- indexpagehoffnum,
- istatus->idxoffnum, delstate->iblknum,
- RelationGetRelationName(delstate->irel))));
+ /*
+ * Under HOT-indexed updates, a stale btree entry can legitimately
+ * point at an LP that has since been reclaimed to LP_UNUSED by
+ * pruning before VACUUM processed the index. Treat that as "the
+ * chain is vacuumable" (caller's downstream chain walk will reach the
+ * same conclusion) rather than an index-corruption error.
+ */
+ return;
}
+
+ /*
+ * A redirect target (LP_REDIRECT) is a valid chain root: an index entry
+ * pointing at it is legitimate and the caller's chain walk decides
+ * deletability. Only genuinely normal tuples are inspected below.
+ *
+ * A normal tuple that is heap-only (HeapTupleHeaderIsHeapOnly) is also
+ * tolerated without further checks: a HOT-indexed update plants a fresh
+ * index entry that points directly at such a tuple (it carries
+ * HEAP_INDEXED_UPDATED), and a stale btree entry can likewise arrive at
+ * a heap-only tuple when its chain root was pruned out. Both are legal
+ * under HOT-indexed; the caller's chain walk decides whether the entry
+ * is deletable, so there is nothing to check here for that case.
+ */
}
/*
@@ -8436,7 +8663,7 @@ heap_index_delete_tuples(Relation rel, TM_IndexDeleteOp *delstate)
/* Are any tuples from this HOT chain non-vacuumable? */
if (heap_hot_search_buffer(&tmp, rel, buf, &SnapshotNonVacuumable,
- &heapTuple, NULL, true))
+ &heapTuple, NULL, true, NULL, NULL, NULL))
continue; /* can't delete entry */
/* Caller will delete, since whole HOT chain is vacuumable */
@@ -9018,9 +9245,20 @@ log_heap_update(Relation reln, Buffer oldbuf,
}
}
- /* If new tuple is the single and first tuple on page... */
+ /*
+ * If new tuple is the single and first tuple on page, replay can reinit
+ * the page from scratch.
+ *
+ * Also require that the page's tuple area contains nothing other than this
+ * tuple. See heap_insert for why this matters when vacuum has left orphan
+ * tuple bytes behind an LP_UNUSED slot.
+ *
+ * NOTE: this must mirror the same logic in heap_insert()
+ */
if (ItemPointerGetOffsetNumber(&(newtup->t_self)) == FirstOffsetNumber &&
- PageGetMaxOffsetNumber(page) == FirstOffsetNumber)
+ PageGetMaxOffsetNumber(page) == FirstOffsetNumber &&
+ ((PageHeader) page)->pd_upper ==
+ ((PageHeader) page)->pd_special - MAXALIGN(newtup->t_len))
{
info |= XLOG_HEAP_INIT_PAGE;
init = true;
@@ -9082,6 +9320,7 @@ log_heap_update(Relation reln, Buffer oldbuf,
* The 'data' doesn't include the common prefix or suffix.
*/
XLogRegisterBufData(0, &xlhdr, SizeOfHeapHeader);
+
if (prefixlen == 0)
{
XLogRegisterBufData(0,
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 75b4d40aafc..07df9abc27a 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -218,48 +218,37 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, uint32 options,
Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
- const Bitmapset *modified_idx_attrs, TU_UpdateIndexes *update_indexes)
+ Bitmapset **modified_attrs)
{
bool shouldFree = true;
HeapTuple tuple = ExecFetchSlotHeapTuple(slot, true, &shouldFree);
- bool hot_allowed;
- bool summarized_only;
+ HeapUpdateIndexMode hot_mode;
TM_Result result;
Assert(ItemPointerIsValid(otid));
- hot_allowed = HeapUpdateHotAllowable(relation, modified_idx_attrs, &summarized_only);
- *lockmode = HeapUpdateDetermineLockmode(relation, modified_idx_attrs);
+ hot_mode = HeapUpdateHotAllowable(relation, *modified_attrs);
+ *lockmode = HeapUpdateDetermineLockmode(relation, *modified_attrs);
/* Update the tuple with table oid */
slot->tts_tableOid = RelationGetRelid(relation);
tuple->t_tableOid = slot->tts_tableOid;
- result = heap_update(relation, otid, tuple, cid, options, crosscheck, wait,
- tmfd, *lockmode, modified_idx_attrs, hot_allowed);
+ result = heap_update(relation, otid, tuple, cid, options,
+ crosscheck, wait,
+ tmfd, *lockmode, *modified_attrs, hot_mode);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
/*
- * Decide whether new index entries are needed for the tuple
- *
- * Note: heap_update returns the tid (location) of the new tuple in the
- * t_self field.
- *
- * If the update is not HOT, we must update all indexes. If the update is
- * HOT, it could be that we updated summarized columns, so we either
- * update only summarized indexes, or none at all.
+ * Tell the caller whether every index needs a new entry. If the new
+ * tuple is not heap-only the update was not HOT: it is an independent
+ * version requiring a fresh entry in every index, which we signal by
+ * adding the whole-row attribute to *modified_attrs. Otherwise (classic
+ * HOT or HOT-indexed) the caller consults the per-index attributes.
*/
- *update_indexes = TU_None;
- if (result == TM_Ok)
- {
- if (HeapTupleIsHeapOnly(tuple))
- {
- if (summarized_only)
- *update_indexes = TU_Summarizing;
- }
- else
- *update_indexes = TU_All;
- }
+ if (result == TM_Ok && !HeapTupleIsHeapOnly(tuple))
+ *modified_attrs = bms_add_member(*modified_attrs,
+ TableTupleUpdateAllIndexes);
if (shouldFree)
pfree(tuple);
@@ -725,9 +714,33 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
if (!index_getnext_slot(indexScan, ForwardScanDirection, slot))
break;
- /* Since we used no scan keys, should never need to recheck */
+ /*
+ * CLUSTER uses a no-key full-index scan; it cannot do any
+ * tuple-level filtering itself. The HOT-indexed reader path
+ * routinely sets xs_recheck when walking chain entries whose
+ * index key may be stale relative to the visible heap tuple.
+ * Those entries cause the same live tuple to be visited via the
+ * fresh hot-indexed-inserted entry too; including them would
+ * duplicate rows in the rewritten heap. Skip them here -- the
+ * tuple is reachable through its canonical index entry.
+ *
+ * If xs_recheck is set with actual scan keys, that's a real lossy
+ * index scenario CLUSTER can't handle (historical restriction).
+ */
if (indexScan->xs_recheck)
- elog(ERROR, "CLUSTER does not support lossy index conditions");
+ {
+ if (indexScan->numberOfKeys > 0)
+ elog(ERROR, "CLUSTER does not support lossy index conditions");
+ continue;
+ }
+
+ /*
+ * Same reasoning as for xs_recheck: a HOT-indexed stale entry
+ * would re-emit an already-visited tuple via its canonical fresh
+ * entry. Skip.
+ */
+ if (indexScan->xs_hot_indexed_stale)
+ continue;
}
else
{
@@ -1641,30 +1654,48 @@ heapam_index_build_range_scan(Relation heapRelation,
offnum = ItemPointerGetOffsetNumber(&heapTuple->t_self);
- /*
- * If a HOT tuple points to a root that we don't know about,
- * obtain root items afresh. If that still fails, report it as
- * corruption.
- */
- if (root_offsets[offnum - 1] == InvalidOffsetNumber)
+ if ((heapTuple->t_data->t_infomask2 & HEAP_INDEXED_UPDATED) != 0)
{
- Page page = BufferGetPage(hscan->rs_cbuf);
-
- LockBuffer(hscan->rs_cbuf, BUFFER_LOCK_SHARE);
- heap_get_root_tuples(page, root_offsets);
- LockBuffer(hscan->rs_cbuf, BUFFER_LOCK_UNLOCK);
+ /*
+ * HOT-indexed (Selective Index Update) live tuple: index it
+ * under its OWN TID, not the chain root. Its indexed values
+ * differ from earlier chain members', and the bitmap-overlap
+ * read path keeps an entry only when no hop after the entry's
+ * target changed the index's attributes. That holds for an
+ * entry pointing directly at the live tuple (no later hop);
+ * an entry pointed at the root would be dropped as stale,
+ * losing the row.
+ */
+ ItemPointerSet(&tid, ItemPointerGetBlockNumber(&heapTuple->t_self),
+ offnum);
}
+ else
+ {
+ /*
+ * If a HOT tuple points to a root that we don't know about,
+ * obtain root items afresh. If that still fails, report it
+ * as corruption.
+ */
+ if (root_offsets[offnum - 1] == InvalidOffsetNumber)
+ {
+ Page page = BufferGetPage(hscan->rs_cbuf);
- if (!OffsetNumberIsValid(root_offsets[offnum - 1]))
- ereport(ERROR,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg_internal("failed to find parent tuple for heap-only tuple at (%u,%u) in table \"%s\"",
- ItemPointerGetBlockNumber(&heapTuple->t_self),
- offnum,
- RelationGetRelationName(heapRelation))));
+ LockBuffer(hscan->rs_cbuf, BUFFER_LOCK_SHARE);
+ heap_get_root_tuples(page, root_offsets);
+ LockBuffer(hscan->rs_cbuf, BUFFER_LOCK_UNLOCK);
+ }
- ItemPointerSet(&tid, ItemPointerGetBlockNumber(&heapTuple->t_self),
- root_offsets[offnum - 1]);
+ if (!OffsetNumberIsValid(root_offsets[offnum - 1]))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("failed to find parent tuple for heap-only tuple at (%u,%u) in table \"%s\"",
+ ItemPointerGetBlockNumber(&heapTuple->t_self),
+ offnum,
+ RelationGetRelationName(heapRelation))));
+
+ ItemPointerSet(&tid, ItemPointerGetBlockNumber(&heapTuple->t_self),
+ root_offsets[offnum - 1]);
+ }
/* Call the AM's callback routine to process the tuple */
callback(indexRelation, &tid, values, isnull, tupleIsAlive,
@@ -1829,7 +1860,8 @@ heapam_index_validate_scan(Relation heapRelation,
rootTuple = *heapcursor;
root_offnum = ItemPointerGetOffsetNumber(heapcursor);
- if (HeapTupleIsHeapOnly(heapTuple))
+ if (HeapTupleIsHeapOnly(heapTuple) &&
+ (heapTuple->t_data->t_infomask2 & HEAP_INDEXED_UPDATED) == 0)
{
root_offnum = root_offsets[root_offnum - 1];
if (!OffsetNumberIsValid(root_offnum))
@@ -2586,6 +2618,7 @@ BitmapHeapScanNextBlock(TableScanDesc scan,
* offset.
*/
int curslot;
+ bool page_had_hot_indexed = false;
/* We must have extracted the tuple offsets by now */
Assert(noffsets > -1);
@@ -2595,11 +2628,63 @@ BitmapHeapScanNextBlock(TableScanDesc scan,
OffsetNumber offnum = offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
+ bool hot_indexed_stale = false;
ItemPointerSet(&tid, block, offnum);
if (heap_hot_search_buffer(&tid, scan->rs_rd, buffer, snapshot,
- &heapTuple, NULL, true))
- hscan->rs_vistuples[ntup++] = ItemPointerGetOffsetNumber(&tid);
+ &heapTuple, NULL, true,
+ &hot_indexed_stale, NULL, NULL))
+ {
+ OffsetNumber resolved = ItemPointerGetOffsetNumber(&tid);
+ bool already_have = false;
+
+ /*
+ * A bitmap heap scan cannot attribute a TID to one index, so
+ * any crossed in-chain HOT/SIU hop means the arriving entry
+ * may be stale; recheck/dedup conservatively.
+ */
+ if (hot_indexed_stale)
+ page_had_hot_indexed = true;
+
+ /*
+ * With HOT-indexed updates, more than one bitmap entry on the
+ * same block can chain-resolve to the same live tuple (a
+ * stale old-key entry plus the fresh new-key entry, or
+ * multiple stale entries from successive hot-indexed
+ * updates). Once we've seen any hot-indexed hop on this
+ * block dedup inline so upper nodes (e.g., MERGE) don't see
+ * the same row twice. Preserve original insertion order:
+ * MERGE's RETURNING ordering and test harness stability both
+ * depend on it. In the absence of hot-indexed on the page we
+ * skip the linear scan entirely -- the TBM's TIDs are already
+ * distinct by construction.
+ */
+ if (page_had_hot_indexed)
+ {
+ for (int j = 0; j < ntup; j++)
+ {
+ if (hscan->rs_vistuples[j] == resolved)
+ {
+ already_have = true;
+ break;
+ }
+ }
+ }
+
+ if (!already_have)
+ hscan->rs_vistuples[ntup++] = resolved;
+
+ /*
+ * If we reached the visible tuple through a HOT-indexed
+ * (hot-indexed) hop, the bitmap index entry that pointed us
+ * at the chain root may describe key values the visible tuple
+ * no longer has. Force BitmapHeapScan to run its recheck
+ * qual against these tuples even if the bitmap page was
+ * otherwise exact.
+ */
+ if (hot_indexed_stale)
+ *recheck = true;
+ }
}
}
else
diff --git a/src/backend/access/heap/heapam_indexscan.c b/src/backend/access/heap/heapam_indexscan.c
index 33d14f1de7d..82a3fb2b324 100644
--- a/src/backend/access/heap/heapam_indexscan.c
+++ b/src/backend/access/heap/heapam_indexscan.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/heapam.h"
+#include "access/hot_indexed.h"
#include "access/relscan.h"
#include "storage/predicate.h"
@@ -35,6 +36,14 @@ heapam_index_fetch_begin(Relation rel, uint32 flags)
hscan->xs_blk = InvalidBlockNumber;
hscan->xs_vmbuffer = InvalidBuffer;
+ /*
+ * Scratch space for the union of modified-attrs bitmaps that a HOT/SIU
+ * chain walk crosses, sized for this relation's column count. Threaded
+ * back out through xs_hot_indexed_crossed for the index-access layer.
+ */
+ hscan->xs_base.xs_hot_indexed_crossed =
+ palloc0(HotIndexedBitmapBytes(RelationGetNumberOfAttributes(rel)));
+
return &hscan->xs_base;
}
@@ -63,6 +72,9 @@ heapam_index_fetch_end(IndexFetchTableData *scan)
if (BufferIsValid(hscan->xs_vmbuffer))
ReleaseBuffer(hscan->xs_vmbuffer);
+ if (hscan->xs_base.xs_hot_indexed_crossed != NULL)
+ pfree(hscan->xs_base.xs_hot_indexed_crossed);
+
pfree(hscan);
}
@@ -83,13 +95,24 @@ heapam_index_fetch_end(IndexFetchTableData *scan)
* globally dead; *all_dead is set true if all members of the HOT chain
* are vacuumable, false if not.
*
+ * If hot_indexed_recheck is not NULL, *hot_indexed_recheck is set true iff the
+ * walk crossed a HOT-selectively-updated (HOT/SIU) hop after the entry tuple
+ * on the way to the returned tuple -- i.e. the arriving index entry's stored
+ * key may no longer match the live tuple, so the caller must recheck it (via
+ * a leaf-key comparison or a qual recheck). The entry tuple's own producing
+ * hop is excluded, so a fresh entry pointing directly at its tuple is not
+ * flagged. When no such hop was crossed, *hot_indexed_recheck is left false.
+ *
* Unlike heap_fetch, the caller must already have pin and (at least) share
* lock on the buffer; it is still pinned/locked at exit.
*/
bool
heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer,
Snapshot snapshot, HeapTuple heapTuple,
- bool *all_dead, bool first_call)
+ bool *all_dead, bool first_call,
+ bool *hot_indexed_recheck,
+ uint8 *crossed_bitmap,
+ bool *prefix_all_dead)
{
Page page = BufferGetPage(buffer);
TransactionId prev_xmax = InvalidTransactionId;
@@ -98,12 +121,30 @@ heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer,
bool at_chain_start;
bool valid;
bool skip;
+ bool prefix_dead;
GlobalVisState *vistest = NULL;
+ int relnatts = RelationGetNumberOfAttributes(relation);
+
+ /* Only track prefix_dead when the caller actually wants it. */
+ prefix_dead = (prefix_all_dead != NULL);
/* If this is not the first call, previous call returned a (live!) tuple */
if (all_dead)
*all_dead = first_call;
+ /*
+ * On the first call, clear the recheck flag and the crossed-attrs union.
+ * On subsequent calls (same chain continuing) keep whatever an earlier
+ * hop already accumulated.
+ */
+ if (first_call)
+ {
+ if (hot_indexed_recheck)
+ *hot_indexed_recheck = false;
+ if (crossed_bitmap)
+ memset(crossed_bitmap, 0, HotIndexedBitmapBytes(relnatts));
+ }
+
blkno = ItemPointerGetBlockNumber(tid);
offnum = ItemPointerGetOffsetNumber(tid);
at_chain_start = first_call;
@@ -130,7 +171,17 @@ heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer,
/* We should only see a redirect at start of chain */
if (ItemIdIsRedirected(lp) && at_chain_start)
{
- /* Follow the redirect */
+ /*
+ * Follow the redirect. A collapsed dead prefix is preserved
+ * as a run of forwarding stubs, each carrying its segment's
+ * modified-attrs bitmap, ending at the first live tuple;
+ * chain collapse reclaims a dead member only when its
+ * attributes are a subset of the surviving later hops (see
+ * pruneheap.c). So the stubs and live hops this walk crosses
+ * below contribute the complete union of every collapsed
+ * hop's modified attributes, and that union drives the
+ * overlap staleness test for the index-access layer.
+ */
offnum = ItemIdGetRedirect(lp);
at_chain_start = false;
continue;
@@ -151,10 +202,111 @@ heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer,
ItemPointerSet(&heapTuple->t_self, blkno, offnum);
/*
- * Shouldn't see a HEAP_ONLY tuple at chain start.
+ * A collapse-survivor stub is an LP_NORMAL item but not a real tuple:
+ * it is a freeze-safe forwarding node carrying the modified-attrs
+ * bitmap for the chain segment it represents. Treat it like a
+ * crossed HOT/SIU hop -- arm the recheck and OR its bitmap into the
+ * crossed union (unless we arrived directly at it, in which case the
+ * arriving entry already reflects this segment's value) -- then
+ * follow its forward link. A stub is never visible and never
+ * returned, and its forward link is a logical, not xid-continuous,
+ * edge, so reset prev_xmax to skip the chain-integrity check on the
+ * next member.
+ */
+ if (HotIndexedHeaderIsStub(heapTuple->t_data))
+ {
+ if (!at_chain_start)
+ {
+ if (hot_indexed_recheck)
+ *hot_indexed_recheck = true;
+ if (crossed_bitmap)
+ {
+ int bmnatts =
+ HotIndexedTupleBitmapNatts(heapTuple->t_data);
+
+ /*
+ * A hop's write-time natts can never legitimately exceed
+ * the relation's current natts. On a corrupt page a
+ * stub's unbounded stashed natts could otherwise overflow
+ * crossed_bitmap, which is allocated for relnatts; clamp
+ * defensively.
+ */
+ Assert(bmnatts <= relnatts);
+ if (bmnatts > relnatts)
+ bmnatts = relnatts;
+
+ HotIndexedBitmapUnion(crossed_bitmap,
+ HotIndexedGetModifiedBitmap(heapTuple->t_data,
+ heapTuple->t_len,
+ bmnatts),
+ bmnatts);
+ }
+ }
+ offnum = HotIndexedStubGetForward(heapTuple->t_data);
+ at_chain_start = false;
+ prev_xmax = InvalidTransactionId;
+ continue;
+ }
+
+ /*
+ * Shouldn't see a HEAP_ONLY tuple at chain start, unless that tuple
+ * is the target of a freshly-inserted hot-indexed index entry: then
+ * arriving directly at a heap-only HOT-indexed tuple is legal and the
+ * tuple is the canonical visible version, so we fall through and
+ * apply normal visibility checks to it. Otherwise, treat it as a
+ * broken chain.
*/
if (at_chain_start && HeapTupleIsHeapOnly(heapTuple))
- break;
+ {
+ if ((heapTuple->t_data->t_infomask2 & HEAP_INDEXED_UPDATED) == 0)
+ break;
+
+ /*
+ * We were pointed directly at this hot-indexed tuple. The index
+ * entry we arrived through was inserted *for* this update, so it
+ * reflects this tuple's current attribute values; its own
+ * producing hop is not a crossed hop, so it is not flagged for
+ * recheck (a fresh entry is never stale for its own index).
+ */
+ }
+ else if (hot_indexed_recheck != NULL &&
+ (heapTuple->t_data->t_infomask2 & HEAP_INDEXED_UPDATED) != 0)
+ {
+ /*
+ * A HOT/SIU hop reached by following the chain (or a redirect)
+ * from an earlier entry: this hop is crossed, so the arriving
+ * entry's stored key may no longer match the live tuple. Set the
+ * recheck flag to tell the index-access layer to consult the
+ * crossed-attrs union; that union (accumulated below) is what
+ * decides staleness.
+ */
+ *hot_indexed_recheck = true;
+
+ /*
+ * Accumulate this hop's modified-attrs bitmap into the crossed
+ * union. A tuple's inline bitmap records the indexed attributes
+ * that changed at the hop INTO it, which is exactly the hop we
+ * just crossed by advancing to it; ORing each crossed hop yields
+ * the indexed attributes that changed after the entry's own
+ * tuple.
+ */
+ if (crossed_bitmap)
+ {
+ int bmnatts =
+ HotIndexedTupleBitmapNatts(heapTuple->t_data);
+
+ /* See the comment on the stub case's crossed_bitmap use. */
+ Assert(bmnatts <= relnatts);
+ if (bmnatts > relnatts)
+ bmnatts = relnatts;
+
+ HotIndexedBitmapUnion(crossed_bitmap,
+ HotIndexedGetModifiedBitmap(heapTuple->t_data,
+ heapTuple->t_len,
+ bmnatts),
+ bmnatts);
+ }
+ }
/*
* The xmin should match the previous xmax value, else chain is
@@ -186,6 +338,15 @@ heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer,
HeapTupleHeaderGetXmin(heapTuple->t_data));
if (all_dead)
*all_dead = false;
+
+ /*
+ * Report whether every chain member skipped before this
+ * visible tuple is dead to all transactions. With a stale
+ * verdict this lets the caller kill the arriving leaf safely.
+ */
+ if (prefix_all_dead)
+ *prefix_all_dead = prefix_dead;
+
return true;
}
}
@@ -194,18 +355,25 @@ heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer,
/*
* If we can't see it, maybe no one else can either. At caller
* request, check whether all chain members are dead to all
- * transactions.
+ * transactions. The same surely-dead test feeds prefix_dead, which
+ * (unlike all_dead) is not reset when a visible tuple is found, so it
+ * records whether the members skipped ahead of the returned tuple are
+ * all dead to all -- the safe-to-kill-this-leaf condition.
*
* Note: if you change the criterion here for what is "dead", fix the
* planner's get_actual_variable_range() function to match.
*/
- if (all_dead && *all_dead)
+ if ((all_dead && *all_dead) || (prefix_all_dead && prefix_dead))
{
if (!vistest)
vistest = GlobalVisTestFor(relation);
if (!HeapTupleIsSurelyDead(heapTuple, vistest))
- *all_dead = false;
+ {
+ if (all_dead)
+ *all_dead = false;
+ prefix_dead = false;
+ }
}
/*
@@ -273,7 +441,15 @@ heapam_index_fetch_tuple(struct IndexFetchTableData *scan,
snapshot,
&bslot->base.tupdata,
all_dead,
- !*heap_continue);
+ !*heap_continue,
+ &scan->xs_hot_indexed_recheck,
+ scan->xs_hot_indexed_crossed,
+ &scan->xs_prefix_all_dead);
+ if (!got_heap_tuple)
+ {
+ scan->xs_hot_indexed_recheck = false;
+ scan->xs_prefix_all_dead = false;
+ }
bslot->base.tupdata.t_self = *tid;
LockBuffer(hscan->xs_cbuf, BUFFER_LOCK_UNLOCK);
diff --git a/src/backend/access/index/genam.c b/src/backend/access/index/genam.c
index 1408989c568..6628f9bf85d 100644
--- a/src/backend/access/index/genam.c
+++ b/src/backend/access/index/genam.c
@@ -103,6 +103,9 @@ RelationGetIndexScan(Relation indexRelation, int nkeys, int norderbys)
scan->orderByData = NULL;
scan->xs_want_itup = false; /* may be set later */
+ scan->xs_index_only = false; /* may be set later */
+
+ scan->xs_hot_indexed_stale = false;
/*
* During recovery we ignore killed tuples and don't bother to kill them
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 7967e939847..7237777e61c 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -44,6 +44,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/hot_indexed.h"
#include "access/relation.h"
#include "access/reloptions.h"
#include "access/relscan.h"
@@ -606,6 +607,15 @@ index_getnext_tid(IndexScanDesc scan, ScanDirection direction)
/* XXX: we should assert that a snapshot is pushed or registered */
Assert(TransactionIdIsValid(RecentXmin));
+ /*
+ * Reset the HOT-indexed recheck flag: it is set by the heap AM during
+ * index_fetch_heap and is per-fetched-tuple, not per-index-entry. For
+ * IndexOnlyScan, which may skip index_fetch_heap when the VM says the
+ * entry is visible-to-all, this ensures we don't carry a stale value from
+ * a previous entry.
+ */
+ scan->xs_hot_indexed_stale = false;
+
/*
* The AM's amgettuple proc finds the next index entry matching the scan
* keys, and puts the TID into scan->xs_heaptid. It should also set
@@ -666,15 +676,97 @@ index_fetch_heap(IndexScanDesc scan, TupleTableSlot *slot)
if (found)
pgstat_count_heap_fetch(scan->indexRelation);
+ /*
+ * The table AM reported, via xs_hot_indexed_recheck, whether the walk to
+ * the live tuple crossed a HOT-indexed hop after the arriving index
+ * entry's own tuple. When it did, the entry's stored key may no longer
+ * agree with the live tuple, and we must decide whether to drop it.
+ *
+ * The crossed-attribute bitmap (xs_hot_indexed_crossed) is the staleness
+ * authority. It is the union of the per-hop modified-attribute bitmaps
+ * of every hop the walk crossed, and it is complete: each crossed live
+ * hop, collapse-survivor stub, and redirected (collapsed) prefix
+ * contributes its segment's bitmap, and chain collapse only ever reclaims
+ * a member whose attributes are a subset of the surviving later hops (see
+ * pruneheap.c). Therefore:
+ *
+ * - if the union is disjoint from the heap columns this index references,
+ * none of the index's inputs changed across the chain, so the entry's key
+ * still matches the live tuple: keep it; and
+ *
+ * - if the union overlaps them, one of this index's key columns changed
+ * after the entry's own tuple, so the entry is stale: drop it.
+ *
+ * Dropping on overlap is correct even when the key was cycled away and
+ * back to its original value (an ABA update): the update that set the
+ * value back created a fresh entry pointing at its own (live) tuple,
+ * whose walk crosses no later key-changing hop, so that entry uniquely
+ * supplies the row while this stale ancestor entry is dropped. No
+ * value-recheck is needed, so this works for any access method; the
+ * staleness decision is purely attribute-based.
+ */
+ scan->xs_hot_indexed_stale = false;
+ if (found &&
+ scan->xs_heapfetch->xs_hot_indexed_recheck &&
+ scan->xs_heapfetch->xs_hot_indexed_crossed != NULL)
+ {
+ Bitmapset *idxattrs = RelationGetIndexedAttrs(scan->indexRelation);
+ int x = -1;
+
+ while ((x = bms_next_member(idxattrs, x)) >= 0)
+ {
+ AttrNumber attnum = x + FirstLowInvalidHeapAttributeNumber;
+
+ /* the crossed bitmap records only user attributes */
+ if (attnum >= 1 &&
+ HotIndexedAttrIsModified(scan->xs_heapfetch->xs_hot_indexed_crossed,
+ attnum))
+ {
+ scan->xs_hot_indexed_stale = true;
+ break;
+ }
+ }
+ bms_free(idxattrs);
+ }
+
/*
* If we scanned a whole HOT chain and found only dead tuples, tell index
* AM to kill its entry for that TID (this will take effect in the next
* amgettuple call, in index_getnext_tid). We do not do this when in
* recovery because it may violate MVCC to do so. See comments in
* RelationGetIndexScan().
+ *
+ * Additionally kill a stale HOT-indexed leaf (one whose key the live
+ * tuple no longer holds) when every chain member skipped before the
+ * returned tuple is dead to all transactions (xs_prefix_all_dead): no
+ * snapshot can reach a matching version through this leaf, so it is
+ * redundant and reclaiming it bounds the index bloat HOT-indexed updates
+ * create.
+ *
+ * Two independent conditions make this safe:
+ *
+ * - The surely-dead prefix gate (xs_prefix_all_dead) means no snapshot,
+ * including older ones still running, can reach a version through this
+ * leaf whose key matches: every member ahead of the live tuple is dead
+ * to all. This is what makes it MVCC-safe, exactly as for the
+ * all_dead case.
+ *
+ * - The leaf is genuinely redundant, not the row's only entry. A stale
+ * verdict means the crossed-hop union overlaps this index's columns,
+ * i.e. one of this index's attributes changed on a hop after this
+ * leaf's target. The update that made that change maintained this
+ * index (its attribute changed), so it planted a fresh entry pointing
+ * at its own live tuple; that fresh entry crosses no later
+ * key-changing hop and uniquely supplies the row. Dropping the stale
+ * ancestor therefore never removes the row's last reachable entry.
+ * This holds even under ABA key cycling (X -> Y -> X): the X-restoring
+ * update changed this index's column (Y -> X) and so planted the fresh
+ * entry.
*/
if (!scan->xactStartedInRecovery)
- scan->kill_prior_tuple = all_dead;
+ scan->kill_prior_tuple =
+ all_dead ||
+ (scan->xs_hot_indexed_stale && scan->xs_heapfetch->xs_prefix_all_dead);
return found;
}
diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c
index c8af97dd23d..2a106398d51 100644
--- a/src/backend/access/nbtree/nbtinsert.c
+++ b/src/backend/access/nbtree/nbtinsert.c
@@ -15,6 +15,8 @@
#include "postgres.h"
+#include "access/genam.h"
+#include "access/htup_details.h"
#include "access/nbtree.h"
#include "access/nbtxlog.h"
#include "access/tableam.h"
@@ -22,18 +24,24 @@
#include "access/xloginsert.h"
#include "common/int.h"
#include "common/pg_prng.h"
+#include "executor/tuptable.h"
#include "lib/qunique.h"
#include "miscadmin.h"
#include "storage/lmgr.h"
#include "storage/predicate.h"
+#include "utils/datum.h"
#include "utils/injection_point.h"
-
/* Minimum tree height for application of fastpath optimization */
#define BTREE_FASTPATH_MIN_LEVEL 2
static BTStack _bt_search_insert(Relation rel, Relation heaprel,
BTInsertState insertstate);
+
+/* Internal helper: HOT-indexed leaf-key staleness check for _bt_check_unique. */
+static bool _bt_heap_keys_equal_leaf(Relation rel, IndexTuple leaftup,
+ struct TupleTableSlot *heapSlot);
+
static TransactionId _bt_check_unique(Relation rel, BTInsertState insertstate,
Relation heapRel,
IndexUniqueCheck checkUnique, bool *is_unique,
@@ -426,6 +434,8 @@ _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel,
bool inposting = false;
bool prevalldead = true;
int curposti = 0;
+ TupleTableSlot *chain_walk_slot = NULL;
+ bool hi_recheck = false;
/* Assume unique until we find a duplicate */
*is_unique = true;
@@ -509,6 +519,7 @@ _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel,
{
ItemPointerData htid;
bool all_dead = false;
+ bool hot_indexed_stale = false;
if (!inposting)
{
@@ -559,13 +570,80 @@ _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel,
* satisfying SnapshotDirty. This is necessary because for AMs
* with optimizations like heap's HOT, we have just a single
* index entry for the entire chain.
+ *
+ * The fetch reports (hi_recheck) whether the chain walk to
+ * the live tuple crossed a HOT-selectively-updated (HOT/SIU)
+ * hop. In classic HOT the chain preserves the index key, so a
+ * live tuple anywhere in the chain is a definite conflict;
+ * with HOT/SIU that invariant no longer holds -- an old index
+ * entry for key K may chain-lead to a heap tuple whose actual
+ * index key is K'. When a hop was crossed we recheck the
+ * leaf key against the live tuple below; a stale entry is
+ * filtered out, not treated as a conflict. chain_walk_slot
+ * holds the live tuple for that recheck and is freed at every
+ * exit.
*/
- else if (table_index_fetch_tuple_check(heapRel, &htid,
+ else if ((chain_walk_slot != NULL ||
+ (chain_walk_slot = table_slot_create(heapRel, NULL))) &&
+ table_index_fetch_tuple_check(heapRel, &htid,
&SnapshotDirty,
- &all_dead))
+ &all_dead,
+ &hi_recheck,
+ chain_walk_slot))
{
TransactionId xwait;
+ /*
+ * The chain walk reported (hi_recheck) that it crossed at
+ * least one HOT/SIU hop on the way to the live tuple, so
+ * the classic "live tuple in the chain implies the same
+ * index key" invariant may not hold: an old index entry
+ * for key K may chain-lead to a tuple whose current key
+ * is K'. Recheck the leaf's stored key against the live
+ * tuple's current index form. A mismatch means the leaf
+ * is stale (not a conflict): skip it; the fresh entry
+ * inserted for the current value is the canonical one.
+ * Because the leaf still resolves to a live tuple, clear
+ * prevalldead so the caller never marks it LP_DEAD
+ * (killable).
+ */
+ hot_indexed_stale =
+ (hi_recheck &&
+ !_bt_heap_keys_equal_leaf(rel, curitup, chain_walk_slot));
+
+ if (hot_indexed_stale)
+ {
+ prevalldead = false;
+ if (nbuf != InvalidBuffer)
+ _bt_relbuf(rel, nbuf);
+ nbuf = InvalidBuffer;
+ ExecClearTuple(chain_walk_slot);
+ goto bt_chain_walk_skip;
+ }
+
+ /*
+ * The leaf's key still matches the live tuple. If the
+ * chain walk crossed a HOT-indexed hop and resolved to
+ * the very tuple the caller is inserting an entry for,
+ * this is not a duplicate -- it is the same logical row
+ * being re-indexed (e.g. a HOT-indexed UPDATE that left
+ * this index's key unchanged, or a key cycled away and
+ * back). Skip it rather than raising a spurious unique
+ * violation.
+ */
+ if (hi_recheck &&
+ ItemPointerCompare(&htid, &itup->t_tid) == 0)
+ {
+ prevalldead = false;
+ if (nbuf != InvalidBuffer)
+ _bt_relbuf(rel, nbuf);
+ nbuf = InvalidBuffer;
+ ExecClearTuple(chain_walk_slot);
+ goto bt_chain_walk_skip;
+ }
+ if (chain_walk_slot != NULL)
+ ExecClearTuple(chain_walk_slot);
+
/*
* It is a duplicate. If we are only doing a partial
* check, then don't bother checking if the tuple is being
@@ -578,6 +656,8 @@ _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel,
{
if (nbuf != InvalidBuffer)
_bt_relbuf(rel, nbuf);
+ if (chain_walk_slot)
+ ExecDropSingleTupleTableSlot(chain_walk_slot);
*is_unique = false;
return InvalidTransactionId;
}
@@ -593,6 +673,8 @@ _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel,
{
if (nbuf != InvalidBuffer)
_bt_relbuf(rel, nbuf);
+ if (chain_walk_slot)
+ ExecDropSingleTupleTableSlot(chain_walk_slot);
/* Tell _bt_doinsert to wait... */
*speculativeToken = SnapshotDirty.speculativeToken;
/* Caller releases lock on buf immediately */
@@ -619,7 +701,8 @@ _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel,
*/
htid = itup->t_tid;
if (table_index_fetch_tuple_check(heapRel, &htid,
- SnapshotSelf, NULL))
+ SnapshotSelf, NULL,
+ NULL, NULL))
{
/* Normal case --- it's still live */
}
@@ -654,6 +737,8 @@ _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel,
_bt_relbuf(rel, insertstate->buf);
insertstate->buf = InvalidBuffer;
insertstate->bounds_valid = false;
+ if (chain_walk_slot)
+ ExecDropSingleTupleTableSlot(chain_walk_slot);
{
Datum values[INDEX_MAX_KEYS];
@@ -715,6 +800,9 @@ _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel,
*/
if (!all_dead && inposting)
prevalldead = false;
+
+ bt_chain_walk_skip:
+ ;
}
}
@@ -782,9 +870,84 @@ _bt_check_unique(Relation rel, BTInsertState insertstate, Relation heapRel,
if (nbuf != InvalidBuffer)
_bt_relbuf(rel, nbuf);
+ if (chain_walk_slot)
+ ExecDropSingleTupleTableSlot(chain_walk_slot);
+
return InvalidTransactionId;
}
+/*
+ * _bt_heap_keys_equal_leaf() -- Compare a heap tuple's current btree key
+ * against the key stored in a leaf IndexTuple.
+ *
+ * The HOT-indexed unique-check path uses this to distinguish a live tuple
+ * whose current key still matches the arriving leaf (a genuine conflict)
+ * from a stale chain hit: with a HOT-indexed (Selective Index Update) chain
+ * the leaf entry for an old key still resolves to the live tuple, whose
+ * current index form may differ.
+ *
+ * Equality must agree with the index's own notion of equality, because the
+ * caller uses the verdict to decide whether to raise a unique violation.
+ * We compare each key column with its btree ordering procedure (BTORDER_PROC,
+ * the same support function _bt_mkscankey uses) under the column's collation
+ * -- not a bitwise image comparison. Bitwise equality would wrongly treat
+ * opclass-equal but image-distinct values (numeric 1.0 vs 1.00, float -0.0
+ * vs 0.0, text under a nondeterministic collation) as "not equal" and skip a
+ * genuine duplicate.
+ *
+ * This is called from _bt_check_unique while the leaf buffer is locked, so it
+ * deliberately avoids executor machinery: it fetches each key attribute
+ * straight from the slot. It is only ever reached for an index receiving a
+ * fresh entry during a HOT-indexed update, and HeapUpdateHotAllowable
+ * disqualifies any UPDATE that touches an expression-index attribute, so the
+ * index here has no expression key column (every indkey is a real attribute
+ * number). We assert that rather than handle a keycol == 0 case that cannot
+ * occur; if expression-index selective maintenance is implemented in the
+ * future, this is where an expression-evaluating comparison would be added.
+ *
+ * heapSlot must already be populated by the caller (via
+ * table_index_fetch_tuple / table_index_fetch_tuple_check).
+ */
+static bool
+_bt_heap_keys_equal_leaf(Relation rel, IndexTuple leaftup,
+ struct TupleTableSlot *heapSlot)
+{
+ TupleDesc indexDesc = RelationGetDescr(rel);
+ int nkey = IndexRelationGetNumberOfKeyAttributes(rel);
+ Form_pg_index indexStruct = rel->rd_index;
+
+ Assert(leaftup != NULL);
+ Assert(heapSlot != NULL && !TTS_EMPTY(heapSlot));
+
+ for (int i = 0; i < nkey; i++)
+ {
+ AttrNumber keycol = indexStruct->indkey.values[i];
+ Datum heap_datum;
+ bool heap_isnull;
+ Datum leaf_datum;
+ bool leaf_isnull;
+ FmgrInfo *cmpproc;
+
+ /* Expression key columns cannot reach here (see header). */
+ Assert(keycol != 0);
+
+ heap_datum = slot_getattr(heapSlot, keycol, &heap_isnull);
+ leaf_datum = index_getattr(leaftup, i + 1, indexDesc, &leaf_isnull);
+
+ if (heap_isnull != leaf_isnull)
+ return false;
+ if (heap_isnull)
+ continue;
+
+ /* opclass 3-way compare under the column's collation; 0 == equal */
+ cmpproc = index_getprocinfo(rel, i + 1, BTORDER_PROC);
+ if (DatumGetInt32(FunctionCall2Coll(cmpproc, rel->rd_indcollation[i],
+ heap_datum, leaf_datum)) != 0)
+ return false;
+ }
+
+ return true;
+}
/*
* _bt_findinsertloc() -- Finds an insert location for a tuple
diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c
index 3df2c752ead..5b269eaaa93 100644
--- a/src/backend/access/nbtree/nbtree.c
+++ b/src/backend/access/nbtree/nbtree.c
@@ -408,6 +408,16 @@ btrescan(IndexScanDesc scan, ScanKey scankey, int nscankeys,
* race condition involving VACUUM setting pages all-visible in the VM.
* It's also unsafe for plain index scans that use a non-MVCC snapshot.
*
+ * Note that wanting the index tuple (xs_want_itup) is not by itself a
+ * reason to retain the pin: btree copies each returned IndexTuple into
+ * so->currTuples (scan-local memory) and points xs_itup there, so the
+ * tuple stays valid after the pin is dropped. Only genuine index-only
+ * scans (xs_index_only), which may return a tuple without fetching the
+ * heap and therefore rely on the VM, must keep the pin. A plain index
+ * scan that sets xs_want_itup merely to inspect or recheck the index
+ * tuple still fetches and visibility-checks the heap, so it has no VM
+ * race and may drop pins like any other plain scan.
+ *
* Also opt out of dropping leaf page pins eagerly during bitmap scans.
* Pins cannot be held for more than an instant during bitmap scans either
* way, so we might as well avoid wasting cycles on acquiring page LSNs.
@@ -416,7 +426,7 @@ btrescan(IndexScanDesc scan, ScanKey scankey, int nscankeys,
*
* Note: so->dropPin should never change across rescans.
*/
- so->dropPin = (!scan->xs_want_itup &&
+ so->dropPin = (!scan->xs_index_only &&
IsMVCCLikeSnapshot(scan->xs_snapshot) &&
scan->heapRelation != NULL);
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index 12c2674cbd7..f05cabc8c65 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -242,19 +242,31 @@ bool
table_index_fetch_tuple_check(Relation rel,
ItemPointer tid,
Snapshot snapshot,
- bool *all_dead)
+ bool *all_dead,
+ bool *hot_indexed_recheck_out,
+ TupleTableSlot *keep_slot)
{
IndexFetchTableData *scan;
TupleTableSlot *slot;
bool call_again = false;
bool found;
- slot = table_slot_create(rel, NULL);
+ slot = keep_slot ? keep_slot : table_slot_create(rel, NULL);
scan = table_index_fetch_begin(rel, SO_NONE);
found = table_index_fetch_tuple(scan, tid, snapshot, slot, &call_again,
all_dead);
+
+ /*
+ * Surface the table AM's HOT/SIU recheck signal to the caller (the index
+ * AM, which rechecks the arriving leaf key against the live tuple); the
+ * scan is freed below, so copy it out.
+ */
+ if (hot_indexed_recheck_out != NULL)
+ *hot_indexed_recheck_out = found && scan->xs_hot_indexed_recheck;
+
table_index_fetch_end(scan);
- ExecDropSingleTupleTableSlot(slot);
+ if (keep_slot == NULL)
+ ExecDropSingleTupleTableSlot(slot);
return found;
}
@@ -361,8 +373,7 @@ void
simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot,
Snapshot snapshot,
- const Bitmapset *modified_idx_attrs,
- TU_UpdateIndexes *update_indexes)
+ Bitmapset **modified_attrs)
{
TM_Result result;
TM_FailureData tmfd;
@@ -373,8 +384,7 @@ simple_table_tuple_update(Relation rel, ItemPointer otid,
0, snapshot, InvalidSnapshot,
true /* wait for commit */ ,
&tmfd, &lockmode,
- modified_idx_attrs,
- update_indexes);
+ modified_attrs);
switch (result)
{
diff --git a/src/backend/catalog/indexing.c b/src/backend/catalog/indexing.c
index fd7d2ec0e3a..efd8a19c224 100644
--- a/src/backend/catalog/indexing.c
+++ b/src/backend/catalog/indexing.c
@@ -18,11 +18,14 @@
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
+#include "access/tableam.h"
#include "access/xact.h"
#include "catalog/index.h"
#include "catalog/indexing.h"
#include "executor/executor.h"
+#include "nodes/bitmapset.h"
#include "utils/rel.h"
+#include "utils/relcache.h"
/*
@@ -69,11 +72,17 @@ CatalogCloseIndexes(CatalogIndexState indstate)
*
* This should be called for each inserted or updated catalog tuple.
*
- * This is effectively a cut-down version of ExecInsertIndexTuples.
+ * This is effectively a cut-down version of ExecInsertIndexTuples. For
+ * UPDATE paths the caller supplies update_all_indexes (from
+ * table_tuple_update / simple_heap_update) so we can tell which indexes
+ * actually need a new entry: update_all_indexes is true for a fresh insert or
+ * a non-HOT update (every index gets an entry), false for a classic-HOT
+ * catalog update (non-summarizing indexes are skipped, since their existing
+ * entries still resolve the chain).
*/
static void
CatalogIndexInsert(CatalogIndexState indstate, HeapTuple heapTuple,
- TU_UpdateIndexes updateIndexes)
+ bool update_all_indexes)
{
int i;
int numIndexes;
@@ -83,20 +92,6 @@ CatalogIndexInsert(CatalogIndexState indstate, HeapTuple heapTuple,
IndexInfo **indexInfoArray;
Datum values[INDEX_MAX_KEYS];
bool isnull[INDEX_MAX_KEYS];
- bool onlySummarized = (updateIndexes == TU_Summarizing);
-
- /*
- * HOT update does not require index inserts. But with asserts enabled we
- * want to check that it'd be legal to currently insert into the
- * table/index.
- */
-#ifndef USE_ASSERT_CHECKING
- if (HeapTupleIsHeapOnly(heapTuple) && !onlySummarized)
- return;
-#endif
-
- /* When only updating summarized indexes, the tuple has to be HOT. */
- Assert((!onlySummarized) || HeapTupleIsHeapOnly(heapTuple));
/*
* Get information from the state structure. Fall out if nothing to do.
@@ -120,6 +115,7 @@ CatalogIndexInsert(CatalogIndexState indstate, HeapTuple heapTuple,
{
IndexInfo *indexInfo;
Relation index;
+ bool index_unchanged;
indexInfo = indexInfoArray[i];
index = relationDescs[i];
@@ -138,20 +134,16 @@ CatalogIndexInsert(CatalogIndexState indstate, HeapTuple heapTuple,
Assert(index->rd_index->indimmediate);
Assert(indexInfo->ii_NumIndexKeyAttrs != 0);
- /* see earlier check above */
-#ifdef USE_ASSERT_CHECKING
- if (HeapTupleIsHeapOnly(heapTuple) && !onlySummarized)
- {
- Assert(!ReindexIsProcessingIndex(RelationGetRelid(index)));
- continue;
- }
-#endif /* USE_ASSERT_CHECKING */
-
/*
- * Skip insertions into non-summarizing indexes if we only need to
- * update summarizing indexes.
+ * Decide whether this index needs a new entry. On INSERT or a
+ * non-HOT update (update_all_indexes) every index gets one. On a
+ * classic-HOT catalog update no indexed attribute changed, so the
+ * non-summarizing indexes are skipped (summarizing indexes always get
+ * a chance to update their block-level summaries below).
*/
- if (onlySummarized && !indexInfo->ii_Summarizing)
+ index_unchanged = !update_all_indexes;
+
+ if (index_unchanged && !indexInfo->ii_Summarizing)
continue;
/*
@@ -240,7 +232,7 @@ CatalogTupleInsert(Relation heapRel, HeapTuple tup)
simple_heap_insert(heapRel, tup);
- CatalogIndexInsert(indstate, tup, TU_All);
+ CatalogIndexInsert(indstate, tup, true);
CatalogCloseIndexes(indstate);
}
@@ -260,7 +252,7 @@ CatalogTupleInsertWithInfo(Relation heapRel, HeapTuple tup,
simple_heap_insert(heapRel, tup);
- CatalogIndexInsert(indstate, tup, TU_All);
+ CatalogIndexInsert(indstate, tup, true);
}
/*
@@ -291,7 +283,7 @@ CatalogTuplesMultiInsertWithInfo(Relation heapRel, TupleTableSlot **slot,
tuple = ExecFetchSlotHeapTuple(slot[i], true, &should_free);
tuple->t_tableOid = slot[i]->tts_tableOid;
- CatalogIndexInsert(indstate, tuple, TU_All);
+ CatalogIndexInsert(indstate, tuple, true);
if (should_free)
heap_freetuple(tuple);
@@ -313,15 +305,15 @@ void
CatalogTupleUpdate(Relation heapRel, const ItemPointerData *otid, HeapTuple tup)
{
CatalogIndexState indstate;
- TU_UpdateIndexes updateIndexes = TU_All;
+ bool update_all_indexes;
CatalogTupleCheckConstraints(heapRel, tup);
indstate = CatalogOpenIndexes(heapRel);
- simple_heap_update(heapRel, otid, tup, &updateIndexes);
+ simple_heap_update(heapRel, otid, tup, &update_all_indexes);
- CatalogIndexInsert(indstate, tup, updateIndexes);
+ CatalogIndexInsert(indstate, tup, update_all_indexes);
CatalogCloseIndexes(indstate);
}
@@ -337,13 +329,13 @@ void
CatalogTupleUpdateWithInfo(Relation heapRel, const ItemPointerData *otid, HeapTuple tup,
CatalogIndexState indstate)
{
- TU_UpdateIndexes updateIndexes = TU_All;
+ bool update_all_indexes;
CatalogTupleCheckConstraints(heapRel, tup);
- simple_heap_update(heapRel, otid, tup, &updateIndexes);
+ simple_heap_update(heapRel, otid, tup, &update_all_indexes);
- CatalogIndexInsert(indstate, tup, updateIndexes);
+ CatalogIndexInsert(indstate, tup, update_all_indexes);
}
/*
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..e0bc01f63d3 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -307,8 +307,6 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
indexInfo->ii_Unique = true;
indexInfo->ii_NullsNotDistinct = false;
indexInfo->ii_ReadyForInserts = true;
- indexInfo->ii_CheckedUnchanged = false;
- indexInfo->ii_IndexUnchanged = false;
indexInfo->ii_Concurrent = false;
indexInfo->ii_BrokenHotChain = false;
indexInfo->ii_ParallelWorkers = 0;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9761bdac9d0..706bc24499c 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -2688,14 +2688,14 @@ apply_concurrent_update(Relation rel, TupleTableSlot *spilled_tuple,
{
LockTupleMode lockmode;
TM_FailureData tmfd;
- TU_UpdateIndexes update_indexes;
Bitmapset *modified_idx_attrs;
TM_Result res;
/*
* Compute the set of modified indexed attributes by comparing the old
- * (ondisk) and new (spilled) tuples; heap_update needs it for a correct
- * HOT decision (a NULL set would look like "no indexed column changed").
+ * (ondisk) and new (spilled) tuples. heap_update needs this to make a
+ * correct HOT decision; without it modified_idx_attrs would be NULL and
+ * heap_update would always treat the update as HOT-eligible.
*/
modified_idx_attrs = ExecUpdateModifiedIdxAttrs(chgcxt->cc_rri,
ondisk_tuple,
@@ -2710,29 +2710,33 @@ apply_concurrent_update(Relation rel, TupleTableSlot *spilled_tuple,
InvalidSnapshot,
InvalidSnapshot,
false,
- &tmfd, &lockmode, modified_idx_attrs, &update_indexes);
+ &tmfd, &lockmode,
+ &modified_idx_attrs);
if (res != TM_Ok)
ereport(ERROR,
errcode(ERRCODE_T_R_SERIALIZATION_FAILURE),
errmsg("could not apply concurrent %s on relation \"%s\"",
"UPDATE", RelationGetRelationName(rel)));
- if (update_indexes != TU_None)
+ if (chgcxt->cc_rri->ri_NumIndices > 0 &&
+ !bms_is_empty(modified_idx_attrs))
{
- uint32 flags = EIIT_IS_UPDATE;
+ bool all_indexes =
+ bms_is_member(TableTupleUpdateAllIndexes, modified_idx_attrs);
- if (update_indexes == TU_Summarizing)
- flags |= EIIT_ONLY_SUMMARIZING;
+ ExecSetIndexUnchanged(chgcxt->cc_rri, modified_idx_attrs);
ExecInsertIndexTuples(chgcxt->cc_rri,
chgcxt->cc_estate,
- flags,
+ EIIT_IS_UPDATE |
+ (all_indexes ?
+ 0 : EIIT_IS_HOT_INDEXED),
spilled_tuple,
NIL, NULL);
}
- pgstat_progress_incr_param(PROGRESS_REPACK_HEAP_TUPLES_UPDATED, 1);
-
bms_free(modified_idx_attrs);
+
+ pgstat_progress_incr_param(PROGRESS_REPACK_HEAP_TUPLES_UPDATED, 1);
}
static void
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index eb383812901..a23144592ea 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -113,11 +113,13 @@
#include "catalog/index.h"
#include "executor/executor.h"
#include "nodes/nodeFuncs.h"
+#include "pgstat.h"
#include "storage/lmgr.h"
#include "utils/injection_point.h"
#include "utils/lsyscache.h"
#include "utils/multirangetypes.h"
#include "utils/rangetypes.h"
+#include "utils/rel.h"
#include "utils/snapmgr.h"
/* waitMode argument to check_exclusion_or_unique_constraint() */
@@ -140,11 +142,6 @@ static bool check_exclusion_or_unique_constraint(Relation heap, Relation index,
static bool index_recheck_constraint(Relation index, const Oid *constr_procs,
const Datum *existing_values, const bool *existing_isnull,
const Datum *new_values);
-static bool index_unchanged_by_update(ResultRelInfo *resultRelInfo,
- EState *estate, IndexInfo *indexInfo,
- Relation indexRelation);
-static bool index_expression_changed_walker(Node *node,
- Bitmapset *allUpdatedCols);
static void ExecWithoutOverlapsNotEmpty(Relation rel, NameData attname, Datum attval,
char typtype, Oid atttypid);
@@ -277,24 +274,12 @@ ExecCloseIndices(ResultRelInfo *resultRelInfo)
* into all the relations indexing the result relation
* when a heap tuple is inserted into the result relation.
*
- * When EIIT_IS_UPDATE is set and EIIT_ONLY_SUMMARIZING isn't,
- * executor is performing an UPDATE that could not use an
- * optimization like heapam's HOT (in more general terms a
- * call to table_tuple_update() took place and set
- * 'update_indexes' to TU_All). Receiving this hint makes
- * us consider if we should pass down the 'indexUnchanged'
- * hint in turn. That's something that we figure out for
- * each index_insert() call iff EIIT_IS_UPDATE is set.
- * (When that flag is not set we already know not to pass the
- * hint to any index.)
- *
- * If EIIT_ONLY_SUMMARIZING is set, an equivalent optimization to
- * HOT has been applied and any updated columns are indexed
- * only by summarizing indexes (or in more general terms a
- * call to table_tuple_update() took place and set
- * 'update_indexes' to TU_Summarizing). We can (and must)
- * therefore only update the indexes that have
- * 'amsummarizing' = true.
+ * When EIIT_IS_UPDATE is set, the executor is performing an
+ * UPDATE. The per-index ii_IndexUnchanged flag (populated by
+ * ExecSetIndexUnchanged()) indicates whether each index's key
+ * values are unchanged by this update. When ii_IndexUnchanged
+ * is true, we pass indexUnchanged=true to index_insert() as a
+ * hint for bottom-up deletion optimization.
*
* Unique and exclusion constraints are enforced at the same
* time. This returns a list of index OIDs for any unique or
@@ -370,11 +355,32 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
continue;
/*
- * Skip processing of non-summarizing indexes if we only update
- * summarizing indexes
+ * UPDATE skip rule. ExecSetIndexUnchanged populated
+ * ii_IndexNeedsUpdate for every index: true when the table AM stored
+ * an independent new version, or when any attribute the index
+ * references (key, INCLUDE, expression, or partial-predicate column)
+ * overlaps the modified-attrs bitmap. When it is false on a
+ * non-summarizing index we skip the insert entirely; the HOT chain
+ * keeps existing entries pointing at the chain root. Summarizing
+ * indexes always get a chance to update their block-level summaries.
*/
- if ((flags & EIIT_ONLY_SUMMARIZING) && !indexInfo->ii_Summarizing)
+ if ((flags & EIIT_IS_UPDATE) &&
+ !indexInfo->ii_IndexNeedsUpdate &&
+ !indexInfo->ii_Summarizing)
+ {
+ /*
+ * This index was skipped because its key attributes did not
+ * change. When the overall update is a HOT-indexed update (some
+ * other non-summarizing index did change), record the skip on
+ * this index's pgstat entry. A classic-HOT update (no indexed
+ * attribute changed) does not reach this path --
+ * ExecInsertIndexTuples is only invoked when at least one index
+ * needs a fresh entry.
+ */
+ if (flags & EIIT_IS_HOT_INDEXED)
+ pgstat_count_hot_indexed_upd_skipped(indexRelation);
continue;
+ }
/* Check for partial index */
if (indexInfo->ii_Predicate != NIL)
@@ -397,6 +403,17 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
continue;
}
+ /*
+ * Non-skipped index under a HOT-indexed update: this index is
+ * receiving a fresh entry because one of its key attributes changed.
+ * Summarizing indexes always insert regardless of the HOT-indexed
+ * decision (same as classic HOT), so they are not counted here. Count
+ * only now that the partial-index predicate (if any) has also passed,
+ * so a predicate-excluded partial index is not counted as matched.
+ */
+ if ((flags & EIIT_IS_HOT_INDEXED) && !indexInfo->ii_Summarizing)
+ pgstat_count_hot_indexed_upd_matched(indexRelation);
+
/*
* FormIndexDatum fills in its values and isnull parameters with the
* appropriate values for the column(s) of the index.
@@ -436,15 +453,13 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
checkUnique = UNIQUE_CHECK_PARTIAL;
/*
- * There's definitely going to be an index_insert() call for this
- * index. If we're being called as part of an UPDATE statement,
- * consider if the 'indexUnchanged' = true hint should be passed.
+ * For UPDATE operations, use the per-index ii_IndexUnchanged flag
+ * (populated by ExecSetIndexUnchanged) to hint whether the index
+ * values are unchanged. This helps the index AM optimize for
+ * bottom-up deletion of duplicate index entries.
*/
- indexUnchanged = ((flags & EIIT_IS_UPDATE) &&
- index_unchanged_by_update(resultRelInfo,
- estate,
- indexInfo,
- indexRelation));
+ indexUnchanged = (flags & EIIT_IS_UPDATE) ?
+ indexInfo->ii_IndexUnchanged : false;
satisfiesConstraint =
index_insert(indexRelation, /* index relation */
@@ -721,6 +736,7 @@ check_exclusion_or_unique_constraint(Relation heap, Relation index,
int i;
bool conflict;
bool found_self;
+ bool found_self_siu_hit;
ExprContext *econtext;
TupleTableSlot *existing_slot;
TupleTableSlot *save_scantuple;
@@ -823,6 +839,7 @@ check_exclusion_or_unique_constraint(Relation heap, Relation index,
retry:
conflict = false;
found_self = false;
+ found_self_siu_hit = false;
index_scan = index_beginscan(heap, index,
&DirtySnapshot, NULL, indnkeyatts, 0,
SO_NONE);
@@ -838,14 +855,28 @@ retry:
char *error_existing;
/*
- * Ignore the entry for the tuple we're trying to check.
+ * Ignore the entry for the tuple we're trying to check. With HOT-
+ * indexed (hot-indexed) updates, several index entries may chain-lead
+ * to the same heap tuple (a stale entry for the old key and a fresh
+ * entry for the new key). They all resolve to the same TID here and
+ * must all be treated as "self", not as a duplicate error. We
+ * tolerate the duplicate self arrival whenever *either* this
+ * iteration or an earlier one saw xs_hot_indexed_stale -- the
+ * canonical direct entry and the stale chain-walk entries can arrive
+ * in either order.
*/
if (ItemPointerIsValid(tupleid) &&
ItemPointerEquals(tupleid, &existing_slot->tts_tid))
{
- if (found_self) /* should not happen */
+ if (index_scan->xs_hot_indexed_stale)
+ found_self_siu_hit = true;
+ if (found_self)
+ {
+ if (found_self_siu_hit)
+ continue;
elog(ERROR, "found self tuple multiple times in index \"%s\"",
RelationGetRelationName(index));
+ }
found_self = true;
continue;
}
@@ -869,6 +900,31 @@ retry:
* conflict */
}
+ /*
+ * HOT-indexed chains can reach this loop via a stale btree leaf entry
+ * whose key is different from the heap tuple's current index-form.
+ * existing_values holds the current heap tuple's index-form
+ * (FormIndexDatum above). Compare it against our new tuple's values
+ * using the same constraint operators; if they don't agree, the
+ * chain-walked tuple is not actually in conflict with our insertion
+ * -- it just shared a TID with a stale leaf entry we happened to scan
+ * through. Skip it.
+ *
+ * This mirrors _bt_check_unique's HOT-indexed recheck path; for
+ * exclusion constraints the user-supplied operator in constr_procs
+ * replaces the btree equality comparator, and
+ * index_recheck_constraint does the right thing for either.
+ */
+ if (index_scan->xs_hot_indexed_stale)
+ {
+ if (!index_recheck_constraint(index,
+ constr_procs,
+ existing_values,
+ existing_isnull,
+ values))
+ continue; /* stale chain hit, not a real conflict */
+ }
+
/*
* At this point we have either a conflict or a potential conflict.
*
@@ -1009,149 +1065,94 @@ index_recheck_constraint(Relation index, const Oid *constr_procs,
}
/*
- * Check if ExecInsertIndexTuples() should pass indexUnchanged hint.
+ * ExecSetIndexUnchanged
+ *
+ * Populate two per-index flags ahead of ExecInsertIndexTuples:
+ *
+ * - ii_IndexNeedsUpdate (wide) drives the skip decision. It is true when
+ * the table AM stored an independent new version (whole-row attribute
+ * present in modified_idx_attrs) or when any attribute the index
+ * references -- key, INCLUDE, expression, or partial-predicate column,
+ * per RelationGetIndexedAttrs() -- changed. A non-summarizing index for
+ * which this is false is skipped: its existing entry keeps resolving the
+ * HOT chain.
*
- * When the executor performs an UPDATE that requires a new round of index
- * tuples, determine if we should pass 'indexUnchanged' = true hint for one
- * single index.
+ * - ii_IndexUnchanged (narrow) is the indexUnchanged hint to aminsert,
+ * consumed by nbtree deduplication / bottom-up deletion. Per the
+ * historical rule it counts only key columns; INCLUDE and predicate
+ * columns are deliberately ignored, and an expression key is treated
+ * conservatively as possibly changed.
*/
-static bool
-index_unchanged_by_update(ResultRelInfo *resultRelInfo, EState *estate,
- IndexInfo *indexInfo, Relation indexRelation)
+void
+ExecSetIndexUnchanged(ResultRelInfo *resultRelInfo,
+ const Bitmapset *modified_idx_attrs)
{
- Bitmapset *updatedCols;
- Bitmapset *extraUpdatedCols;
- Bitmapset *allUpdatedCols;
- bool hasexpression = false;
- List *idxExprs;
-
- /*
- * Check cache first
- */
- if (indexInfo->ii_CheckedUnchanged)
- return indexInfo->ii_IndexUnchanged;
- indexInfo->ii_CheckedUnchanged = true;
-
- /*
- * Check for indexed attribute overlap with updated columns.
- *
- * Only do this for key columns. A change to a non-key column within an
- * INCLUDE index should not be counted here. Non-key column values are
- * opaque payload state to the index AM, a little like an extra table TID.
- *
- * Note that row-level BEFORE triggers won't affect our behavior, since
- * they don't affect the updatedCols bitmaps generally. It doesn't seem
- * worth the trouble of checking which attributes were changed directly.
- */
- updatedCols = ExecGetUpdatedCols(resultRelInfo, estate);
- extraUpdatedCols = ExecGetExtraUpdatedCols(resultRelInfo, estate);
- for (int attr = 0; attr < indexInfo->ii_NumIndexKeyAttrs; attr++)
- {
- int keycol = indexInfo->ii_IndexAttrNumbers[attr];
-
- if (keycol <= 0)
- {
- /*
- * Skip expressions for now, but remember to deal with them later
- * on
- */
- hasexpression = true;
- continue;
- }
+ int numIndices = resultRelInfo->ri_NumIndices;
+ IndexInfo **indexInfoArray = resultRelInfo->ri_IndexRelationInfo;
+ RelationPtr indexDescs = resultRelInfo->ri_IndexRelationDescs;
+ bool all_indexes;
- if (bms_is_member(keycol - FirstLowInvalidHeapAttributeNumber,
- updatedCols) ||
- bms_is_member(keycol - FirstLowInvalidHeapAttributeNumber,
- extraUpdatedCols))
- {
- /* Changed key column -- don't hint for this index */
- indexInfo->ii_IndexUnchanged = false;
- return false;
- }
- }
-
- /*
- * When we get this far and index has no expressions, return true so that
- * index_insert() call will go on to pass 'indexUnchanged' = true hint.
- *
- * The _absence_ of an indexed key attribute that overlaps with updated
- * attributes (in addition to the total absence of indexed expressions)
- * shows that the index as a whole is logically unchanged by UPDATE.
- */
- if (!hasexpression)
- {
- indexInfo->ii_IndexUnchanged = true;
- return true;
- }
+ if (numIndices == 0)
+ return;
/*
- * Need to pass only one bms to expression_tree_walker helper function.
- * Avoid allocating memory in common case where there are no extra cols.
+ * A whole-row entry in modified_idx_attrs means the table AM stored an
+ * independent new version (e.g. at a new TID), so every index needs a
+ * fresh entry regardless of which attributes changed.
*/
- if (!extraUpdatedCols)
- allUpdatedCols = updatedCols;
- else
- allUpdatedCols = bms_union(updatedCols, extraUpdatedCols);
+ all_indexes = bms_is_member(TableTupleUpdateAllIndexes, modified_idx_attrs);
- /*
- * We have to work slightly harder in the event of indexed expressions,
- * but the principle is the same as before: try to find columns (Vars,
- * actually) that overlap with known-updated columns.
- *
- * If we find any matching Vars, don't pass hint for index. Otherwise
- * pass hint.
- */
- idxExprs = RelationGetIndexExpressions(indexRelation);
- hasexpression = index_expression_changed_walker((Node *) idxExprs,
- allUpdatedCols);
- list_free(idxExprs);
- if (extraUpdatedCols)
- bms_free(allUpdatedCols);
-
- if (hasexpression)
+ for (int i = 0; i < numIndices; i++)
{
- indexInfo->ii_IndexUnchanged = false;
- return false;
- }
+ IndexInfo *indexInfo = indexInfoArray[i];
+ Relation indexDesc = indexDescs[i];
+ Bitmapset *indexedattrs;
+ bool keychanged;
- /*
- * Deliberately don't consider index predicates. We should even give the
- * hint when result rel's "updated tuple" has no corresponding index
- * tuple, which is possible with a partial index (provided the usual
- * conditions are met).
- */
- indexInfo->ii_IndexUnchanged = true;
- return true;
-}
-
-/*
- * Indexed expression helper for index_unchanged_by_update().
- *
- * Returns true when Var that appears within allUpdatedCols located.
- */
-static bool
-index_expression_changed_walker(Node *node, Bitmapset *allUpdatedCols)
-{
- if (node == NULL)
- return false;
+ if (indexDesc == NULL)
+ continue;
- if (IsA(node, Var))
- {
- Var *var = (Var *) node;
+ /*
+ * Skip decision (wide). The index needs a new entry if the AM stored
+ * an independent version, or if any attribute it references -- key,
+ * INCLUDE, expression, or partial-predicate column -- changed.
+ * RelationGetIndexedAttrs() covers all of those. (An UPDATE that
+ * touches an expression-index attribute never reaches the HOT-indexed
+ * path: HeapUpdateHotAllowable disqualifies it, pending
+ * expression-aware maintenance.)
+ */
+ indexedattrs = RelationGetIndexedAttrs(indexDesc);
+ indexInfo->ii_IndexNeedsUpdate =
+ all_indexes || bms_overlap(indexedattrs, modified_idx_attrs);
+ bms_free(indexedattrs);
- if (bms_is_member(var->varattno - FirstLowInvalidHeapAttributeNumber,
- allUpdatedCols))
+ /*
+ * aminsert hint (narrow). ii_IndexUnchanged feeds nbtree
+ * deduplication / bottom-up deletion heuristics and, per the
+ * historical rule, counts only key columns: a change to an INCLUDE
+ * column or to a partial-index predicate column does not disqualify
+ * the hint. An expression key column is treated conservatively as
+ * possibly changed.
+ */
+ keychanged = false;
+ for (int k = 0; k < indexInfo->ii_NumIndexKeyAttrs; k++)
{
- /* Var was updated -- indicates that we should not hint */
- return true;
- }
+ AttrNumber keycol = indexInfo->ii_IndexAttrNumbers[k];
- /* Still haven't found a reason to not pass the hint */
- return false;
+ if (keycol == 0) /* expression key: assume it may have changed */
+ {
+ keychanged = true;
+ break;
+ }
+ if (bms_is_member(keycol - FirstLowInvalidHeapAttributeNumber,
+ modified_idx_attrs))
+ {
+ keychanged = true;
+ break;
+ }
+ }
+ indexInfo->ii_IndexUnchanged = !keychanged;
}
-
- return expression_tree_walker(node, index_expression_changed_walker,
- allUpdatedCols);
}
/*
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index 6262f71bd93..7b5bbdbfd7a 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -217,6 +217,18 @@ retry:
/* Try to find the tuple */
while (index_getnext_slot(scan, ForwardScanDirection, outslot))
{
+ /*
+ * A HOT-indexed update can leave a stale index leaf: an entry whose
+ * key is a pre-update value but whose TID chain-resolves to a live
+ * tuple now carrying a different key. Such a tuple is not the
+ * replica-identity match we are looking for (and the PK/RI fast path
+ * below skips the equality recheck that would otherwise catch it), so
+ * drop it -- exactly as IndexScan/IndexOnlyScan do. The fresh leaf
+ * for the current key, if any, is returned by a later iteration.
+ */
+ if (scan->xs_hot_indexed_stale)
+ continue;
+
/*
* Avoid expensive equality check if the index is primary key or
* replica identity index.
@@ -678,6 +690,10 @@ RelationFindDeletedTupleInfoByIndex(Relation rel, Oid idxoid,
/* Try to find the tuple */
while (index_getnext_slot(scan, ForwardScanDirection, scanslot))
{
+ /* Skip stale HOT-indexed leaves (see RelationFindReplTupleByIndex). */
+ if (scan->xs_hot_indexed_stale)
+ continue;
+
/*
* Avoid expensive equality check if the index is primary key or
* replica identity index.
@@ -911,7 +927,7 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
bool skip_tuple = false;
Relation rel = resultRelInfo->ri_RelationDesc;
ItemPointer tid = &(searchslot->tts_tid);
- Bitmapset *modified_idx_attrs;
+ Bitmapset *modified_idx_attrs = NULL;
/*
* We support only non-system tables, with
@@ -934,7 +950,6 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
if (!skip_tuple)
{
List *recheckIndexes = NIL;
- TU_UpdateIndexes update_indexes;
List *conflictindexes;
bool conflict = false;
@@ -953,27 +968,34 @@ ExecSimpleRelationUpdate(ResultRelInfo *resultRelInfo,
modified_idx_attrs = ExecUpdateModifiedIdxAttrs(resultRelInfo,
searchslot, slot);
+ Assert(!bms_is_member(TableTupleUpdateAllIndexes, modified_idx_attrs));
simple_table_tuple_update(rel, tid, slot, estate->es_snapshot,
- modified_idx_attrs, &update_indexes);
- bms_free(modified_idx_attrs);
-
+ &modified_idx_attrs);
conflictindexes = resultRelInfo->ri_onConflictArbiterIndexes;
- if (resultRelInfo->ri_NumIndices > 0 && (update_indexes != TU_None))
+ if (resultRelInfo->ri_NumIndices > 0 &&
+ !bms_is_empty(modified_idx_attrs))
{
+ bool all_indexes =
+ bms_is_member(TableTupleUpdateAllIndexes, modified_idx_attrs);
uint32 flags = EIIT_IS_UPDATE;
if (conflictindexes != NIL)
flags |= EIIT_NO_DUPE_ERROR;
- if (update_indexes == TU_Summarizing)
- flags |= EIIT_ONLY_SUMMARIZING;
+ if (!all_indexes)
+ flags |= EIIT_IS_HOT_INDEXED;
+
+ ExecSetIndexUnchanged(resultRelInfo, modified_idx_attrs);
+
recheckIndexes = ExecInsertIndexTuples(resultRelInfo,
estate, flags,
slot, conflictindexes,
&conflict);
}
+ bms_free(modified_idx_attrs);
+
/*
* Refer to the comments above the call to CheckAndReportConflict() in
* ExecSimpleRelationInsert to understand why this check is done at
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index d52012e8a69..cd0caef1366 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -104,6 +104,7 @@ IndexOnlyNext(IndexOnlyScanState *node)
/* Set it up for index-only scan */
node->ioss_ScanDesc->xs_want_itup = true;
+ node->ioss_ScanDesc->xs_index_only = true;
node->ioss_VMBuffer = InvalidBuffer;
/*
@@ -172,6 +173,25 @@ IndexOnlyNext(IndexOnlyScanState *node)
if (!index_fetch_heap(scandesc, node->ioss_TableSlot))
continue; /* no visible tuple, try next index entry */
+ /*
+ * HOT-indexed stale entry: if the chain walk to reach this tuple
+ * crossed a hot-indexed hop that changed an attribute this index
+ * covers, the leaf we arrived through is stale. For IOS we serve
+ * values out of xs_itup, so a stale leaf would surface the wrong
+ * values; drop it. The fresh entry for the new value returns the
+ * row with correct values via its own path. Prune keeps any page
+ * that can carry such a stale leaf -- one with a redirect to a
+ * live HEAP_INDEXED_UPDATED tuple -- out of the visibility map
+ * (see heap_prune_record_redirect), so an index-only scan always
+ * reaches this heap fetch when staleness could apply.
+ */
+ if (scandesc->xs_hot_indexed_stale)
+ {
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(node->ioss_TableSlot);
+ continue;
+ }
+
ExecClearTuple(node->ioss_TableSlot);
/*
@@ -229,6 +249,16 @@ IndexOnlyNext(IndexOnlyScanState *node)
}
}
+ /*
+ * No HOT-indexed staleness check is needed on the VM-all-visible path
+ * (where we skipped the heap fetch). Prune keeps any page that could
+ * carry a stale leaf -- one with a redirect to a live
+ * HEAP_INDEXED_UPDATED tuple -- out of the visibility map, so an
+ * all-visible entry never crossed a HOT/SIU hop. (index_getnext_tid
+ * also resets xs_hot_indexed_stale per entry, and only the heap fetch
+ * in index_fetch_heap ever sets it, so it cannot be set here anyway.)
+ */
+
/*
* We don't currently support rechecking ORDER BY distances. (In
* principle, if the index can support retrieval of the originally
@@ -775,6 +805,7 @@ ExecIndexOnlyScanInitializeDSM(IndexOnlyScanState *node,
ScanRelIsReadOnly(&node->ss) ?
SO_HINT_REL_READ_ONLY : SO_NONE);
node->ioss_ScanDesc->xs_want_itup = true;
+ node->ioss_ScanDesc->xs_index_only = true;
node->ioss_VMBuffer = InvalidBuffer;
/*
@@ -825,6 +856,7 @@ ExecIndexOnlyScanInitializeWorker(IndexOnlyScanState *node,
ScanRelIsReadOnly(&node->ss) ?
SO_HINT_REL_READ_ONLY : SO_NONE);
node->ioss_ScanDesc->xs_want_itup = true;
+ node->ioss_ScanDesc->xs_index_only = true;
/*
* If no run-time keys to calculate or they are ready, go ahead and pass
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 39f6691ee35..dad925dcb19 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -151,6 +151,20 @@ IndexNext(IndexScanState *node)
}
}
+ /*
+ * HOT-indexed stale entry: the chain we walked to reach this tuple
+ * crossed a hot-indexed hop that changed an attribute this index
+ * covers, so the leaf entry we arrived through is stale. Drop it;
+ * the fresh entry inserted for the new value returns the row through
+ * its own path. Staleness was decided by the heap AM via per-hop
+ * modified-attrs bitmaps (see heap_hot_search_buffer).
+ */
+ if (scandesc->xs_hot_indexed_stale)
+ {
+ InstrCountFiltered2(node, 1);
+ continue;
+ }
+
return slot;
}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index eb6e0ef2ad9..635a67884d8 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -130,7 +130,14 @@ typedef struct ModifyTableContext
typedef struct UpdateContext
{
bool crossPartUpdate; /* was it a cross-partition update? */
- TU_UpdateIndexes updateIndexes; /* Which index updates are required? */
+
+ /*
+ * Set of indexed attributes the UPDATE changed (in/out for the table AM's
+ * update callback). Populated by ExecUpdateAct and consumed by
+ * ExecUpdateEpilogue; the AM adds the whole-row attribute
+ * (TableTupleUpdateAllIndexes) when every index needs a fresh entry.
+ */
+ Bitmapset *modified_attrs;
/*
* Lock mode to acquire on the latest tuple version before performing
@@ -238,25 +245,23 @@ ExecUpdateModifiedIdxAttrs(ResultRelInfo *resultRelInfo,
return NULL;
/*
- * Get the set of all attributes across all indexes for this relation from
- * the relcache, it returns us a copy of the bitmap so we can modify it.
+ * Determine which indexed attributes actually changed value by comparing
+ * the old and new tuples attribute-by-attribute over the relation's full
+ * indexed-attribute set. We deliberately do NOT try to narrow the work
+ * using the SQL UPDATE's target list (ExecGetAllUpdatedCols): that list
+ * does not capture indexed columns mutated outside the SET clause, such
+ * as a column rewritten by a BEFORE/INSTEAD-OF trigger via
+ * heap_modify_tuple (see tsvector_update_trigger() in tsearch.sql), the
+ * implicit temporal range column of a FOR PORTION OF update, or the
+ * pre-built tuples applied by REPACK (CONCURRENTLY) and logical
+ * replication through a synthetic ResultRelInfo. Comparing the actual
+ * tuple values is always correct.
*
- * Note: We intentionally scan all indexed columns when looking for
- * changes rather than reduce that set by intersecting it with
- * ExecGetAllUpdatedCols(). Desipte the name it provides the set of
- * targeted attributes in the SQL used for the UPDATE and any triggers,
- * but that doesn't include any attributes updated using
- * heap_modifiy_tuple(). There is one test in tsearch.sql that does just
- * that, modifies an indexed attribute that isn't specified in the SQL and
- * so isn't present in that bitmapset.
+ * RelationGetIndexAttrBitmap returns a copy we are free to mutate;
+ * ExecCompareSlotAttrs deletes the attributes that did not change and
+ * returns the surviving "modified indexed attributes" set.
*/
attrs = RelationGetIndexAttrBitmap(relation, INDEX_ATTR_BITMAP_INDEXED);
-
- /*
- * When there are indexed attributes mentioned in the UPDATE then we need
- * to find the subset that changed value. That's the
- * "modified_idx_attrs".
- */
attrs = ExecCompareSlotAttrs(attrs, tupdesc, old_tts, new_tts);
return attrs;
@@ -2513,8 +2518,8 @@ ExecUpdateAct(ModifyTableContext *context, ResultRelInfo *resultRelInfo,
bool partition_constraint_failed;
TM_Result result;
- /* The set of modified indexed attributes that trigger new index entries */
- Bitmapset *modified_idx_attrs = NULL;
+ /* Reset any state left over from a previous call */
+ updateCxt->modified_attrs = NULL;
updateCxt->crossPartUpdate = false;
@@ -2638,7 +2643,8 @@ lreplace:
* we will overlook attributes directly modified by heap_modify_tuple()
* which are not known to ExecGetUpdatedCols().
*/
- modified_idx_attrs = ExecUpdateModifiedIdxAttrs(resultRelInfo, oldSlot, slot);
+ updateCxt->modified_attrs =
+ ExecUpdateModifiedIdxAttrs(resultRelInfo, oldSlot, slot);
/*
* Call into the table AM to update the heap tuple.
@@ -2649,6 +2655,8 @@ lreplace:
* for referential integrity updates in transaction-snapshot mode
* transactions.
*/
+ Assert(!bms_is_member(TableTupleUpdateAllIndexes,
+ updateCxt->modified_attrs));
result = table_tuple_update(resultRelationDesc, tupleid, slot,
estate->es_output_cid,
0,
@@ -2656,8 +2664,7 @@ lreplace:
estate->es_crosscheck_snapshot,
true /* wait for commit */ ,
&context->tmfd, &updateCxt->lockmode,
- modified_idx_attrs,
- &updateCxt->updateIndexes);
+ &updateCxt->modified_attrs);
return result;
}
@@ -2678,14 +2685,26 @@ ExecUpdateEpilogue(ModifyTableContext *context, UpdateContext *updateCxt,
List *recheckIndexes = NIL;
/* insert index entries for tuple if necessary */
- if (resultRelInfo->ri_NumIndices > 0 && (updateCxt->updateIndexes != TU_None))
+ if (resultRelInfo->ri_NumIndices > 0 &&
+ !bms_is_empty(updateCxt->modified_attrs))
{
- uint32 flags = EIIT_IS_UPDATE;
+ bool all_indexes =
+ bms_is_member(TableTupleUpdateAllIndexes,
+ updateCxt->modified_attrs);
+
+ /*
+ * Populate per-index ii_IndexUnchanged before inserting. When the AM
+ * stored an independent new version (whole-row attribute present)
+ * every index needs a fresh entry; for a HOT update only those whose
+ * attributes overlap the modified set do.
+ */
+ ExecSetIndexUnchanged(resultRelInfo, updateCxt->modified_attrs);
- if (updateCxt->updateIndexes == TU_Summarizing)
- flags |= EIIT_ONLY_SUMMARIZING;
recheckIndexes = ExecInsertIndexTuples(resultRelInfo, context->estate,
- flags, slot, NIL,
+ EIIT_IS_UPDATE |
+ (all_indexes ?
+ 0 : EIIT_IS_HOT_INDEXED),
+ slot, NIL,
NULL);
}
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index 40b09958ac2..f050c088d28 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -845,8 +845,6 @@ makeIndexInfo(int numattrs, int numkeyattrs, Oid amoid, List *expressions,
n->ii_Unique = unique;
n->ii_NullsNotDistinct = nulls_not_distinct;
n->ii_ReadyForInserts = isready;
- n->ii_CheckedUnchanged = false;
- n->ii_IndexUnchanged = false;
n->ii_Concurrent = concurrent;
n->ii_Summarizing = summarizing;
n->ii_WithoutOverlaps = withoutoverlaps;
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 04f2eb21d0b..dd1140b29fb 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -384,11 +384,17 @@ pgstat_count_heap_insert(Relation rel, PgStat_Counter n)
/*
* count a tuple update
+ *
+ * hot -- the update was a heap-only tuple (classic HOT or HOT-indexed)
+ * hot_indexed -- the update was a HOT-indexed update, a subcase of
+ * hot=true; hot_indexed implies hot
+ * newpage -- the new tuple went to a different buffer than the old one
*/
void
-pgstat_count_heap_update(Relation rel, bool hot, bool newpage)
+pgstat_count_heap_update(Relation rel, bool hot, bool hot_indexed, bool newpage)
{
Assert(!(hot && newpage));
+ Assert(!(hot_indexed && !hot));
if (pgstat_should_count_relation(rel))
{
@@ -398,11 +404,17 @@ pgstat_count_heap_update(Relation rel, bool hot, bool newpage)
pgstat_info->trans->tuples_updated++;
/*
- * tuples_hot_updated and tuples_newpage_updated counters are
- * nontransactional, so just advance them
+ * tuples_hot_updated, tuples_hot_indexed_updated, and
+ * tuples_newpage_updated counters are nontransactional, so just
+ * advance them. tuples_hot_indexed_updated is counted in *addition* to
+ * tuples_hot: every hot-indexed update is also a HOT update.
*/
if (hot)
+ {
pgstat_info->counts.tuples_hot_updated++;
+ if (hot_indexed)
+ pgstat_info->counts.tuples_hot_indexed_updated++;
+ }
else if (newpage)
pgstat_info->counts.tuples_newpage_updated++;
}
@@ -854,7 +866,10 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->tuples_updated += lstats->counts.tuples_updated;
tabentry->tuples_deleted += lstats->counts.tuples_deleted;
tabentry->tuples_hot_updated += lstats->counts.tuples_hot_updated;
+ tabentry->tuples_hot_indexed_updated += lstats->counts.tuples_hot_indexed_updated;
tabentry->tuples_newpage_updated += lstats->counts.tuples_newpage_updated;
+ tabentry->tuples_hot_indexed_upd_skipped += lstats->counts.tuples_hot_indexed_upd_skipped;
+ tabentry->tuples_hot_indexed_upd_matched += lstats->counts.tuples_hot_indexed_upd_matched;
/*
* If table was truncated/dropped, first reset the live/dead counters.
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 055f757107f..ca605265703 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -1586,6 +1586,7 @@ RelationInitIndexAccessInfo(Relation relation)
*/
relation->rd_indexprs = NIL;
relation->rd_indpred = NIL;
+ relation->rd_indattr = NULL;
relation->rd_exclops = NULL;
relation->rd_exclprocs = NULL;
relation->rd_exclstrats = NULL;
@@ -2484,6 +2485,7 @@ RelationDestroyRelation(Relation relation, bool remember_tupdesc)
bms_free(relation->rd_idattr);
bms_free(relation->rd_indexedattr);
bms_free(relation->rd_summarizedattr);
+ bms_free(relation->rd_exprindexattr);
if (relation->rd_pubdesc)
pfree(relation->rd_pubdesc);
if (relation->rd_options)
@@ -5275,6 +5277,130 @@ RelationGetIndexPredicate(Relation relation)
return result;
}
+/*
+ * RelationGetIndexedAttrs -- palloc'd Bitmapset of heap attrs this index
+ * references.
+ *
+ * Includes attributes used as simple key columns, INCLUDE columns, inside
+ * expression columns, and inside the partial-index predicate. Attribute
+ * numbers use the FirstLowInvalidHeapAttributeNumber offset convention so
+ * that system attributes are representable alongside user attributes.
+ *
+ * The function builds up the bitmap from:
+ * - rd_index->indkey (keys + INCLUDE)
+ * - RelationGetIndexExpressions (parsed expression trees, already cached)
+ * - RelationGetIndexPredicate (parsed predicate tree, already cached)
+ * and caches a copy in rd_indexedattr, which lives in rd_indexcxt.
+ *
+ * The returned Bitmapset is allocated in the caller's current memory
+ * context; the caller owns it and must bms_free when done. We never hand
+ * out a borrowed pointer to the cached copy because relcache invalidation
+ * can rebuild rd_indexcxt in place even while a refcount is held.
+ *
+ * Caller must hold an open lock on the index relation.
+ */
+Bitmapset *
+RelationGetIndexedAttrs(Relation indexRel)
+{
+ Bitmapset *attrs = NULL;
+ Form_pg_index indexStruct;
+ List *indexprs;
+ List *indpred;
+ MemoryContext oldcxt;
+
+ Assert(indexRel->rd_rel->relkind == RELKIND_INDEX ||
+ indexRel->rd_rel->relkind == RELKIND_PARTITIONED_INDEX);
+
+ /* Fast path: return a copy of the cached bitmap. */
+ if (indexRel->rd_indattr != NULL)
+ return bms_copy(indexRel->rd_indattr);
+
+ indexStruct = indexRel->rd_index;
+
+ /*
+ * During very early bootstrap rd_indextuple may not be populated yet. In
+ * that case we fall back to just the key columns without caching.
+ */
+ if (indexRel->rd_indextuple == NULL)
+ {
+ for (int i = 0; i < indexStruct->indnatts; i++)
+ {
+ AttrNumber attrnum = indexStruct->indkey.values[i];
+
+ if (attrnum != 0)
+ attrs = bms_add_member(attrs,
+ attrnum - FirstLowInvalidHeapAttributeNumber);
+ }
+ return attrs;
+ }
+
+ /*
+ * Key columns and INCLUDE (covering) columns. INCLUDE columns must be
+ * counted: their values are stored in the index leaf and served by
+ * index-only scans, so an update that changes an INCLUDE column must
+ * insert a fresh index entry (or be disqualified from staying
+ * HOT-indexed) exactly as for a key column. This matches the heap-level
+ * RelationGetIndexAttrBitmap(..., INDEX_ATTR_BITMAP_INDEXED), which also
+ * unions all indnatts. Expression and partial-predicate columns are
+ * added below.
+ */
+ for (int i = 0; i < indexStruct->indnatts; i++)
+ {
+ AttrNumber attrnum = indexStruct->indkey.values[i];
+
+ /* attnum 0 means "expression"; those attrs are picked up below. */
+ if (attrnum != 0)
+ attrs = bms_add_member(attrs,
+ attrnum - FirstLowInvalidHeapAttributeNumber);
+ }
+
+ /*
+ * Expression columns and partial-index predicate columns. Deliberately
+ * do NOT use RelationGetIndexExpressions()/RelationGetIndexPredicate()
+ * here, for the same reason RelationGetIndexAttrBitmap avoids them: those
+ * functions run eval_const_expressions(), which needs a snapshot we may
+ * not have, and can const-fold away a Var reference (e.g. a CASE with a
+ * statically-decidable branch), causing this bitmap to under-report an
+ * attribute that rd_indexedattr/rd_exprindexattr (built from the same raw
+ * catalog text) still record. Parse the raw stored trees instead, so
+ * this function's result stays a superset-consistent match with those.
+ */
+ {
+ Datum datum;
+ bool isnull;
+
+ datum = heap_getattr(indexRel->rd_indextuple, Anum_pg_index_indexprs,
+ GetPgIndexDescriptor(), &isnull);
+ if (!isnull)
+ {
+ indexprs = (List *) stringToNode(TextDatumGetCString(datum));
+ pull_varattnos((Node *) indexprs, 1, &attrs);
+ }
+
+ datum = heap_getattr(indexRel->rd_indextuple, Anum_pg_index_indpred,
+ GetPgIndexDescriptor(), &isnull);
+ if (!isnull)
+ {
+ indpred = (List *) stringToNode(TextDatumGetCString(datum));
+ pull_varattnos((Node *) indpred, 1, &attrs);
+ }
+ }
+
+ /*
+ * Cache a copy inside rd_indexcxt so subsequent calls are cheap. The
+ * cached bitmap is freed along with rd_indexcxt on relcache rebuild, so
+ * it's safe to stash here.
+ */
+ if (indexRel->rd_indexcxt != NULL)
+ {
+ oldcxt = MemoryContextSwitchTo(indexRel->rd_indexcxt);
+ indexRel->rd_indattr = bms_copy(attrs);
+ MemoryContextSwitchTo(oldcxt);
+ }
+
+ return attrs;
+}
+
/*
* RelationGetIndexAttrBitmap -- get a bitmap of index attribute numbers
*
@@ -5313,6 +5439,7 @@ Bitmapset *
RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
{
Bitmapset *uindexattrs; /* columns in unique indexes */
+ Bitmapset *exprindexattrs; /* columns referenced by expression indexes */
Bitmapset *pkindexattrs; /* columns in the primary index */
Bitmapset *idindexattrs; /* columns in the replica identity */
Bitmapset *indexedattrs; /* columns referenced by indexes */
@@ -5339,6 +5466,8 @@ RelationGetIndexAttrBitmap(Relation relation, IndexAttrBitmapKind attrKind)
return bms_copy(relation->rd_indexedattr);
case INDEX_ATTR_BITMAP_SUMMARIZED:
return bms_copy(relation->rd_summarizedattr);
+ case INDEX_ATTR_BITMAP_EXPRESSION:
+ return bms_copy(relation->rd_exprindexattr);
default:
elog(ERROR, "unknown attrKind %u", attrKind);
}
@@ -5383,6 +5512,7 @@ restart:
idindexattrs = NULL;
indexedattrs = NULL;
summarizedattrs = NULL;
+ exprindexattrs = NULL;
foreach(l, indexoidlist)
{
Oid indexOid = lfirst_oid(l);
@@ -5487,6 +5617,28 @@ restart:
/* Collect all attributes in the index predicate, too */
pull_varattnos(indexPredicate, 1, attrs);
+ /*
+ * If this index evaluates an expression, record every heap attribute
+ * it references (key columns, expression vars, predicate vars) in
+ * exprindexattrs. HeapUpdateHotAllowable() disqualifies the
+ * HOT-indexed path for an UPDATE that touches one of these, because
+ * expression-aware selective index maintenance is not implemented
+ * yet.
+ */
+ if (indexExpressions != NULL)
+ {
+ for (i = 0; i < indexDesc->rd_index->indnatts; i++)
+ {
+ int attrnum = indexDesc->rd_index->indkey.values[i];
+
+ if (attrnum != 0)
+ exprindexattrs = bms_add_member(exprindexattrs,
+ attrnum - FirstLowInvalidHeapAttributeNumber);
+ }
+ pull_varattnos(indexExpressions, 1, &exprindexattrs);
+ pull_varattnos(indexPredicate, 1, &exprindexattrs);
+ }
+
index_close(indexDesc, AccessShareLock);
}
@@ -5515,14 +5667,27 @@ restart:
bms_free(idindexattrs);
bms_free(indexedattrs);
bms_free(summarizedattrs);
+ bms_free(exprindexattrs);
goto restart;
}
/*
- * Record what attributes are only referenced by summarizing indexes. Then
- * add that into the other indexed attributes to track all referenced
- * attributes.
+ * Record which attributes are referenced only by summarizing indexes, so
+ * INDEX_ATTR_BITMAP_SUMMARIZED reports columns whose sole indexes are
+ * summarizing ones, then fold those columns into indexedattrs as well.
+ *
+ * INDEX_ATTR_BITMAP_INDEXED must include summarizing-index columns for
+ * the HOT-indexed write path: it compares the old and new tuples over
+ * this bitmap to build the set of modified indexed attributes, and only
+ * maintains indexes when that set is non-empty (or the update is
+ * non-HOT). A change to a column indexed only by a summarizing index
+ * must therefore appear in the bitmap so the summarizing index gets its
+ * block summary refreshed. HeapUpdateHotAllowable's all_summarizing
+ * check still keeps such an update on the classic-HOT path (it stays
+ * classic HOT, since INDEX_ATTR_BITMAP_SUMMARIZED -- summarizing-only --
+ * is a superset of the modified attributes), and the summarizing index
+ * inserts unconditionally via its ii_Summarizing flag.
*/
summarizedattrs = bms_del_members(summarizedattrs, indexedattrs);
indexedattrs = bms_add_members(indexedattrs, summarizedattrs);
@@ -5539,6 +5704,8 @@ restart:
relation->rd_indexedattr = NULL;
bms_free(relation->rd_summarizedattr);
relation->rd_summarizedattr = NULL;
+ bms_free(relation->rd_exprindexattr);
+ relation->rd_exprindexattr = NULL;
/*
* Now save copies of the bitmaps in the relcache entry. We intentionally
@@ -5553,6 +5720,7 @@ restart:
relation->rd_idattr = bms_copy(idindexattrs);
relation->rd_indexedattr = bms_copy(indexedattrs);
relation->rd_summarizedattr = bms_copy(summarizedattrs);
+ relation->rd_exprindexattr = bms_copy(exprindexattrs);
relation->rd_attrsvalid = true;
MemoryContextSwitchTo(oldcxt);
@@ -5569,6 +5737,72 @@ restart:
return indexedattrs;
case INDEX_ATTR_BITMAP_SUMMARIZED:
return summarizedattrs;
+ case INDEX_ATTR_BITMAP_EXPRESSION:
+ return exprindexattrs;
+ default:
+ elog(ERROR, "unknown attrKind %u", attrKind);
+ return NULL;
+ }
+}
+
+/*
+ * RelationGetIndexAttrBitmapNoCopy -- borrowing variant of
+ * RelationGetIndexAttrBitmap
+ *
+ * Returns a pointer to the relcache-owned bitmap for the given attrKind
+ * without making a defensive copy. This is a hot-path optimization for
+ * read-only callers that perform set operations like bms_overlap,
+ * bms_is_subset, bms_equal, or bms_num_members and never mutate the
+ * returned bitmap. The result is conceptually `const Bitmapset *`; callers
+ * must not pass it to anything that could free or modify the underlying
+ * memory (e.g., bms_add_member, bms_int_members, bms_free).
+ *
+ * Lifetime: the pointer is valid only until the next event that could
+ * trigger a relcache invalidation on `relation`. Callers must not invoke
+ * any code that opens a relation, runs catalog lookups, or otherwise
+ * accepts invalidation messages between the fetch and the last use.
+ *
+ * For the common case the relcache entry's attribute bitmaps are already
+ * computed (rd_attrsvalid is true). When they aren't, we go through
+ * RelationGetIndexAttrBitmap to populate the cache (which costs one
+ * throwaway bms_copy on first use) and then return the cached pointer on
+ * the second pass. The first-use path is rare and never on the bench hot
+ * path, so the simplicity is preferred over open-coding the populate-only
+ * variant.
+ */
+const Bitmapset *
+RelationGetIndexAttrBitmapNoCopy(Relation relation, IndexAttrBitmapKind attrKind)
+{
+ if (!relation->rd_attrsvalid)
+ {
+ Bitmapset *populated;
+
+ /* Populate rd_*attr fields; discard the returned copy. */
+ populated = RelationGetIndexAttrBitmap(relation, attrKind);
+ bms_free(populated);
+
+ /*
+ * If the relation has no indexes, RelationGetIndexAttrBitmap returns
+ * NULL without setting rd_attrsvalid. Mirror that here.
+ */
+ if (!relation->rd_attrsvalid)
+ return NULL;
+ }
+
+ switch (attrKind)
+ {
+ case INDEX_ATTR_BITMAP_KEY:
+ return relation->rd_keyattr;
+ case INDEX_ATTR_BITMAP_PRIMARY_KEY:
+ return relation->rd_pkattr;
+ case INDEX_ATTR_BITMAP_IDENTITY_KEY:
+ return relation->rd_idattr;
+ case INDEX_ATTR_BITMAP_INDEXED:
+ return relation->rd_indexedattr;
+ case INDEX_ATTR_BITMAP_SUMMARIZED:
+ return relation->rd_summarizedattr;
+ case INDEX_ATTR_BITMAP_EXPRESSION:
+ return relation->rd_exprindexattr;
default:
elog(ERROR, "unknown attrKind %u", attrKind);
return NULL;
@@ -6508,6 +6742,7 @@ load_relcache_init_file(bool shared)
rel->rd_partcheckcxt = NULL;
rel->rd_indexprs = NIL;
rel->rd_indpred = NIL;
+ rel->rd_indattr = NULL;
rel->rd_exclops = NULL;
rel->rd_exclprocs = NULL;
rel->rd_exclstrats = NULL;
diff --git a/src/include/access/amapi.h b/src/include/access/amapi.h
index 79240333530..17c76e37cde 100644
--- a/src/include/access/amapi.h
+++ b/src/include/access/amapi.h
@@ -29,7 +29,6 @@ typedef struct IndexPath IndexPath;
/* Likewise, this file shouldn't depend on execnodes.h. */
typedef struct IndexInfo IndexInfo;
-
/*
* Properties for amproperty API. This list covers properties known to the
* core code, but an index AM can define its own properties, by matching the
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 2dbfad92113..23676ae71e9 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -384,11 +384,47 @@ extern TM_Result heap_delete(Relation relation, const ItemPointerData *tid,
bool wait, TM_FailureData *tmfd);
extern void heap_finish_speculative(Relation relation, const ItemPointerData *tid);
extern void heap_abort_speculative(Relation relation, const ItemPointerData *tid);
+
+/*
+ * HeapUpdateIndexMode --
+ * Three-valued classification returned by HeapUpdateHotAllowable() that
+ * tells heap_update() whether a HOT update is permitted for this tuple and,
+ * if so, whether the indexes may be maintained selectively.
+ *
+ * HEAP_UPDATE_ALL_INDEXES
+ * HOT is not allowed; the new tuple must go on its own TID and every
+ * index receives a fresh entry. This is the classic pre-HOT-indexed
+ * behavior for updates that modify a non-summarizing indexed attribute.
+ *
+ * HEAP_HEAP_ONLY_UPDATE
+ * Classic HOT update: no non-summarizing indexed attribute changed (only
+ * summarizing ones, if any), so no index needs a new entry.
+ *
+ * HEAP_SELECTIVE_INDEX_UPDATE
+ * HOT with selective index update: at least one non-summarizing index's
+ * attribute changed, but the new tuple can still join the HOT chain on
+ * the same page; only the indexes whose attributes changed receive a new
+ * entry. As for classic HOT, heap_update() still falls back to a
+ * non-HOT update if the new tuple does not fit on the page.
+ *
+ * Callers should spell the exact mode they care about. HEAP_UPDATE_ALL_INDEXES
+ * is the zero/false value and doubles as a distinguished "no HOT" sentinel
+ * (e.g. testing hot_mode != HEAP_UPDATE_ALL_INDEXES), but the values are not
+ * meaningful as a numeric ordering.
+ */
+typedef enum HeapUpdateIndexMode
+{
+ HEAP_UPDATE_ALL_INDEXES = 0,
+ HEAP_HEAP_ONLY_UPDATE = 1,
+ HEAP_SELECTIVE_INDEX_UPDATE = 2,
+} HeapUpdateIndexMode;
+
extern TM_Result heap_update(Relation relation, const ItemPointerData *otid,
HeapTuple newtup, CommandId cid, uint32 options,
Snapshot crosscheck, bool wait,
TM_FailureData *tmfd, const LockTupleMode lockmode,
- const Bitmapset *modified_idx_attrs, const bool hot_allowed);
+ const Bitmapset *modified_idx_attrs,
+ HeapUpdateIndexMode hot_mode);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy,
bool follow_updates,
@@ -423,7 +459,7 @@ extern bool heap_tuple_needs_eventual_freeze(HeapTupleHeader tuple);
extern void simple_heap_insert(Relation relation, HeapTuple tup);
extern void simple_heap_delete(Relation relation, const ItemPointerData *tid);
extern void simple_heap_update(Relation relation, const ItemPointerData *otid,
- HeapTuple tup, TU_UpdateIndexes *update_indexes);
+ HeapTuple tup, bool *update_all_indexes);
extern TransactionId heap_index_delete_tuples(Relation rel,
TM_IndexDeleteOp *delstate);
@@ -434,7 +470,10 @@ extern void heapam_index_fetch_reset(IndexFetchTableData *scan);
extern void heapam_index_fetch_end(IndexFetchTableData *scan);
extern bool heap_hot_search_buffer(ItemPointer tid, Relation relation,
Buffer buffer, Snapshot snapshot, HeapTuple heapTuple,
- bool *all_dead, bool first_call);
+ bool *all_dead, bool first_call,
+ bool *hot_indexed_recheck,
+ uint8 *crossed_bitmap,
+ bool *prefix_all_dead);
extern bool heapam_index_fetch_tuple(struct IndexFetchTableData *scan,
ItemPointer tid, Snapshot snapshot,
TupleTableSlot *slot, bool *heap_continue,
@@ -464,8 +503,9 @@ extern void log_heap_prune_and_freeze(Relation relation, Buffer buffer,
OffsetNumber *unused, int nunused);
/* in heap/heapam.c */
-extern bool HeapUpdateHotAllowable(Relation relation, const Bitmapset *modified_idx_attrs,
- bool *summarized_only);
+
+extern HeapUpdateIndexMode HeapUpdateHotAllowable(Relation relation,
+ const Bitmapset *modified_idx_attrs);
extern LockTupleMode HeapUpdateDetermineLockmode(Relation relation,
const Bitmapset *modified_idx_attrs);
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 2ea06a67a63..fe4469178aa 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -134,6 +134,45 @@ typedef struct IndexFetchTableData
* permitted.
*/
uint32 flags;
+
+ /*
+ * Side channel for table AMs whose update chains can reach a different
+ * set of index-key values than the arriving index entry recorded (heap's
+ * HOT-selectively-updated chains). Set true by the table AM when the
+ * walk to the live tuple crossed a HOT/SIU hop after the entry's own
+ * tuple, meaning the arriving entry's stored key may no longer match the
+ * live tuple and the index-access layer must recheck it. Left false when
+ * no such hop was crossed (the entry is definitely current), and always
+ * false for AMs without such chains.
+ */
+ bool xs_hot_indexed_recheck;
+
+ /*
+ * Companion to xs_hot_indexed_recheck. xs_hot_indexed_crossed is the
+ * union of the per-hop modified-attrs bitmaps the walk crossed after the
+ * entry's own tuple, over heap attribute numbers (bit attnum-1 for a
+ * 1-based attnum). The index-access layer tests it against the arriving
+ * index's key columns to judge staleness without a key comparison: any
+ * overlap means a crossed hop changed one of the index's inputs, so the
+ * entry is stale. The union is complete (every crossed live hop and
+ * collapse-survivor stub contributes its bitmap, and collapse only
+ * reclaims members subsumed by surviving hops), so disjointness reliably
+ * means fresh. It is NULL for AMs without such chains and is sized by
+ * the table AM for the heap relation's column count.
+ */
+ uint8 *xs_hot_indexed_crossed;
+
+ /*
+ * Set by the table AM when it returns a tuple: true iff every chain
+ * member the walk skipped before reaching the returned (visible) tuple is
+ * dead to all transactions (below the global xmin horizon). Combined
+ * with a stale verdict (the crossed-attribute bitmap overlapped the
+ * index's key columns), this lets the index-access layer
+ * kill the arriving leaf: no snapshot can reach a matching version
+ * through it, so it is redundant. AMs without such chains leave it
+ * false.
+ */
+ bool xs_prefix_all_dead;
} IndexFetchTableData;
struct IndexScanInstrumentation;
@@ -154,6 +193,13 @@ typedef struct IndexScanDescData
struct ScanKeyData *keyData; /* array of index qualifier descriptors */
struct ScanKeyData *orderByData; /* array of ordering op descriptors */
bool xs_want_itup; /* caller requests index tuples */
+ bool xs_index_only; /* caller is an index-only scan that may
+ * return tuples without fetching the heap;
+ * AMs must retain leaf-page pins for such
+ * scans (VM all-visible / TID-recycle race),
+ * whereas a plain scan that sets xs_want_itup
+ * only to inspect the index tuple still
+ * fetches the heap and may drop pins */
bool xs_temp_snap; /* unregister snapshot at scan end? */
/* signaling to index AM about killing index tuples */
@@ -189,6 +235,20 @@ typedef struct IndexScanDescData
bool xs_recheck; /* T means scan keys must be rechecked */
+ /*
+ * T means the index entry that reached xs_heaptid is stale: the HOT chain
+ * walked to reach the tuple crossed a HOT-selectively-updated (HOT/SIU)
+ * hop that changed an attribute this index covers, so the arriving
+ * entry's stored key no longer matches the live tuple. The executor
+ * drops such a tuple; the row is re-supplied by the fresh entry inserted
+ * for the new value. Unlike xs_recheck (set by lossy AMs such as GiST
+ * and GIN), this is computed by the index-access layer by testing the
+ * heap AM's crossed-attribute bitmap (xs_hot_indexed_crossed) against
+ * this index's key columns: any overlap means a crossed hop changed one
+ * of the index's inputs, so the entry is stale.
+ */
+ bool xs_hot_indexed_stale;
+
/*
* When fetching with an ordering operator, the values of the ORDER BY
* expressions of the last returned tuple, according to the index. If
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index a9778b3528d..e7bd9f3e6fc 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -19,6 +19,7 @@
#include "access/relscan.h"
#include "access/sdir.h"
+#include "access/sysattr.h"
#include "access/xact.h"
#include "executor/tuptable.h"
#include "storage/read_stream.h"
@@ -28,6 +29,18 @@
#define DEFAULT_TABLE_ACCESS_METHOD "heap"
+/*
+ * Whole-row sentinel for the in/out modified-attributes set of
+ * table_tuple_update(). On input the caller supplies the indexed attributes
+ * whose values changed. A table AM that stored the new tuple as an
+ * independent version not reachable through the existing index entries (for
+ * heap, a non-HOT update) adds this whole-row attribute (attribute number 0,
+ * FirstLowInvalidHeapAttributeNumber convention) on output, signalling that
+ * every index needs a new entry. Diffing real columns never yields attribute
+ * 0, so it is unambiguous as this sentinel.
+ */
+#define TableTupleUpdateAllIndexes (0 - FirstLowInvalidHeapAttributeNumber)
+
/* GUCs */
extern PGDLLIMPORT char *default_table_access_method;
extern PGDLLIMPORT bool synchronize_seqscans;
@@ -125,22 +138,6 @@ typedef enum TM_Result
TM_WouldBlock,
} TM_Result;
-/*
- * Result codes for table_update(..., update_indexes*..).
- * Used to determine which indexes to update.
- */
-typedef enum TU_UpdateIndexes
-{
- /* No indexed columns were updated (incl. TID addressing of tuple) */
- TU_None,
-
- /* A non-summarizing indexed column was updated, or the TID has changed */
- TU_All,
-
- /* Only summarized columns were updated, TID is unchanged */
- TU_Summarizing,
-} TU_UpdateIndexes;
-
/*
* When table_tuple_update, table_tuple_delete, or table_tuple_lock fail
* because the target tuple is already outdated, they fill in this struct to
@@ -488,6 +485,13 @@ typedef struct TableAmRoutine
* index_fetch_tuple iff it is guaranteed that no backend needs to see
* that tuple. Index AMs can use that to avoid returning that tid in
* future searches.
+ *
+ * If a tuple is returned and the table AM reached it by walking a HOT
+ * chain that crossed a HOT-selectively-updated (HOT/SIU) hop after the
+ * arriving entry's own tuple, it sets scan->xs_hot_indexed_recheck (see
+ * struct IndexFetchTableData) to tell the index-access layer to recheck
+ * the arriving leaf key against the live tuple. AMs without such update
+ * chains leave it false.
*/
bool (*index_fetch_tuple) (struct IndexFetchTableData *scan,
ItemPointer tid,
@@ -586,8 +590,7 @@ typedef struct TableAmRoutine
bool wait,
TM_FailureData *tmfd,
LockTupleMode *lockmode,
- const Bitmapset *modified_idx_attrs,
- TU_UpdateIndexes *update_indexes);
+ Bitmapset **modified_attrs);
/* see table_tuple_lock() for reference about parameters */
TM_Result (*tuple_lock) (Relation rel,
@@ -1319,11 +1322,20 @@ table_index_fetch_tuple(struct IndexFetchTableData *scan,
* returns whether there are table tuple items corresponding to an index
* entry. This likely is only useful to verify if there's a conflict in a
* unique index.
+ *
+ * If keep_slot is non-NULL, on a positive result the function stores the
+ * fetched tuple into *keep_slot (which must be a valid slot of the
+ * relation's type) and returns with the slot populated; the caller is
+ * responsible for clearing the slot. When keep_slot is NULL a temporary
+ * slot is created internally and dropped before return, matching the
+ * pre-existing behaviour.
*/
extern bool table_index_fetch_tuple_check(Relation rel,
ItemPointer tid,
Snapshot snapshot,
- bool *all_dead);
+ bool *all_dead,
+ bool *hot_indexed_recheck_out,
+ TupleTableSlot *keep_slot);
/* ------------------------------------------------------------------------
@@ -1574,12 +1586,20 @@ table_tuple_delete(Relation rel, ItemPointer tid, CommandId cid,
* TABLE_UPDATE_NO_LOGICAL -- force-disables the emitting of logical
* decoding information for the tuple.
*
+ * In parameters:
+ * modified_attrs - in/out; on input, the set of indexed attributes whose
+ * values changed (FirstLowInvalidHeapAttributeNumber convention). A
+ * table AM may use this to choose between HOT and non-HOT storage of the
+ * new tuple. On output the AM adds the whole-row attribute
+ * (TableTupleUpdateAllIndexes) iff it stored the new tuple as an
+ * independent version requiring a fresh entry in every index; otherwise
+ * the caller consults each index's own attributes against this set to
+ * decide per index (the standard HOT / selective-index-update cases).
+ *
* Output parameters:
* slot - newly constructed tuple data to store
* tmfd - filled in failure cases (see below)
* lockmode - filled with lock mode acquired on tuple
- * update_indexes - in success cases this is set if new index entries
- * are required for this tuple; see TU_UpdateIndexes
*
* Normal, successful return value is TM_Ok, which means we did actually
* update it. Failure return codes are TM_SelfModified, TM_Updated, and
@@ -1600,12 +1620,14 @@ table_tuple_update(Relation rel, ItemPointer otid, TupleTableSlot *slot,
CommandId cid, uint32 options,
Snapshot snapshot, Snapshot crosscheck,
bool wait, TM_FailureData *tmfd, LockTupleMode *lockmode,
- const Bitmapset *modified_idx_attrs, TU_UpdateIndexes *update_indexes)
+ Bitmapset **modified_attrs)
{
+ Assert(modified_attrs == NULL ||
+ !bms_is_member(TableTupleUpdateAllIndexes, *modified_attrs));
return rel->rd_tableam->tuple_update(rel, otid, slot,
cid, options, snapshot, crosscheck,
wait, tmfd, lockmode,
- modified_idx_attrs, update_indexes);
+ modified_attrs);
}
/*
@@ -2090,8 +2112,7 @@ extern void simple_table_tuple_delete(Relation rel, ItemPointer tid,
Snapshot snapshot);
extern void simple_table_tuple_update(Relation rel, ItemPointer otid,
TupleTableSlot *slot, Snapshot snapshot,
- const Bitmapset *modified_idx_attrs,
- TU_UpdateIndexes *update_indexes);
+ Bitmapset **modified_attrs);
/* ----------------------------------------------------------------------------
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 16661bc66d9..f7211ececff 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -18,7 +18,6 @@
#include "datatype/timestamp.h"
#include "executor/execdesc.h"
#include "fmgr.h"
-#include "nodes/execnodes.h"
#include "nodes/lockoptions.h"
#include "nodes/parsenodes.h"
#include "utils/memutils.h"
@@ -755,11 +754,13 @@ extern Bitmapset *ExecGetAllUpdatedCols(ResultRelInfo *relinfo, EState *estate);
*/
extern void ExecOpenIndices(ResultRelInfo *resultRelInfo, bool speculative);
extern void ExecCloseIndices(ResultRelInfo *resultRelInfo);
+extern void ExecSetIndexUnchanged(ResultRelInfo *resultRelInfo,
+ const Bitmapset *modified_idx_attrs);
/* flags for ExecInsertIndexTuples */
#define EIIT_IS_UPDATE (1<<0)
#define EIIT_NO_DUPE_ERROR (1<<1)
-#define EIIT_ONLY_SUMMARIZING (1<<2)
+#define EIIT_IS_HOT_INDEXED (1<<2)
extern List *ExecInsertIndexTuples(ResultRelInfo *resultRelInfo, EState *estate,
uint32 flags, TupleTableSlot *slot,
List *arbiterIndexes,
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index e64fd8c7ea3..2e3712205a4 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -216,10 +216,18 @@ typedef struct IndexInfo
bool ii_NullsNotDistinct;
/* is it valid for inserts? */
bool ii_ReadyForInserts;
- /* IndexUnchanged status determined yet? */
- bool ii_CheckedUnchanged;
- /* aminsert hint, cached for retail inserts */
+ /*
+ * aminsert hint: index logically unchanged by UPDATE? Narrow rule: key
+ * columns only; INCLUDE columns and the partial-index predicate are not
+ * considered (expression indexes are treated conservatively).
+ */
bool ii_IndexUnchanged;
+ /*
+ * selective UPDATE: does this index need a new entry? Wide rule: true if
+ * any key, INCLUDE, expression, or predicate column it references changed
+ * (or the AM stored an independent new version).
+ */
+ bool ii_IndexNeedsUpdate;
/* are we doing a concurrent index build? */
bool ii_Concurrent;
/* did we detect any broken HOT chains? */
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..28b79370f0d 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -151,7 +151,19 @@ typedef struct PgStat_TableCounts
PgStat_Counter tuples_updated;
PgStat_Counter tuples_deleted;
PgStat_Counter tuples_hot_updated;
+ PgStat_Counter tuples_hot_indexed_updated;
PgStat_Counter tuples_newpage_updated;
+
+ /*
+ * Per-index HOT-indexed update counters. Maintained on pgstat entries
+ * keyed on an index oid, not on the owning table's entry. They count how
+ * many HOT-indexed updates skipped this index (key unchanged) vs.
+ * inserted a fresh entry (key changed). Summarizing indexes do not
+ * contribute to either counter.
+ */
+ PgStat_Counter tuples_hot_indexed_upd_skipped;
+ PgStat_Counter tuples_hot_indexed_upd_matched;
+
bool truncdropped;
PgStat_Counter delta_live_tuples;
@@ -218,7 +230,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBC
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCBD
typedef struct PgStat_ArchiverStats
{
@@ -460,8 +472,13 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter tuples_updated;
PgStat_Counter tuples_deleted;
PgStat_Counter tuples_hot_updated;
+ PgStat_Counter tuples_hot_indexed_updated;
PgStat_Counter tuples_newpage_updated;
+ /* Per-index HOT-indexed update counters (see PgStat_TableCounts). */
+ PgStat_Counter tuples_hot_indexed_upd_skipped;
+ PgStat_Counter tuples_hot_indexed_upd_matched;
+
PgStat_Counter live_tuples;
PgStat_Counter dead_tuples;
PgStat_Counter mod_since_analyze;
@@ -752,6 +769,16 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.tuples_returned += (n); \
} while (0)
+#define pgstat_count_hot_indexed_upd_skipped(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.tuples_hot_indexed_upd_skipped++;\
+ } while (0)
+#define pgstat_count_hot_indexed_upd_matched(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.tuples_hot_indexed_upd_matched++;\
+ } while (0)
#define pgstat_count_buffer_read(rel) \
do { \
if (pgstat_should_count_relation(rel)) \
@@ -764,7 +791,7 @@ extern void pgstat_report_analyze(Relation rel,
} while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
-extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
+extern void pgstat_count_heap_update(Relation rel, bool hot, bool hot_indexed, bool newpage);
extern void pgstat_count_heap_delete(Relation rel);
extern void pgstat_count_truncate(Relation rel);
extern void pgstat_update_heap_dead_tuples(Relation rel, int delta);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 17dbdbe645d..304783f0bbc 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -164,6 +164,7 @@ typedef struct RelationData
Bitmapset *rd_idattr; /* included in replica identity index */
Bitmapset *rd_indexedattr; /* all cols referenced by indexes */
Bitmapset *rd_summarizedattr; /* cols indexed by summarizing indexes */
+ Bitmapset *rd_exprindexattr; /* cols referenced by expression indexes */
PublicationDesc *rd_pubdesc; /* publication descriptor, or NULL */
@@ -217,6 +218,16 @@ typedef struct RelationData
Oid *rd_indcollation; /* OIDs of index collations */
bytea **rd_opcoptions; /* parsed opclass-specific options */
+ /*
+ * Bitmap of heap attribute numbers referenced by this index (simple keys,
+ * INCLUDE columns, expression columns, and partial-index predicate
+ * columns), offset by FirstLowInvalidHeapAttributeNumber. Lazily built by
+ * RelationGetIndexedAttrs() and cached in rd_indexcxt. Consumers must
+ * bms_copy before relying on the pointer beyond any potential
+ * AcceptInvalidationMessages() call.
+ */
+ Bitmapset *rd_indattr;
+
/*
* rd_amcache is available for index and table AMs to cache private data
* about the relation. This must be just a cache since it may get reset
diff --git a/src/include/utils/relcache.h b/src/include/utils/relcache.h
index 89788091576..a381aa3e095 100644
--- a/src/include/utils/relcache.h
+++ b/src/include/utils/relcache.h
@@ -62,6 +62,19 @@ extern List *RelationGetDummyIndexExpressions(Relation relation);
extern List *RelationGetIndexPredicate(Relation relation);
extern bytea **RelationGetIndexAttOptions(Relation relation, bool copy);
+/*
+ * RelationGetIndexedAttrs -- return a freshly-palloc'd Bitmapset of every
+ * heap attribute this index references, via keys, INCLUDE columns,
+ * expressions, or partial-index predicates.
+ *
+ * The argument must be an index Relation (not its owning heap). Attribute
+ * numbers are offset by FirstLowInvalidHeapAttributeNumber. The result is
+ * palloc'd in the caller's context; bms_free when done. The relcache
+ * caches its own copy in rd_indexcxt so subsequent calls only pay for the
+ * final bms_copy.
+ */
+extern Bitmapset *RelationGetIndexedAttrs(Relation indexRel);
+
/*
* Which set of columns to return by RelationGetIndexAttrBitmap.
*/
@@ -72,11 +85,15 @@ typedef enum IndexAttrBitmapKind
INDEX_ATTR_BITMAP_IDENTITY_KEY,
INDEX_ATTR_BITMAP_INDEXED,
INDEX_ATTR_BITMAP_SUMMARIZED,
+ INDEX_ATTR_BITMAP_EXPRESSION,
} IndexAttrBitmapKind;
extern Bitmapset *RelationGetIndexAttrBitmap(Relation relation,
IndexAttrBitmapKind attrKind);
+extern const Bitmapset *RelationGetIndexAttrBitmapNoCopy(Relation relation,
+ IndexAttrBitmapKind attrKind);
+
extern Bitmapset *RelationGetIdentityKeyBitmap(Relation relation);
extern void RelationGetExclusionInfo(Relation indexRelation,
diff --git a/src/test/regress/expected/hot_updates.out b/src/test/regress/expected/hot_updates.out
index 273fe3310da..06979ed31d0 100644
--- a/src/test/regress/expected/hot_updates.out
+++ b/src/test/regress/expected/hot_updates.out
@@ -1,147 +1,132 @@
--
-- HOT_UPDATES
--- Test Heap-Only Tuple (HOT) update decisions
+-- Test classic Heap-Only Tuple (HOT) update decisions
--
--- This test systematically verifies that HOT updates are used when appropriate
--- and avoided when necessary (e.g., when indexed columns are modified).
+-- This file covers HOT decisions that apply identically on a pre-hot-indexed
+-- server: every UPDATE here either leaves all indexed attributes
+-- unchanged or touches only summarizing-index (BRIN) attributes, so the
+-- HOT vs non-HOT choice does not depend on whether Selective Index
+-- Update (hot-indexed) is enabled. hot-indexed-specific behaviour (UPDATEs that modify
+-- a non-summarizing indexed attribute) is covered in
+-- hot_indexed_updates.sql.
--
--- We use multiple validation methods:
--- 1. Statistics functions (pg_stat_get_tuples_hot_updated)
--- 2. pageinspect extension for HOT chain examination
--- 3. EXPLAIN to verify index usage after updates
+-- Validation methods:
+-- 1. Statistics (pg_stat_get_tuples_hot_updated)
+-- 2. pageinspect for HOT chain structure
+-- 3. EXPLAIN to confirm the planner still picks the index
--
-- Load required extensions
CREATE EXTENSION IF NOT EXISTS pageinspect;
--- Function to get HOT update count
+-- Sum of committed and in-progress (non-HOT, HOT) update counters.
CREATE OR REPLACE FUNCTION get_hot_count(rel_name text)
RETURNS TABLE (
updates BIGINT,
hot BIGINT
) AS $$
DECLARE
- rel_oid oid;
+ rel_oid oid;
BEGIN
- rel_oid := rel_name::regclass::oid;
-
- -- Read both committed and transaction-local stats
- -- In autocommit mode (default for regression tests), this works correctly
- -- Note: In explicit transactions (BEGIN/COMMIT), committed stats already
- -- include flushed updates, so this would double-count. For explicit
- -- transaction testing, call pg_stat_force_next_flush() before this function.
- updates := COALESCE(pg_stat_get_tuples_updated(rel_oid), 0) +
- COALESCE(pg_stat_get_xact_tuples_updated(rel_oid), 0);
- hot := COALESCE(pg_stat_get_tuples_hot_updated(rel_oid), 0) +
- COALESCE(pg_stat_get_xact_tuples_hot_updated(rel_oid), 0);
-
- RETURN NEXT;
+ rel_oid := rel_name::regclass::oid;
+ updates := COALESCE(pg_stat_get_tuples_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_updated(rel_oid), 0);
+ hot := COALESCE(pg_stat_get_tuples_hot_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_hot_updated(rel_oid), 0);
+ RETURN NEXT;
END;
$$ LANGUAGE plpgsql;
--- Check if a tuple is part of a HOT chain (has a predecessor on same page)
+-- True iff target_ctid is the TAIL of a HOT chain on the same page.
CREATE OR REPLACE FUNCTION has_hot_chain(rel_name text, target_ctid tid)
RETURNS boolean AS $$
DECLARE
- block_num int;
- page_item record;
+ block_num int;
+ page_item record;
BEGIN
- block_num := (target_ctid::text::point)[0]::int;
-
- -- Look for a different tuple on the same page that points to our target tuple
- FOR page_item IN
- SELECT lp, lp_flags, t_ctid
- FROM heap_page_items(get_raw_page(rel_name, block_num))
- WHERE lp_flags = 1
- AND t_ctid IS NOT NULL
- AND t_ctid = target_ctid
- AND ('(' || block_num::text || ',' || lp::text || ')')::tid != target_ctid
- LOOP
- RETURN true;
- END LOOP;
-
- RETURN false;
+ block_num := (target_ctid::text::point)[0]::int;
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp_flags = 1
+ AND t_ctid IS NOT NULL
+ AND t_ctid = target_ctid
+ AND ('(' || block_num::text || ',' || lp::text || ')')::tid != target_ctid
+ LOOP
+ RETURN true;
+ END LOOP;
+ RETURN false;
END;
$$ LANGUAGE plpgsql;
--- Print the HOT chain starting from a given tuple
+-- Emit the HOT chain rooted at start_ctid.
CREATE OR REPLACE FUNCTION print_hot_chain(rel_name text, start_ctid tid)
RETURNS TABLE(chain_position int, ctid tid, lp_flags text, t_ctid tid, chain_end boolean) AS
$$
#variable_conflict use_column
DECLARE
- block_num int;
- line_ptr int;
- current_ctid tid := start_ctid;
- next_ctid tid;
- position int := 0;
- max_iterations int := 100;
- page_item record;
- found_predecessor boolean := false;
- flags_name text;
+ block_num int;
+ line_ptr int;
+ current_ctid tid := start_ctid;
+ next_ctid tid;
+ position int := 0;
+ max_iterations int := 100;
+ page_item record;
+ found_predecessor boolean := false;
+ flags_name text;
BEGIN
- block_num := (start_ctid::text::point)[0]::int;
-
- -- Find the predecessor (old tuple pointing to our start_ctid)
- FOR page_item IN
- SELECT lp, lp_flags, t_ctid
- FROM heap_page_items(get_raw_page(rel_name, block_num))
- WHERE lp_flags = 1
- AND t_ctid = start_ctid
- LOOP
- current_ctid := ('(' || block_num::text || ',' || page_item.lp::text || ')')::tid;
- found_predecessor := true;
- EXIT;
- END LOOP;
-
- -- If no predecessor found, start with the given ctid
- IF NOT found_predecessor THEN
- current_ctid := start_ctid;
- END IF;
-
- -- Follow the chain forward
- WHILE position < max_iterations LOOP
- line_ptr := (current_ctid::text::point)[1]::int;
+ block_num := (start_ctid::text::point)[0]::int;
FOR page_item IN
- SELECT lp, lp_flags, t_ctid
- FROM heap_page_items(get_raw_page(rel_name, block_num))
- WHERE lp = line_ptr
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp_flags = 1
+ AND t_ctid = start_ctid
LOOP
- -- Map lp_flags to names
- flags_name := CASE page_item.lp_flags
- WHEN 0 THEN 'unused (0)'
- WHEN 1 THEN 'normal (1)'
- WHEN 2 THEN 'redirect (2)'
- WHEN 3 THEN 'dead (3)'
- ELSE 'unknown (' || page_item.lp_flags::text || ')'
- END;
-
- RETURN QUERY SELECT
- position,
- current_ctid,
- flags_name,
- page_item.t_ctid,
- (page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid)::boolean
- ;
-
- IF page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid THEN
- RETURN;
- END IF;
-
- next_ctid := page_item.t_ctid;
-
- IF (next_ctid::text::point)[0]::int != block_num THEN
- RETURN;
- END IF;
-
- current_ctid := next_ctid;
- position := position + 1;
+ current_ctid := ('(' || block_num::text || ',' || page_item.lp::text || ')')::tid;
+ found_predecessor := true;
+ EXIT;
END LOOP;
-
- IF position = 0 THEN
- RETURN;
+ IF NOT found_predecessor THEN
+ current_ctid := start_ctid;
END IF;
- END LOOP;
+
+ WHILE position < max_iterations LOOP
+ line_ptr := (current_ctid::text::point)[1]::int;
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp = line_ptr
+ LOOP
+ flags_name := CASE page_item.lp_flags
+ WHEN 0 THEN 'unused (0)'
+ WHEN 1 THEN 'normal (1)'
+ WHEN 2 THEN 'redirect (2)'
+ WHEN 3 THEN 'dead (3)'
+ ELSE 'unknown (' || page_item.lp_flags::text || ')'
+ END;
+ RETURN QUERY SELECT
+ position,
+ current_ctid,
+ flags_name,
+ page_item.t_ctid,
+ (page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid)::boolean;
+
+ IF page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid THEN
+ RETURN;
+ END IF;
+ next_ctid := page_item.t_ctid;
+ IF (next_ctid::text::point)[0]::int != block_num THEN
+ RETURN;
+ END IF;
+ current_ctid := next_ctid;
+ position := position + 1;
+ END LOOP;
+ IF position = 0 THEN
+ RETURN;
+ END IF;
+ END LOOP;
END;
$$ LANGUAGE plpgsql;
--- Basic HOT update (update non-indexed column)
+-- ---------------------------------------------------------------------------
+-- 1. Basic HOT: update of a non-indexed column
+-- ---------------------------------------------------------------------------
CREATE TABLE hot_test (
id int PRIMARY KEY,
indexed_col int,
@@ -151,239 +136,218 @@ CREATE INDEX hot_test_indexed_idx ON hot_test(indexed_col);
INSERT INTO hot_test VALUES (1, 100, 'initial');
INSERT INTO hot_test VALUES (2, 200, 'initial');
INSERT INTO hot_test VALUES (3, 300, 'initial');
--- Get baseline
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
SELECT * FROM get_hot_count('hot_test');
updates | hot
---------+-----
0 | 0
(1 row)
--- Should be HOT updates (only non-indexed column modified)
+-- Three classic HOT updates (non-indexed col).
UPDATE hot_test SET non_indexed_col = 'updated1' WHERE id = 1;
UPDATE hot_test SET non_indexed_col = 'updated2' WHERE id = 2;
UPDATE hot_test SET non_indexed_col = 'updated3' WHERE id = 3;
--- Verify HOT updates occurred
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
SELECT * FROM get_hot_count('hot_test');
updates | hot
---------+-----
3 | 3
(1 row)
--- Dump the HOT chain before VACUUMing
-WITH current_tuple AS (
- SELECT ctid FROM hot_test WHERE id = 1
-)
-SELECT
- has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
- chain_position,
- print_hot_chain.ctid,
- lp_flags,
- t_ctid
-FROM current_tuple,
-LATERAL print_hot_chain('hot_test', current_tuple.ctid);
+-- Chain-of-1 on id=1 still has a predecessor line pointer.
+WITH current_tuple AS (SELECT ctid FROM hot_test WHERE id = 1)
+SELECT has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position, print_hot_chain.ctid, lp_flags, t_ctid
+FROM current_tuple, LATERAL print_hot_chain('hot_test', current_tuple.ctid);
has_chain | chain_position | ctid | lp_flags | t_ctid
-----------+----------------+-------+------------+--------
t | 0 | (0,1) | normal (1) | (0,4)
t | 1 | (0,4) | normal (1) | (0,4)
(2 rows)
--- Vacuum the relation, expect the HOT chain to collapse
+-- VACUUM collapses the chain.
VACUUM hot_test;
--- Show that there is no chain after vacuum
-WITH current_tuple AS (
- SELECT ctid FROM hot_test WHERE id = 1
-)
-SELECT
- has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
- chain_position,
- print_hot_chain.ctid,
- lp_flags,
- t_ctid
-FROM current_tuple,
-LATERAL print_hot_chain('hot_test', current_tuple.ctid);
+WITH current_tuple AS (SELECT ctid FROM hot_test WHERE id = 1)
+SELECT has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position, print_hot_chain.ctid, lp_flags, t_ctid
+FROM current_tuple, LATERAL print_hot_chain('hot_test', current_tuple.ctid);
has_chain | chain_position | ctid | lp_flags | t_ctid
-----------+----------------+-------+------------+--------
f | 0 | (0,4) | normal (1) | (0,4)
(1 row)
--- Non-HOT update (update indexed column)
-UPDATE hot_test SET indexed_col = 150 WHERE id = 1;
+DROP TABLE hot_test;
+-- ---------------------------------------------------------------------------
+-- 2. Summarizing indexes (BRIN) do not block HOT
+-- ---------------------------------------------------------------------------
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ ts timestamp,
+ value int,
+ brin_col int
+) WITH (fillfactor = 50);
+CREATE INDEX hot_test_ts_brin ON hot_test USING brin(ts);
+CREATE INDEX hot_test_brin_col_brin ON hot_test USING brin(brin_col);
+INSERT INTO hot_test VALUES (1, '2024-01-01', 100, 1000);
+-- BRIN columns are summarizing; updating them stays classic HOT even
+-- though their values change.
+UPDATE hot_test SET ts = '2024-01-02', brin_col = 2000 WHERE id = 1;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
SELECT * FROM get_hot_count('hot_test');
updates | hot
---------+-----
- 4 | 3
+ 1 | 1
(1 row)
--- Verify index was updated (new value findable)
-SET enable_seqscan = off;
-EXPLAIN (COSTS OFF) SELECT id, indexed_col FROM hot_test WHERE indexed_col = 150;
- QUERY PLAN
----------------------------------------------------
- Index Scan using hot_test_indexed_idx on hot_test
- Index Cond: (indexed_col = 150)
-(2 rows)
-
-SELECT id, indexed_col FROM hot_test WHERE indexed_col = 150;
- id | indexed_col
-----+-------------
- 1 | 150
+-- Non-indexed column: also HOT.
+UPDATE hot_test SET value = 200 WHERE id = 1;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
(1 row)
--- Verify old value no longer in index
-EXPLAIN (COSTS OFF) SELECT id FROM hot_test WHERE indexed_col = 100;
- QUERY PLAN
----------------------------------------------------
- Index Scan using hot_test_indexed_idx on hot_test
- Index Cond: (indexed_col = 100)
-(2 rows)
-
-SELECT id FROM hot_test WHERE indexed_col = 100;
- id
-----
-(0 rows)
+SELECT * FROM get_hot_count('hot_test');
+ updates | hot
+---------+-----
+ 2 | 2
+(1 row)
-RESET enable_seqscan;
--- All-or-none property: updating one indexed column requires ALL index updates
DROP TABLE hot_test;
+-- ---------------------------------------------------------------------------
+-- 3. TOAST participates in HOT (non-indexed column paths only)
+-- ---------------------------------------------------------------------------
CREATE TABLE hot_test (
id int PRIMARY KEY,
- col_a int,
- col_b int,
- col_c int,
- non_indexed text
+ indexed_col int,
+ large_text text,
+ small_text text
) WITH (fillfactor = 50);
-CREATE INDEX hot_test_a_idx ON hot_test(col_a);
-CREATE INDEX hot_test_b_idx ON hot_test(col_b);
-CREATE INDEX hot_test_c_idx ON hot_test(col_c);
-INSERT INTO hot_test VALUES (1, 10, 20, 30, 'initial');
--- Update only col_a - should NOT be HOT because an indexed column changed
--- This means ALL indexes must be updated (all-or-none property)
-UPDATE hot_test SET col_a = 15 WHERE id = 1;
+CREATE INDEX hot_test_idx ON hot_test(indexed_col);
+INSERT INTO hot_test VALUES (1, 100, repeat('x', 3000), 'small');
+-- Non-indexed, non-TOAST column: HOT.
+UPDATE hot_test SET small_text = 'updated';
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
SELECT * FROM get_hot_count('hot_test');
updates | hot
---------+-----
- 1 | 0
+ 1 | 1
+(1 row)
+
+-- TOAST column, indexed_col unchanged: HOT.
+UPDATE hot_test SET large_text = repeat('y', 3000);
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
(1 row)
--- Now update only non-indexed column - should be HOT
-UPDATE hot_test SET non_indexed = 'updated';
SELECT * FROM get_hot_count('hot_test');
updates | hot
---------+-----
- 2 | 1
+ 2 | 2
(1 row)
--- Partial index: both old and new outside predicate (conservative = non-HOT)
DROP TABLE hot_test;
+-- ---------------------------------------------------------------------------
+-- 4. Partial index where update leaves indexed attrs unchanged
+-- ---------------------------------------------------------------------------
CREATE TABLE hot_test (
id int PRIMARY KEY,
status text,
data text
) WITH (fillfactor = 50);
--- Partial index only covers status = 'active'
CREATE INDEX hot_test_active_idx ON hot_test(status) WHERE status = 'active';
INSERT INTO hot_test VALUES (1, 'active', 'data1');
INSERT INTO hot_test VALUES (2, 'inactive', 'data2');
INSERT INTO hot_test VALUES (3, 'deleted', 'data3');
--- Update non-indexed column on 'active' row (in predicate, status unchanged)
--- Should be HOT
+-- Update data on a row whose status matches the partial predicate: HOT.
UPDATE hot_test SET data = 'updated1' WHERE id = 1;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
SELECT * FROM get_hot_count('hot_test');
updates | hot
---------+-----
1 | 1
(1 row)
--- Update non-indexed column on 'inactive' row (outside predicate)
--- Should be HOT
+-- Update data on a row outside the predicate: HOT.
UPDATE hot_test SET data = 'updated2' WHERE id = 2;
-SELECT * FROM get_hot_count('hot_test');
- updates | hot
----------+-----
- 2 | 2
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
(1 row)
--- Update status from 'inactive' to 'deleted' (both outside predicate)
--- PostgreSQL is conservative: heap insert happens before predicate check
--- So this is NON-HOT even though both values are outside predicate
-UPDATE hot_test SET status = 'deleted' WHERE id = 2;
SELECT * FROM get_hot_count('hot_test');
updates | hot
---------+-----
- 3 | 2
+ 2 | 2
(1 row)
--- Verify index still works for 'active' rows
SELECT id, status FROM hot_test WHERE status = 'active';
id | status
----+--------
1 | active
(1 row)
--- Only BRIN (summarizing) indexes on non-PK columns
DROP TABLE hot_test;
+-- ---------------------------------------------------------------------------
+-- 5. Multi-column btree: update of non-indexed column
+-- ---------------------------------------------------------------------------
CREATE TABLE hot_test (
id int PRIMARY KEY,
- ts timestamp,
- value int,
- brin_col int
-) WITH (fillfactor = 50);
-CREATE INDEX hot_test_ts_brin ON hot_test USING brin(ts);
-CREATE INDEX hot_test_brin_col_brin ON hot_test USING brin(brin_col);
-INSERT INTO hot_test VALUES (1, '2024-01-01', 100, 1000);
--- Update both BRIN columns - should still be HOT (only summarizing indexes)
-UPDATE hot_test SET ts = '2024-01-02', brin_col = 2000 WHERE id = 1;
-SELECT * FROM get_hot_count('hot_test');
- updates | hot
----------+-----
- 1 | 1
-(1 row)
-
--- Update non-indexed column - should also be HOT
-UPDATE hot_test SET value = 200 WHERE id = 1;
-SELECT * FROM get_hot_count('hot_test');
- updates | hot
----------+-----
- 2 | 2
-(1 row)
-
--- TOAST and HOT: TOASTed columns can participate in HOT
-DROP TABLE hot_test;
-CREATE TABLE hot_test (
- id int PRIMARY KEY,
- indexed_col int,
- large_text text,
- small_text text
+ col_a int,
+ col_b int,
+ col_c int,
+ data text
) WITH (fillfactor = 50);
-CREATE INDEX hot_test_idx ON hot_test(indexed_col);
--- Insert row with TOASTed column (> 2KB)
-INSERT INTO hot_test VALUES (1, 100, repeat('x', 3000), 'small');
--- Update non-indexed, non-TOASTed column - should be HOT
-UPDATE hot_test SET small_text = 'updated';
-SELECT * FROM get_hot_count('hot_test');
- updates | hot
----------+-----
- 1 | 1
+CREATE INDEX hot_test_ab_idx ON hot_test(col_a, col_b);
+INSERT INTO hot_test VALUES (1, 10, 20, 30, 'data');
+-- col_c not in any index: HOT.
+UPDATE hot_test SET col_c = 35;
+-- data not in any index: HOT.
+UPDATE hot_test SET data = 'updated';
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
(1 row)
--- Update TOASTed column - should be HOT if indexed column unchanged
-UPDATE hot_test SET large_text = repeat('y', 3000);
SELECT * FROM get_hot_count('hot_test');
updates | hot
---------+-----
2 | 2
(1 row)
--- Update indexed column - should NOT be HOT
-UPDATE hot_test SET indexed_col = 200;
-SELECT * FROM get_hot_count('hot_test');
- updates | hot
----------+-----
- 3 | 2
-(1 row)
-
--- Unique constraint (unique index) behaves like regular index
DROP TABLE hot_test;
+-- ---------------------------------------------------------------------------
+-- 6. Unique index: update of non-indexed column + uniqueness enforcement
+-- ---------------------------------------------------------------------------
CREATE TABLE hot_test (
id int PRIMARY KEY,
unique_col int UNIQUE,
@@ -391,15 +355,19 @@ CREATE TABLE hot_test (
) WITH (fillfactor = 50);
INSERT INTO hot_test VALUES (1, 100, 'data1');
INSERT INTO hot_test VALUES (2, 200, 'data2');
--- Update data (non-indexed) - should be HOT
UPDATE hot_test SET data = 'updated';
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
SELECT * FROM get_hot_count('hot_test');
updates | hot
---------+-----
2 | 2
(1 row)
--- Verify unique constraint still enforced
SELECT id, unique_col, data FROM hot_test ORDER BY id;
id | unique_col | data
----+------------+---------
@@ -407,60 +375,14 @@ SELECT id, unique_col, data FROM hot_test ORDER BY id;
2 | 200 | updated
(2 rows)
--- This should fail (unique violation)
+-- Unique constraint still enforced on any path.
UPDATE hot_test SET unique_col = 100 WHERE id = 2;
ERROR: duplicate key value violates unique constraint "hot_test_unique_col_key"
DETAIL: Key (unique_col)=(100) already exists.
--- Multi-column index: any column change = non-HOT
DROP TABLE hot_test;
-CREATE TABLE hot_test (
- id int PRIMARY KEY,
- col_a int,
- col_b int,
- col_c int,
- data text
-) WITH (fillfactor = 50);
-CREATE INDEX hot_test_ab_idx ON hot_test(col_a, col_b);
-INSERT INTO hot_test VALUES (1, 10, 20, 30, 'data');
--- Update col_a (part of multi-column index) - should NOT be HOT
-UPDATE hot_test SET col_a = 15;
-SELECT * FROM get_hot_count('hot_test');
- updates | hot
----------+-----
- 1 | 0
-(1 row)
-
--- Reset
-UPDATE hot_test SET col_a = 10;
--- Update col_b (part of multi-column index) - should NOT be HOT
-UPDATE hot_test SET col_b = 25;
-SELECT * FROM get_hot_count('hot_test');
- updates | hot
----------+-----
- 3 | 0
-(1 row)
-
--- Reset
-UPDATE hot_test SET col_b = 20;
-SELECT * FROM get_hot_count('hot_test');
- updates | hot
----------+-----
- 4 | 0
-(1 row)
-
--- Update col_c (not indexed) - should be HOT
-UPDATE hot_test SET col_c = 35;
--- Update data (not indexed) - should be HOT
-UPDATE hot_test SET data = 'updated';
-SELECT * FROM get_hot_count('hot_test');
- updates | hot
----------+-----
- 6 | 2
-(1 row)
-
--- Partitioned tables: HOT works within partitions
-DROP TABLE IF EXISTS hot_test_partitioned CASCADE;
-NOTICE: table "hot_test_partitioned" does not exist, skipping
+-- ---------------------------------------------------------------------------
+-- 7. Partitioned tables: HOT within a partition
+-- ---------------------------------------------------------------------------
CREATE TABLE hot_test_partitioned (
id int,
partition_key int,
@@ -475,23 +397,32 @@ CREATE TABLE hot_test_part2 PARTITION OF hot_test_partitioned
CREATE INDEX hot_test_part_idx ON hot_test_partitioned(indexed_col);
INSERT INTO hot_test_partitioned VALUES (1, 50, 100, 'initial1');
INSERT INTO hot_test_partitioned VALUES (2, 150, 200, 'initial2');
--- Update in partition 1 (non-indexed column) - should be HOT
UPDATE hot_test_partitioned SET data = 'updated1' WHERE id = 1;
--- Update in partition 2 (non-indexed column) - should be HOT
UPDATE hot_test_partitioned SET data = 'updated2' WHERE id = 2;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
SELECT * FROM get_hot_count('hot_test_part1');
updates | hot
---------+-----
1 | 1
(1 row)
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
SELECT * FROM get_hot_count('hot_test_part2');
updates | hot
---------+-----
1 | 1
(1 row)
--- Verify indexes work on partitions
SELECT id FROM hot_test_partitioned WHERE indexed_col = 100;
id
----
@@ -504,242 +435,100 @@ SELECT id FROM hot_test_partitioned WHERE indexed_col = 200;
2
(1 row)
--- Update indexed column in partition - should NOT be HOT
-UPDATE hot_test_partitioned SET indexed_col = 150 WHERE id = 1;
-SELECT * FROM get_hot_count('hot_test_part1');
- updates | hot
----------+-----
- 2 | 1
-(1 row)
-
--- Verify index was updated
-SELECT id FROM hot_test_partitioned WHERE indexed_col = 150;
- id
-----
- 1
-(1 row)
-
--- ============================================================================
--- Trigger modifications: heap_modify_tuple() and HOT
--- ============================================================================
--- Test that we correctly detect when triggers modify indexed columns via
--- heap_modify_tuple(), even when those columns aren't in the UPDATE's SET clause
-CREATE TABLE hot_trigger_test (
- id int PRIMARY KEY,
- triggered_col int,
- data text
-) WITH (fillfactor = 50);
-CREATE INDEX hot_trigger_idx ON hot_trigger_test(triggered_col);
--- Create a trigger that modifies an indexed column
-CREATE OR REPLACE FUNCTION modify_triggered_col()
-RETURNS TRIGGER AS $$
-BEGIN
- NEW.triggered_col = NEW.triggered_col + 1;
- RETURN NEW;
-END;
-$$ LANGUAGE plpgsql;
-CREATE TRIGGER before_update_modify
- BEFORE UPDATE ON hot_trigger_test
- FOR EACH ROW
- EXECUTE FUNCTION modify_triggered_col();
-INSERT INTO hot_trigger_test VALUES (1, 100, 'initial');
-SELECT * FROM get_hot_count('hot_trigger_test');
- updates | hot
----------+-----
- 0 | 0
-(1 row)
-
--- Update only data column, but trigger modifies indexed column
--- Should NOT be HOT because trigger modified an indexed column
-UPDATE hot_trigger_test SET data = 'updated' WHERE id = 1;
--- Verify it was NOT a HOT update (indexed column was modified by trigger)
-SELECT * FROM get_hot_count('hot_trigger_test');
- updates | hot
----------+-----
- 1 | 0
-(1 row)
-
--- Verify the triggered column was actually modified
-SELECT triggered_col FROM hot_trigger_test WHERE id = 1;
- triggered_col
----------------
- 101
-(1 row)
-
-DROP TABLE hot_trigger_test CASCADE;
-DROP FUNCTION modify_triggered_col();
--- ============================================================================
--- JSONB expression indexes and sub-attribute tracking
--- ============================================================================
--- Test that updates to non-indexed JSONB paths can be HOT updates
+DROP TABLE hot_test_partitioned CASCADE;
+-- ---------------------------------------------------------------------------
+-- 8. JSONB expression index: non-indexed path change is HOT
+-- ---------------------------------------------------------------------------
CREATE TABLE hot_jsonb_test (
id int PRIMARY KEY,
data jsonb
) WITH (fillfactor = 50);
--- Create expression index on a specific JSON path
CREATE INDEX hot_jsonb_name_idx ON hot_jsonb_test ((data->>'name'));
INSERT INTO hot_jsonb_test VALUES
(1, '{"name":"Alice","age":30,"city":"NYC"}'),
(2, '{"name":"Bob","age":25,"city":"LA"}');
-SELECT * FROM get_hot_count('hot_jsonb_test');
- updates | hot
----------+-----
- 0 | 0
+-- The jsonb column is the expression index's input, so HOT-indexed is
+-- disqualified (expression indexes are not yet supported) and the jsonb
+-- change blocks classic HOT: non-HOT update.
+UPDATE hot_jsonb_test SET data = jsonb_set(data, '{age}', '31') WHERE id = 1;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
(1 row)
--- Update non-indexed JSON path (age) - should be HOT after instrumentation
-UPDATE hot_jsonb_test SET data = jsonb_set(data, '{age}', '31') WHERE id = 1;
SELECT * FROM get_hot_count('hot_jsonb_test');
updates | hot
---------+-----
1 | 0
(1 row)
--- Update indexed JSON path (name) - should NOT be HOT
-UPDATE hot_jsonb_test SET data = jsonb_set(data, '{name}', '"Alice2"') WHERE id = 1;
+-- Likewise non-HOT: expression index disqualifies HOT-indexed.
+UPDATE hot_jsonb_test SET data = data - 'city' WHERE id = 2;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
SELECT * FROM get_hot_count('hot_jsonb_test');
updates | hot
---------+-----
2 | 0
(1 row)
--- Verify index works
-SELECT id FROM hot_jsonb_test WHERE data->>'name' = 'Alice2';
- id
-----
- 1
+-- Likewise non-HOT: expression index disqualifies HOT-indexed.
+UPDATE hot_jsonb_test SET data = jsonb_insert(data, '{country}', '"USA"') WHERE id = 2;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
(1 row)
--- Test jsonb_delete on non-indexed path - should be HOT after instrumentation
-UPDATE hot_jsonb_test SET data = data - 'city' WHERE id = 2;
SELECT * FROM get_hot_count('hot_jsonb_test');
updates | hot
---------+-----
3 | 0
(1 row)
--- Test jsonb_insert on non-indexed path - should be HOT after instrumentation
-UPDATE hot_jsonb_test SET data = jsonb_insert(data, '{country}', '"USA"') WHERE id = 2;
-SELECT * FROM get_hot_count('hot_jsonb_test');
- updates | hot
----------+-----
- 4 | 0
-(1 row)
-
DROP TABLE hot_jsonb_test;
--- ============================================================================
--- XML expression indexes and sub-attribute tracking
--- ============================================================================
--- Test that updates to non-indexed XML paths can be HOT updates
-CREATE TABLE hot_xml_test (
- id int PRIMARY KEY,
- doc xml
-) WITH (fillfactor = 50);
--- Create expression index on a specific XPath
-CREATE INDEX hot_xml_name_idx ON hot_xml_test ((xpath('/person/name/text()', doc)));
-INSERT INTO hot_xml_test VALUES
- (1, '<person><name>Alice</name><age>30</age></person>'),
- (2, '<person><name>Bob</name><age>25</age></person>');
-ERROR: could not identify a comparison function for type xml
-SELECT * FROM get_hot_count('hot_xml_test');
- updates | hot
----------+-----
- 0 | 0
-(1 row)
-
--- Update non-indexed XPath (age) - behavior depends on XML comparison fallback
--- Full XML value replacement means non-indexed path updates still require index comparison
-UPDATE hot_xml_test SET doc = '<person><name>Alice</name><age>31</age></person>' WHERE id = 1;
-SELECT * FROM get_hot_count('hot_xml_test');
- updates | hot
----------+-----
- 0 | 0
-(1 row)
-
--- Update indexed XPath (name) - should NOT be HOT
-UPDATE hot_xml_test SET doc = '<person><name>Alice2</name><age>31</age></person>' WHERE id = 1;
-SELECT * FROM get_hot_count('hot_xml_test');
- updates | hot
----------+-----
- 0 | 0
-(1 row)
-
--- Verify index works
-SELECT id FROM hot_xml_test WHERE xpath('/person/name/text()', doc) = ARRAY['Alice2'::text];
-ERROR: operator does not exist: xml[] = text[]
-LINE 1: ..._xml_test WHERE xpath('/person/name/text()', doc) = ARRAY['A...
- ^
-DETAIL: No operator of that name accepts the given argument types.
-HINT: You might need to add explicit type casts.
-DROP TABLE hot_xml_test;
--- ============================================================================
--- GIN indexes and amcomparedatums for JSONB
--- ============================================================================
--- Test that GIN indexes can use amcomparedatums to enable HOT when extracted keys match
+-- ---------------------------------------------------------------------------
+-- 9. A change to a GIN-indexed column is HOT-indexed
+--
+-- The read side filters a stale leaf via the crossed-attribute bitmap, which
+-- is access-method agnostic, so a GIN-covered column is HOT-indexed like any
+-- other: only the GIN index is maintained, and a GIN scan (which rechecks on
+-- the heap) returns correct results across the chain.
+-- ---------------------------------------------------------------------------
CREATE TABLE hot_gin_test (
id int PRIMARY KEY,
tags text[],
properties jsonb
) WITH (fillfactor = 50);
--- GIN index on text array
CREATE INDEX hot_gin_tags_idx ON hot_gin_test USING gin (tags);
--- GIN index on JSONB (jsonb_ops - keys and values)
CREATE INDEX hot_gin_props_idx ON hot_gin_test USING gin (properties);
INSERT INTO hot_gin_test VALUES
(1, ARRAY['tag1', 'tag2'], '{"key1":"val1","key2":"val2"}'),
(2, ARRAY['tag3', 'tag4'], '{"key3":"val3","key4":"val4"}');
-SELECT * FROM get_hot_count('hot_gin_test');
- updates | hot
----------+-----
- 0 | 0
-(1 row)
-
--- Update that changes tag order but not content - after amcomparedatums should be HOT
--- (GIN extracts same keys, just different order)
+-- Reorder tags: a GIN-covered column changes, so this is HOT-indexed.
UPDATE hot_gin_test SET tags = ARRAY['tag2', 'tag1'] WHERE id = 1;
-SELECT * FROM get_hot_count('hot_gin_test');
- updates | hot
----------+-----
- 1 | 0
-(1 row)
-
--- Update JSONB value (not key) - after amcomparedatums may be HOT or non-HOT
--- depending on GIN operator class (jsonb_ops indexes both keys and values)
-UPDATE hot_gin_test SET properties = '{"key1":"val1_new","key2":"val2"}' WHERE id = 1;
-SELECT * FROM get_hot_count('hot_gin_test');
- updates | hot
----------+-----
- 2 | 0
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
(1 row)
--- Add new tag - should NOT be HOT (different extracted keys)
-UPDATE hot_gin_test SET tags = ARRAY['tag2', 'tag1', 'tag5'] WHERE id = 1;
SELECT * FROM get_hot_count('hot_gin_test');
updates | hot
---------+-----
- 3 | 0
-(1 row)
-
--- Verify GIN indexes work
-SELECT id FROM hot_gin_test WHERE tags @> ARRAY['tag5'];
- id
-----
- 1
-(1 row)
-
-SELECT id FROM hot_gin_test WHERE properties @> '{"key1":"val1_new"}';
- id
-----
- 1
+ 1 | 1
(1 row)
DROP TABLE hot_gin_test;
--- ============================================================================
+-- ---------------------------------------------------------------------------
-- Cleanup
--- ============================================================================
-DROP TABLE IF EXISTS hot_test;
-DROP TABLE IF EXISTS hot_test_partitioned CASCADE;
-DROP FUNCTION IF EXISTS has_hot_chain(text, tid);
-DROP FUNCTION IF EXISTS print_hot_chain(text, tid);
-DROP FUNCTION IF EXISTS get_hot_count(text);
+-- ---------------------------------------------------------------------------
+DROP FUNCTION has_hot_chain(text, tid);
+DROP FUNCTION print_hot_chain(text, tid);
+DROP FUNCTION get_hot_count(text);
DROP EXTENSION pageinspect;
diff --git a/src/test/regress/sql/hot_updates.sql b/src/test/regress/sql/hot_updates.sql
index a8894006177..35ce7e1cdcd 100644
--- a/src/test/regress/sql/hot_updates.sql
+++ b/src/test/regress/sql/hot_updates.sql
@@ -1,354 +1,258 @@
--
-- HOT_UPDATES
--- Test Heap-Only Tuple (HOT) update decisions
+-- Test classic Heap-Only Tuple (HOT) update decisions
--
--- This test systematically verifies that HOT updates are used when appropriate
--- and avoided when necessary (e.g., when indexed columns are modified).
+-- This file covers HOT decisions that apply identically on a pre-hot-indexed
+-- server: every UPDATE here either leaves all indexed attributes
+-- unchanged or touches only summarizing-index (BRIN) attributes, so the
+-- HOT vs non-HOT choice does not depend on whether Selective Index
+-- Update (hot-indexed) is enabled. hot-indexed-specific behaviour (UPDATEs that modify
+-- a non-summarizing indexed attribute) is covered in
+-- hot_indexed_updates.sql.
--
--- We use multiple validation methods:
--- 1. Statistics functions (pg_stat_get_tuples_hot_updated)
--- 2. pageinspect extension for HOT chain examination
--- 3. EXPLAIN to verify index usage after updates
+-- Validation methods:
+-- 1. Statistics (pg_stat_get_tuples_hot_updated)
+-- 2. pageinspect for HOT chain structure
+-- 3. EXPLAIN to confirm the planner still picks the index
--
-- Load required extensions
CREATE EXTENSION IF NOT EXISTS pageinspect;
--- Function to get HOT update count
+-- Sum of committed and in-progress (non-HOT, HOT) update counters.
CREATE OR REPLACE FUNCTION get_hot_count(rel_name text)
RETURNS TABLE (
updates BIGINT,
hot BIGINT
) AS $$
DECLARE
- rel_oid oid;
+ rel_oid oid;
BEGIN
- rel_oid := rel_name::regclass::oid;
-
- -- Read both committed and transaction-local stats
- -- In autocommit mode (default for regression tests), this works correctly
- -- Note: In explicit transactions (BEGIN/COMMIT), committed stats already
- -- include flushed updates, so this would double-count. For explicit
- -- transaction testing, call pg_stat_force_next_flush() before this function.
- updates := COALESCE(pg_stat_get_tuples_updated(rel_oid), 0) +
- COALESCE(pg_stat_get_xact_tuples_updated(rel_oid), 0);
- hot := COALESCE(pg_stat_get_tuples_hot_updated(rel_oid), 0) +
- COALESCE(pg_stat_get_xact_tuples_hot_updated(rel_oid), 0);
-
- RETURN NEXT;
+ rel_oid := rel_name::regclass::oid;
+ updates := COALESCE(pg_stat_get_tuples_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_updated(rel_oid), 0);
+ hot := COALESCE(pg_stat_get_tuples_hot_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_hot_updated(rel_oid), 0);
+ RETURN NEXT;
END;
$$ LANGUAGE plpgsql;
--- Check if a tuple is part of a HOT chain (has a predecessor on same page)
+-- True iff target_ctid is the TAIL of a HOT chain on the same page.
CREATE OR REPLACE FUNCTION has_hot_chain(rel_name text, target_ctid tid)
RETURNS boolean AS $$
DECLARE
- block_num int;
- page_item record;
+ block_num int;
+ page_item record;
BEGIN
- block_num := (target_ctid::text::point)[0]::int;
-
- -- Look for a different tuple on the same page that points to our target tuple
- FOR page_item IN
- SELECT lp, lp_flags, t_ctid
- FROM heap_page_items(get_raw_page(rel_name, block_num))
- WHERE lp_flags = 1
- AND t_ctid IS NOT NULL
- AND t_ctid = target_ctid
- AND ('(' || block_num::text || ',' || lp::text || ')')::tid != target_ctid
- LOOP
- RETURN true;
- END LOOP;
-
- RETURN false;
+ block_num := (target_ctid::text::point)[0]::int;
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp_flags = 1
+ AND t_ctid IS NOT NULL
+ AND t_ctid = target_ctid
+ AND ('(' || block_num::text || ',' || lp::text || ')')::tid != target_ctid
+ LOOP
+ RETURN true;
+ END LOOP;
+ RETURN false;
END;
$$ LANGUAGE plpgsql;
--- Print the HOT chain starting from a given tuple
+-- Emit the HOT chain rooted at start_ctid.
CREATE OR REPLACE FUNCTION print_hot_chain(rel_name text, start_ctid tid)
RETURNS TABLE(chain_position int, ctid tid, lp_flags text, t_ctid tid, chain_end boolean) AS
$$
#variable_conflict use_column
DECLARE
- block_num int;
- line_ptr int;
- current_ctid tid := start_ctid;
- next_ctid tid;
- position int := 0;
- max_iterations int := 100;
- page_item record;
- found_predecessor boolean := false;
- flags_name text;
+ block_num int;
+ line_ptr int;
+ current_ctid tid := start_ctid;
+ next_ctid tid;
+ position int := 0;
+ max_iterations int := 100;
+ page_item record;
+ found_predecessor boolean := false;
+ flags_name text;
BEGIN
- block_num := (start_ctid::text::point)[0]::int;
-
- -- Find the predecessor (old tuple pointing to our start_ctid)
- FOR page_item IN
- SELECT lp, lp_flags, t_ctid
- FROM heap_page_items(get_raw_page(rel_name, block_num))
- WHERE lp_flags = 1
- AND t_ctid = start_ctid
- LOOP
- current_ctid := ('(' || block_num::text || ',' || page_item.lp::text || ')')::tid;
- found_predecessor := true;
- EXIT;
- END LOOP;
-
- -- If no predecessor found, start with the given ctid
- IF NOT found_predecessor THEN
- current_ctid := start_ctid;
- END IF;
-
- -- Follow the chain forward
- WHILE position < max_iterations LOOP
- line_ptr := (current_ctid::text::point)[1]::int;
+ block_num := (start_ctid::text::point)[0]::int;
FOR page_item IN
- SELECT lp, lp_flags, t_ctid
- FROM heap_page_items(get_raw_page(rel_name, block_num))
- WHERE lp = line_ptr
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp_flags = 1
+ AND t_ctid = start_ctid
LOOP
- -- Map lp_flags to names
- flags_name := CASE page_item.lp_flags
- WHEN 0 THEN 'unused (0)'
- WHEN 1 THEN 'normal (1)'
- WHEN 2 THEN 'redirect (2)'
- WHEN 3 THEN 'dead (3)'
- ELSE 'unknown (' || page_item.lp_flags::text || ')'
- END;
-
- RETURN QUERY SELECT
- position,
- current_ctid,
- flags_name,
- page_item.t_ctid,
- (page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid)::boolean
- ;
-
- IF page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid THEN
- RETURN;
- END IF;
-
- next_ctid := page_item.t_ctid;
-
- IF (next_ctid::text::point)[0]::int != block_num THEN
- RETURN;
- END IF;
-
- current_ctid := next_ctid;
- position := position + 1;
+ current_ctid := ('(' || block_num::text || ',' || page_item.lp::text || ')')::tid;
+ found_predecessor := true;
+ EXIT;
END LOOP;
-
- IF position = 0 THEN
- RETURN;
+ IF NOT found_predecessor THEN
+ current_ctid := start_ctid;
END IF;
- END LOOP;
+
+ WHILE position < max_iterations LOOP
+ line_ptr := (current_ctid::text::point)[1]::int;
+ FOR page_item IN
+ SELECT lp, lp_flags, t_ctid
+ FROM heap_page_items(get_raw_page(rel_name, block_num))
+ WHERE lp = line_ptr
+ LOOP
+ flags_name := CASE page_item.lp_flags
+ WHEN 0 THEN 'unused (0)'
+ WHEN 1 THEN 'normal (1)'
+ WHEN 2 THEN 'redirect (2)'
+ WHEN 3 THEN 'dead (3)'
+ ELSE 'unknown (' || page_item.lp_flags::text || ')'
+ END;
+ RETURN QUERY SELECT
+ position,
+ current_ctid,
+ flags_name,
+ page_item.t_ctid,
+ (page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid)::boolean;
+
+ IF page_item.t_ctid IS NULL OR page_item.t_ctid = current_ctid THEN
+ RETURN;
+ END IF;
+ next_ctid := page_item.t_ctid;
+ IF (next_ctid::text::point)[0]::int != block_num THEN
+ RETURN;
+ END IF;
+ current_ctid := next_ctid;
+ position := position + 1;
+ END LOOP;
+ IF position = 0 THEN
+ RETURN;
+ END IF;
+ END LOOP;
END;
$$ LANGUAGE plpgsql;
--- Basic HOT update (update non-indexed column)
+
+-- ---------------------------------------------------------------------------
+-- 1. Basic HOT: update of a non-indexed column
+-- ---------------------------------------------------------------------------
CREATE TABLE hot_test (
id int PRIMARY KEY,
indexed_col int,
non_indexed_col text
) WITH (fillfactor = 50);
-
CREATE INDEX hot_test_indexed_idx ON hot_test(indexed_col);
INSERT INTO hot_test VALUES (1, 100, 'initial');
INSERT INTO hot_test VALUES (2, 200, 'initial');
INSERT INTO hot_test VALUES (3, 300, 'initial');
--- Get baseline
+SELECT pg_stat_force_next_flush();
SELECT * FROM get_hot_count('hot_test');
--- Should be HOT updates (only non-indexed column modified)
+-- Three classic HOT updates (non-indexed col).
UPDATE hot_test SET non_indexed_col = 'updated1' WHERE id = 1;
UPDATE hot_test SET non_indexed_col = 'updated2' WHERE id = 2;
UPDATE hot_test SET non_indexed_col = 'updated3' WHERE id = 3;
-
--- Verify HOT updates occurred
-SELECT * FROM get_hot_count('hot_test');
-
--- Dump the HOT chain before VACUUMing
-WITH current_tuple AS (
- SELECT ctid FROM hot_test WHERE id = 1
-)
-SELECT
- has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
- chain_position,
- print_hot_chain.ctid,
- lp_flags,
- t_ctid
-FROM current_tuple,
-LATERAL print_hot_chain('hot_test', current_tuple.ctid);
-
--- Vacuum the relation, expect the HOT chain to collapse
-VACUUM hot_test;
-
--- Show that there is no chain after vacuum
-WITH current_tuple AS (
- SELECT ctid FROM hot_test WHERE id = 1
-)
-SELECT
- has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
- chain_position,
- print_hot_chain.ctid,
- lp_flags,
- t_ctid
-FROM current_tuple,
-LATERAL print_hot_chain('hot_test', current_tuple.ctid);
-
--- Non-HOT update (update indexed column)
-UPDATE hot_test SET indexed_col = 150 WHERE id = 1;
-SELECT * FROM get_hot_count('hot_test');
-
--- Verify index was updated (new value findable)
-SET enable_seqscan = off;
-EXPLAIN (COSTS OFF) SELECT id, indexed_col FROM hot_test WHERE indexed_col = 150;
-SELECT id, indexed_col FROM hot_test WHERE indexed_col = 150;
-
--- Verify old value no longer in index
-EXPLAIN (COSTS OFF) SELECT id FROM hot_test WHERE indexed_col = 100;
-SELECT id FROM hot_test WHERE indexed_col = 100;
-RESET enable_seqscan;
-
--- All-or-none property: updating one indexed column requires ALL index updates
-DROP TABLE hot_test;
-
-CREATE TABLE hot_test (
- id int PRIMARY KEY,
- col_a int,
- col_b int,
- col_c int,
- non_indexed text
-) WITH (fillfactor = 50);
-
-CREATE INDEX hot_test_a_idx ON hot_test(col_a);
-CREATE INDEX hot_test_b_idx ON hot_test(col_b);
-CREATE INDEX hot_test_c_idx ON hot_test(col_c);
-
-INSERT INTO hot_test VALUES (1, 10, 20, 30, 'initial');
-
--- Update only col_a - should NOT be HOT because an indexed column changed
--- This means ALL indexes must be updated (all-or-none property)
-UPDATE hot_test SET col_a = 15 WHERE id = 1;
+SELECT pg_stat_force_next_flush();
SELECT * FROM get_hot_count('hot_test');
--- Now update only non-indexed column - should be HOT
-UPDATE hot_test SET non_indexed = 'updated';
-SELECT * FROM get_hot_count('hot_test');
+-- Chain-of-1 on id=1 still has a predecessor line pointer.
+WITH current_tuple AS (SELECT ctid FROM hot_test WHERE id = 1)
+SELECT has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position, print_hot_chain.ctid, lp_flags, t_ctid
+FROM current_tuple, LATERAL print_hot_chain('hot_test', current_tuple.ctid);
--- Partial index: both old and new outside predicate (conservative = non-HOT)
-DROP TABLE hot_test;
-
-CREATE TABLE hot_test (
- id int PRIMARY KEY,
- status text,
- data text
-) WITH (fillfactor = 50);
-
--- Partial index only covers status = 'active'
-CREATE INDEX hot_test_active_idx ON hot_test(status) WHERE status = 'active';
-
-INSERT INTO hot_test VALUES (1, 'active', 'data1');
-INSERT INTO hot_test VALUES (2, 'inactive', 'data2');
-INSERT INTO hot_test VALUES (3, 'deleted', 'data3');
-
--- Update non-indexed column on 'active' row (in predicate, status unchanged)
--- Should be HOT
-UPDATE hot_test SET data = 'updated1' WHERE id = 1;
-SELECT * FROM get_hot_count('hot_test');
-
--- Update non-indexed column on 'inactive' row (outside predicate)
--- Should be HOT
-UPDATE hot_test SET data = 'updated2' WHERE id = 2;
-SELECT * FROM get_hot_count('hot_test');
+-- VACUUM collapses the chain.
+VACUUM hot_test;
--- Update status from 'inactive' to 'deleted' (both outside predicate)
--- PostgreSQL is conservative: heap insert happens before predicate check
--- So this is NON-HOT even though both values are outside predicate
-UPDATE hot_test SET status = 'deleted' WHERE id = 2;
-SELECT * FROM get_hot_count('hot_test');
+WITH current_tuple AS (SELECT ctid FROM hot_test WHERE id = 1)
+SELECT has_hot_chain('hot_test', current_tuple.ctid) AS has_chain,
+ chain_position, print_hot_chain.ctid, lp_flags, t_ctid
+FROM current_tuple, LATERAL print_hot_chain('hot_test', current_tuple.ctid);
--- Verify index still works for 'active' rows
-SELECT id, status FROM hot_test WHERE status = 'active';
-
--- Only BRIN (summarizing) indexes on non-PK columns
DROP TABLE hot_test;
+-- ---------------------------------------------------------------------------
+-- 2. Summarizing indexes (BRIN) do not block HOT
+-- ---------------------------------------------------------------------------
CREATE TABLE hot_test (
id int PRIMARY KEY,
ts timestamp,
value int,
brin_col int
) WITH (fillfactor = 50);
-
CREATE INDEX hot_test_ts_brin ON hot_test USING brin(ts);
CREATE INDEX hot_test_brin_col_brin ON hot_test USING brin(brin_col);
INSERT INTO hot_test VALUES (1, '2024-01-01', 100, 1000);
--- Update both BRIN columns - should still be HOT (only summarizing indexes)
+-- BRIN columns are summarizing; updating them stays classic HOT even
+-- though their values change.
UPDATE hot_test SET ts = '2024-01-02', brin_col = 2000 WHERE id = 1;
+SELECT pg_stat_force_next_flush();
SELECT * FROM get_hot_count('hot_test');
--- Update non-indexed column - should also be HOT
+-- Non-indexed column: also HOT.
UPDATE hot_test SET value = 200 WHERE id = 1;
+SELECT pg_stat_force_next_flush();
SELECT * FROM get_hot_count('hot_test');
--- TOAST and HOT: TOASTed columns can participate in HOT
DROP TABLE hot_test;
+-- ---------------------------------------------------------------------------
+-- 3. TOAST participates in HOT (non-indexed column paths only)
+-- ---------------------------------------------------------------------------
CREATE TABLE hot_test (
id int PRIMARY KEY,
indexed_col int,
large_text text,
small_text text
) WITH (fillfactor = 50);
-
CREATE INDEX hot_test_idx ON hot_test(indexed_col);
--- Insert row with TOASTed column (> 2KB)
INSERT INTO hot_test VALUES (1, 100, repeat('x', 3000), 'small');
--- Update non-indexed, non-TOASTed column - should be HOT
+-- Non-indexed, non-TOAST column: HOT.
UPDATE hot_test SET small_text = 'updated';
+SELECT pg_stat_force_next_flush();
SELECT * FROM get_hot_count('hot_test');
--- Update TOASTed column - should be HOT if indexed column unchanged
+-- TOAST column, indexed_col unchanged: HOT.
UPDATE hot_test SET large_text = repeat('y', 3000);
+SELECT pg_stat_force_next_flush();
SELECT * FROM get_hot_count('hot_test');
--- Update indexed column - should NOT be HOT
-UPDATE hot_test SET indexed_col = 200;
-SELECT * FROM get_hot_count('hot_test');
-
--- Unique constraint (unique index) behaves like regular index
DROP TABLE hot_test;
+-- ---------------------------------------------------------------------------
+-- 4. Partial index where update leaves indexed attrs unchanged
+-- ---------------------------------------------------------------------------
CREATE TABLE hot_test (
id int PRIMARY KEY,
- unique_col int UNIQUE,
+ status text,
data text
) WITH (fillfactor = 50);
+CREATE INDEX hot_test_active_idx ON hot_test(status) WHERE status = 'active';
-INSERT INTO hot_test VALUES (1, 100, 'data1');
-INSERT INTO hot_test VALUES (2, 200, 'data2');
+INSERT INTO hot_test VALUES (1, 'active', 'data1');
+INSERT INTO hot_test VALUES (2, 'inactive', 'data2');
+INSERT INTO hot_test VALUES (3, 'deleted', 'data3');
--- Update data (non-indexed) - should be HOT
-UPDATE hot_test SET data = 'updated';
+-- Update data on a row whose status matches the partial predicate: HOT.
+UPDATE hot_test SET data = 'updated1' WHERE id = 1;
+SELECT pg_stat_force_next_flush();
SELECT * FROM get_hot_count('hot_test');
--- Verify unique constraint still enforced
-SELECT id, unique_col, data FROM hot_test ORDER BY id;
+-- Update data on a row outside the predicate: HOT.
+UPDATE hot_test SET data = 'updated2' WHERE id = 2;
+SELECT pg_stat_force_next_flush();
+SELECT * FROM get_hot_count('hot_test');
--- This should fail (unique violation)
-UPDATE hot_test SET unique_col = 100 WHERE id = 2;
+SELECT id, status FROM hot_test WHERE status = 'active';
--- Multi-column index: any column change = non-HOT
DROP TABLE hot_test;
+-- ---------------------------------------------------------------------------
+-- 5. Multi-column btree: update of non-indexed column
+-- ---------------------------------------------------------------------------
CREATE TABLE hot_test (
id int PRIMARY KEY,
col_a int,
@@ -356,36 +260,45 @@ CREATE TABLE hot_test (
col_c int,
data text
) WITH (fillfactor = 50);
-
CREATE INDEX hot_test_ab_idx ON hot_test(col_a, col_b);
INSERT INTO hot_test VALUES (1, 10, 20, 30, 'data');
--- Update col_a (part of multi-column index) - should NOT be HOT
-UPDATE hot_test SET col_a = 15;
+-- col_c not in any index: HOT.
+UPDATE hot_test SET col_c = 35;
+-- data not in any index: HOT.
+UPDATE hot_test SET data = 'updated';
+SELECT pg_stat_force_next_flush();
SELECT * FROM get_hot_count('hot_test');
--- Reset
-UPDATE hot_test SET col_a = 10;
-
--- Update col_b (part of multi-column index) - should NOT be HOT
-UPDATE hot_test SET col_b = 25;
-SELECT * FROM get_hot_count('hot_test');
+DROP TABLE hot_test;
--- Reset
-UPDATE hot_test SET col_b = 20;
-SELECT * FROM get_hot_count('hot_test');
+-- ---------------------------------------------------------------------------
+-- 6. Unique index: update of non-indexed column + uniqueness enforcement
+-- ---------------------------------------------------------------------------
+CREATE TABLE hot_test (
+ id int PRIMARY KEY,
+ unique_col int UNIQUE,
+ data text
+) WITH (fillfactor = 50);
--- Update col_c (not indexed) - should be HOT
-UPDATE hot_test SET col_c = 35;
+INSERT INTO hot_test VALUES (1, 100, 'data1');
+INSERT INTO hot_test VALUES (2, 200, 'data2');
--- Update data (not indexed) - should be HOT
UPDATE hot_test SET data = 'updated';
+SELECT pg_stat_force_next_flush();
SELECT * FROM get_hot_count('hot_test');
--- Partitioned tables: HOT works within partitions
-DROP TABLE IF EXISTS hot_test_partitioned CASCADE;
+SELECT id, unique_col, data FROM hot_test ORDER BY id;
+-- Unique constraint still enforced on any path.
+UPDATE hot_test SET unique_col = 100 WHERE id = 2;
+
+DROP TABLE hot_test;
+
+-- ---------------------------------------------------------------------------
+-- 7. Partitioned tables: HOT within a partition
+-- ---------------------------------------------------------------------------
CREATE TABLE hot_test_partitioned (
id int,
partition_key int,
@@ -404,202 +317,82 @@ CREATE INDEX hot_test_part_idx ON hot_test_partitioned(indexed_col);
INSERT INTO hot_test_partitioned VALUES (1, 50, 100, 'initial1');
INSERT INTO hot_test_partitioned VALUES (2, 150, 200, 'initial2');
--- Update in partition 1 (non-indexed column) - should be HOT
UPDATE hot_test_partitioned SET data = 'updated1' WHERE id = 1;
-
--- Update in partition 2 (non-indexed column) - should be HOT
UPDATE hot_test_partitioned SET data = 'updated2' WHERE id = 2;
+SELECT pg_stat_force_next_flush();
SELECT * FROM get_hot_count('hot_test_part1');
+SELECT pg_stat_force_next_flush();
SELECT * FROM get_hot_count('hot_test_part2');
--- Verify indexes work on partitions
SELECT id FROM hot_test_partitioned WHERE indexed_col = 100;
SELECT id FROM hot_test_partitioned WHERE indexed_col = 200;
--- Update indexed column in partition - should NOT be HOT
-UPDATE hot_test_partitioned SET indexed_col = 150 WHERE id = 1;
-SELECT * FROM get_hot_count('hot_test_part1');
-
--- Verify index was updated
-SELECT id FROM hot_test_partitioned WHERE indexed_col = 150;
-
--- ============================================================================
--- Trigger modifications: heap_modify_tuple() and HOT
--- ============================================================================
--- Test that we correctly detect when triggers modify indexed columns via
--- heap_modify_tuple(), even when those columns aren't in the UPDATE's SET clause
-
-CREATE TABLE hot_trigger_test (
- id int PRIMARY KEY,
- triggered_col int,
- data text
-) WITH (fillfactor = 50);
-
-CREATE INDEX hot_trigger_idx ON hot_trigger_test(triggered_col);
-
--- Create a trigger that modifies an indexed column
-CREATE OR REPLACE FUNCTION modify_triggered_col()
-RETURNS TRIGGER AS $$
-BEGIN
- NEW.triggered_col = NEW.triggered_col + 1;
- RETURN NEW;
-END;
-$$ LANGUAGE plpgsql;
-
-CREATE TRIGGER before_update_modify
- BEFORE UPDATE ON hot_trigger_test
- FOR EACH ROW
- EXECUTE FUNCTION modify_triggered_col();
-
-INSERT INTO hot_trigger_test VALUES (1, 100, 'initial');
-
-SELECT * FROM get_hot_count('hot_trigger_test');
-
--- Update only data column, but trigger modifies indexed column
--- Should NOT be HOT because trigger modified an indexed column
-UPDATE hot_trigger_test SET data = 'updated' WHERE id = 1;
-
--- Verify it was NOT a HOT update (indexed column was modified by trigger)
-SELECT * FROM get_hot_count('hot_trigger_test');
-
--- Verify the triggered column was actually modified
-SELECT triggered_col FROM hot_trigger_test WHERE id = 1;
-
-DROP TABLE hot_trigger_test CASCADE;
-DROP FUNCTION modify_triggered_col();
-
--- ============================================================================
--- JSONB expression indexes and sub-attribute tracking
--- ============================================================================
--- Test that updates to non-indexed JSONB paths can be HOT updates
+DROP TABLE hot_test_partitioned CASCADE;
+-- ---------------------------------------------------------------------------
+-- 8. JSONB expression index: non-indexed path change is HOT
+-- ---------------------------------------------------------------------------
CREATE TABLE hot_jsonb_test (
id int PRIMARY KEY,
data jsonb
) WITH (fillfactor = 50);
-
--- Create expression index on a specific JSON path
CREATE INDEX hot_jsonb_name_idx ON hot_jsonb_test ((data->>'name'));
INSERT INTO hot_jsonb_test VALUES
(1, '{"name":"Alice","age":30,"city":"NYC"}'),
(2, '{"name":"Bob","age":25,"city":"LA"}');
-SELECT * FROM get_hot_count('hot_jsonb_test');
-
--- Update non-indexed JSON path (age) - should be HOT after instrumentation
+-- The jsonb column is the expression index's input, so HOT-indexed is
+-- disqualified (expression indexes are not yet supported) and the jsonb
+-- change blocks classic HOT: non-HOT update.
UPDATE hot_jsonb_test SET data = jsonb_set(data, '{age}', '31') WHERE id = 1;
-
-SELECT * FROM get_hot_count('hot_jsonb_test');
-
--- Update indexed JSON path (name) - should NOT be HOT
-UPDATE hot_jsonb_test SET data = jsonb_set(data, '{name}', '"Alice2"') WHERE id = 1;
-
+SELECT pg_stat_force_next_flush();
SELECT * FROM get_hot_count('hot_jsonb_test');
--- Verify index works
-SELECT id FROM hot_jsonb_test WHERE data->>'name' = 'Alice2';
-
--- Test jsonb_delete on non-indexed path - should be HOT after instrumentation
+-- Likewise non-HOT: expression index disqualifies HOT-indexed.
UPDATE hot_jsonb_test SET data = data - 'city' WHERE id = 2;
-
+SELECT pg_stat_force_next_flush();
SELECT * FROM get_hot_count('hot_jsonb_test');
--- Test jsonb_insert on non-indexed path - should be HOT after instrumentation
+-- Likewise non-HOT: expression index disqualifies HOT-indexed.
UPDATE hot_jsonb_test SET data = jsonb_insert(data, '{country}', '"USA"') WHERE id = 2;
-
+SELECT pg_stat_force_next_flush();
SELECT * FROM get_hot_count('hot_jsonb_test');
DROP TABLE hot_jsonb_test;
--- ============================================================================
--- XML expression indexes and sub-attribute tracking
--- ============================================================================
--- Test that updates to non-indexed XML paths can be HOT updates
-
-CREATE TABLE hot_xml_test (
- id int PRIMARY KEY,
- doc xml
-) WITH (fillfactor = 50);
-
--- Create expression index on a specific XPath
-CREATE INDEX hot_xml_name_idx ON hot_xml_test ((xpath('/person/name/text()', doc)));
-
-INSERT INTO hot_xml_test VALUES
- (1, '<person><name>Alice</name><age>30</age></person>'),
- (2, '<person><name>Bob</name><age>25</age></person>');
-
-SELECT * FROM get_hot_count('hot_xml_test');
-
--- Update non-indexed XPath (age) - behavior depends on XML comparison fallback
--- Full XML value replacement means non-indexed path updates still require index comparison
-UPDATE hot_xml_test SET doc = '<person><name>Alice</name><age>31</age></person>' WHERE id = 1;
-
-SELECT * FROM get_hot_count('hot_xml_test');
-
--- Update indexed XPath (name) - should NOT be HOT
-UPDATE hot_xml_test SET doc = '<person><name>Alice2</name><age>31</age></person>' WHERE id = 1;
-
-SELECT * FROM get_hot_count('hot_xml_test');
-
--- Verify index works
-SELECT id FROM hot_xml_test WHERE xpath('/person/name/text()', doc) = ARRAY['Alice2'::text];
-
-DROP TABLE hot_xml_test;
-
--- ============================================================================
--- GIN indexes and amcomparedatums for JSONB
--- ============================================================================
--- Test that GIN indexes can use amcomparedatums to enable HOT when extracted keys match
-
+-- ---------------------------------------------------------------------------
+-- 9. A change to a GIN-indexed column is HOT-indexed
+--
+-- The read side filters a stale leaf via the crossed-attribute bitmap, which
+-- is access-method agnostic, so a GIN-covered column is HOT-indexed like any
+-- other: only the GIN index is maintained, and a GIN scan (which rechecks on
+-- the heap) returns correct results across the chain.
+-- ---------------------------------------------------------------------------
CREATE TABLE hot_gin_test (
id int PRIMARY KEY,
tags text[],
properties jsonb
) WITH (fillfactor = 50);
-
--- GIN index on text array
CREATE INDEX hot_gin_tags_idx ON hot_gin_test USING gin (tags);
-
--- GIN index on JSONB (jsonb_ops - keys and values)
CREATE INDEX hot_gin_props_idx ON hot_gin_test USING gin (properties);
INSERT INTO hot_gin_test VALUES
(1, ARRAY['tag1', 'tag2'], '{"key1":"val1","key2":"val2"}'),
(2, ARRAY['tag3', 'tag4'], '{"key3":"val3","key4":"val4"}');
-SELECT * FROM get_hot_count('hot_gin_test');
-
--- Update that changes tag order but not content - after amcomparedatums should be HOT
--- (GIN extracts same keys, just different order)
+-- Reorder tags: a GIN-covered column changes, so this is HOT-indexed.
UPDATE hot_gin_test SET tags = ARRAY['tag2', 'tag1'] WHERE id = 1;
-
-SELECT * FROM get_hot_count('hot_gin_test');
-
--- Update JSONB value (not key) - after amcomparedatums may be HOT or non-HOT
--- depending on GIN operator class (jsonb_ops indexes both keys and values)
-UPDATE hot_gin_test SET properties = '{"key1":"val1_new","key2":"val2"}' WHERE id = 1;
-
+SELECT pg_stat_force_next_flush();
SELECT * FROM get_hot_count('hot_gin_test');
--- Add new tag - should NOT be HOT (different extracted keys)
-UPDATE hot_gin_test SET tags = ARRAY['tag2', 'tag1', 'tag5'] WHERE id = 1;
-
-SELECT * FROM get_hot_count('hot_gin_test');
-
--- Verify GIN indexes work
-SELECT id FROM hot_gin_test WHERE tags @> ARRAY['tag5'];
-SELECT id FROM hot_gin_test WHERE properties @> '{"key1":"val1_new"}';
-
DROP TABLE hot_gin_test;
--- ============================================================================
+-- ---------------------------------------------------------------------------
-- Cleanup
--- ============================================================================
-DROP TABLE IF EXISTS hot_test;
-DROP TABLE IF EXISTS hot_test_partitioned CASCADE;
-DROP FUNCTION IF EXISTS has_hot_chain(text, tid);
-DROP FUNCTION IF EXISTS print_hot_chain(text, tid);
-DROP FUNCTION IF EXISTS get_hot_count(text);
+-- ---------------------------------------------------------------------------
+DROP FUNCTION has_hot_chain(text, tid);
+DROP FUNCTION print_hot_chain(text, tid);
+DROP FUNCTION get_hot_count(text);
DROP EXTENSION pageinspect;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index ffb413ab612..0bc1d514341 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1277,6 +1277,7 @@ HeapTupleFreeze
HeapTupleHeader
HeapTupleHeaderData
HeapTupleTableSlot
+HeapUpdateIndexMode
HistControl
HostCacheEntry
HostsFileLoadResult
@@ -3138,7 +3139,6 @@ TSVectorStat
TState
TStatus
TStoreState
-TU_UpdateIndexes
TXNEntryFile
TYPCATEGORY
T_Action
--
2.50.1
[text/x-patch] v57-0005-Collapse-dead-HOT-indexed-chains-to-xid-free-stu.patch (54.1K, ../[email protected]/6-v57-0005-Collapse-dead-HOT-indexed-chains-to-xid-free-stu.patch)
download | inline diff:
From b9a09164e183ff332f91dacd12857d53a0bc8655 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Wed, 17 Jun 2026 21:28:33 -0400
Subject: [PATCH v57 5/9] Collapse dead HOT-indexed chains to xid-free stubs on
prune/vacuum
A HOT-indexed update plants index entries that point at mid-chain heap-only
tuples, so a dead chain member cannot simply be removed: a not-yet-swept index
entry may still arrive at it, and the per-hop modified-attrs bitmap on it is
what a reader unions to judge staleness.
Teach prune to collapse a dead chain prefix into xid-free forwarding stubs:
each preserved dead key tuple is rewritten in place to a stub (frozen,
natts == 0, HEAP_INDEXED_UPDATED, forwarding via t_ctid.offnum) that keeps its
segment's modified-attrs bitmap, and a member whose attributes are wholly
subsumed by later hops is reclaimed instead. Readers step through stubs
transparently and still cross every surviving hop's bitmap. The collapse back
to classic HOT is driven by prune: once a chain is fully dead, a later prune
(heap_prune_chain / heap_prune_chain_find_live) reclaims its members and
re-points the root redirect straight at the first live tuple. VACUUM's index
cleanup sweeps the stale leaves; its second pass (lazy_vacuum_heap_page) does
the usual LP_DEAD -> LP_UNUSED conversion and leaves the HOT-indexed collapse
to prune.
The collapse reuses the existing prune/freeze WAL via an xlhp_prune_items
sub-record carrying the (offset, forward) stub pairs; no new record type is
introduced. A page that still carries a preserved stub (or a redirect that
forwards into a live HOT-indexed member) is kept non-all-visible so index-only
scans heap-fetch through the chain; heap_page_would_be_all_visible recognizes
both the redirect-to-SIU and the stub case explicitly.
Co-authored-by: Greg Burd <[email protected]>
Co-authored-by: Nathan Bossart <[email protected]>
---
src/backend/access/heap/README.HOT-INDEXED | 38 ++
src/backend/access/heap/heapam_xlog.c | 10 +-
src/backend/access/heap/pruneheap.c | 668 +++++++++++++++++++--
src/backend/access/heap/vacuumlazy.c | 92 ++-
src/backend/access/rmgrdesc/heapdesc.c | 36 +-
src/include/access/heapam.h | 6 +-
src/include/access/heapam_xlog.h | 19 +-
7 files changed, 809 insertions(+), 60 deletions(-)
diff --git a/src/backend/access/heap/README.HOT-INDEXED b/src/backend/access/heap/README.HOT-INDEXED
index 5d4a2c7d66c..ab4f8bc1881 100644
--- a/src/backend/access/heap/README.HOT-INDEXED
+++ b/src/backend/access/heap/README.HOT-INDEXED
@@ -206,6 +206,44 @@ under the opclass even if not bitwise-identical, e.g. numeric 1.0 vs 1.00) is
still detected. (Appendix A motivates this recheck in detail.)
+Prune and chain collapse
+-------------------------
+
+Because a HOT-indexed update plants an index entry pointing at a mid-chain
+heap-only tuple's own TID, classic HOT's assumption that mid-chain line
+pointers have no external references no longer holds. Pruning therefore must
+not reclaim such a line pointer while a not-yet-swept index entry can still
+arrive at it.
+
+heap_prune_chain collapses a run of dead chain members to a single
+LP_REDIRECT that forwards to the first live tuple, and preserves the line
+pointer of a live HOT-indexed member (heap_prune_item_preserves_hot_indexed)
+so a reader arriving via a stale entry still finds a walkable hop. More than
+one LP_REDIRECT may forward to the same live tuple. The redirect lifecycle
+reuses the existing prune WAL records; there is no new on-disk format.
+
+
+Vacuum reclamation
+------------------
+
+VACUUM's index cleanup sweeps the stale index entries. The collapse back to
+classic HOT is driven by prune, not by VACUUM's second pass: once a chain is
+fully dead, a later prune (heap_prune_chain / heap_prune_chain_find_live)
+reclaims its members and re-points the root redirect straight at first_live.
+Re-pointing a redirect preserves reachability (every walker still reaches
+first_live), so it is safe under the exclusive lock prune already holds.
+
+VACUUM's second pass (lazy_vacuum_heap_page) does not itself re-point
+redirects or reclaim stubs; it performs the usual LP_DEAD -> LP_UNUSED
+conversion and leaves the HOT-indexed collapse to prune.
+
+A page that still carries a preserved HOT-indexed member or a collapse-survivor
+stub is deliberately left non-all-visible, so that an index-only scan
+heap-fetches through the chain and the crossed-attribute bitmap can filter
+stale entries (enforced in heap_prune_record_redirect, the stub recorders, and
+heap_page_would_be_all_visible).
+
+
Appendices
----------
diff --git a/src/backend/access/heap/heapam_xlog.c b/src/backend/access/heap/heapam_xlog.c
index 9ed7024e814..571ea5a4db6 100644
--- a/src/backend/access/heap/heapam_xlog.c
+++ b/src/backend/access/heap/heapam_xlog.c
@@ -103,6 +103,8 @@ heap_xlog_prune_freeze(XLogReaderState *record)
Size datalen;
xlhp_freeze_plan *plans;
OffsetNumber *frz_offsets;
+ OffsetNumber *stubs;
+ int nstubs;
char *dataptr = XLogRecGetBlockData(record, 0, &datalen);
bool do_prune;
@@ -110,9 +112,10 @@ heap_xlog_prune_freeze(XLogReaderState *record)
&nplans, &plans, &frz_offsets,
&nredirected, &redirected,
&ndead, &nowdead,
- &nunused, &nowunused);
+ &nunused, &nowunused,
+ &nstubs, &stubs);
- do_prune = nredirected > 0 || ndead > 0 || nunused > 0;
+ do_prune = nredirected > 0 || ndead > 0 || nunused > 0 || nstubs > 0;
/* Ensure the record does something */
Assert(do_prune || nplans > 0 || vmflags & VISIBILITYMAP_VALID_BITS);
@@ -126,7 +129,8 @@ heap_xlog_prune_freeze(XLogReaderState *record)
(xlrec.flags & XLHP_CLEANUP_LOCK) == 0,
redirected, nredirected,
nowdead, ndead,
- nowunused, nunused);
+ nowunused, nunused,
+ stubs, nstubs);
/* Freeze tuples */
for (int p = 0; p < nplans; p++)
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index fdddd23035b..ee3b340c5ac 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -16,6 +16,7 @@
#include "access/heapam.h"
#include "access/heapam_xlog.h"
+#include "access/hot_indexed.h"
#include "access/htup_details.h"
#include "access/multixact.h"
#include "access/transam.h"
@@ -67,11 +68,20 @@ typedef struct
int nredirected; /* numbers of entries in arrays below */
int ndead;
int nunused;
+ int nstubs;
int nfrozen;
/* arrays that accumulate indexes of items to be changed */
OffsetNumber redirected[MaxHeapTuplesPerPage * 2];
OffsetNumber nowdead[MaxHeapTuplesPerPage];
OffsetNumber nowunused[MaxHeapTuplesPerPage];
+
+ /*
+ * HOT-selectively-updated collapse-survivor stubs: (offset, forward)
+ * pairs of line pointers rewritten in place into xid-free forwarding
+ * stubs that preserve their segment's modified-attrs bitmap.
+ */
+ OffsetNumber stubs[MaxHeapTuplesPerPage * 2];
+
HeapTupleFreeze frozen[MaxHeapTuplesPerPage];
/*
@@ -220,6 +230,10 @@ static void heap_prune_record_prunable(PruneState *prstate, TransactionId xid,
static void heap_prune_record_redirect(PruneState *prstate,
OffsetNumber offnum, OffsetNumber rdoffnum,
bool was_normal);
+static void heap_prune_record_stub(PruneState *prstate,
+ OffsetNumber offnum, OffsetNumber forward);
+static void heap_prune_record_unchanged_lp_stub(PruneState *prstate,
+ OffsetNumber offnum);
static void heap_prune_record_dead(PruneState *prstate, OffsetNumber offnum,
bool was_normal);
static void heap_prune_record_dead_or_unused(PruneState *prstate, OffsetNumber offnum,
@@ -230,6 +244,7 @@ static void heap_prune_record_unchanged_lp_unused(PruneState *prstate, OffsetNum
static void heap_prune_record_unchanged_lp_normal(PruneState *prstate, OffsetNumber offnum);
static void heap_prune_record_unchanged_lp_dead(PruneState *prstate, OffsetNumber offnum);
static void heap_prune_record_unchanged_lp_redirect(PruneState *prstate, OffsetNumber offnum);
+static bool heap_prune_item_preserves_hot_indexed(Page page, OffsetNumber offnum);
static void page_verify_redirects(Page page);
@@ -439,6 +454,7 @@ prune_freeze_setup(PruneFreezeParams *params,
prstate->new_prune_xid = InvalidTransactionId;
prstate->latest_xid_removed = InvalidTransactionId;
prstate->nredirected = prstate->ndead = prstate->nunused = 0;
+ prstate->nstubs = 0;
prstate->nfrozen = 0;
prstate->nroot_items = 0;
prstate->nheaponly_items = 0;
@@ -607,6 +623,23 @@ prune_freeze_plan(PruneState *prstate, OffsetNumber *off_loc)
* Get the tuple's visibility status and queue it up for processing.
*/
htup = (HeapTupleHeader) PageGetItem(page, itemid);
+
+ /*
+ * A collapse-survivor stub is an LP_NORMAL item that is not a real
+ * tuple: HEAP_INDEXED_UPDATED with natts == 0, permanently invisible
+ * (HEAP_XMIN_INVALID), carrying a forward link and the modified-attrs
+ * bitmap for the chain segment it stands in for.
+ * heap_prune_satisfies_vacuum() would classify it HEAPTUPLE_DEAD and
+ * pruning would reclaim it, destroying the bitmap a read needs.
+ * Record it as an unchanged item so it is preserved; the HOT chain
+ * walk steps through it transparently to reach the live tuple.
+ */
+ if (HotIndexedHeaderIsStub(htup))
+ {
+ heap_prune_record_unchanged_lp_stub(prstate, offnum);
+ continue;
+ }
+
tup.t_data = htup;
tup.t_len = ItemIdGetLength(itemid);
ItemPointerSet(&tup.t_self, blockno, offnum);
@@ -677,18 +710,38 @@ prune_freeze_plan(PruneState *prstate, OffsetNumber *off_loc)
ItemId itemid = PageGetItemId(page, offnum);
HeapTupleHeader htup = (HeapTupleHeader) PageGetItem(page, itemid);
- if (likely(!HeapTupleHeaderIsHotUpdated(htup)))
- {
- HeapTupleHeaderAdvanceConflictHorizon(htup,
- &prstate->latest_xid_removed);
+ /*
+ * A dead heap-only tuple that carries a stale btree leaf of its
+ * own (HEAP_INDEXED_UPDATED, natts > 0) is a HOT-indexed chain
+ * member not reached by any chain walk (an aborted
+ * HOT-selectively-updated sub-chain, or a member whose live root
+ * stopped the walk). Mark it LP_DEAD instead of reclaiming it
+ * outright: that pins the slot against reuse and adds it to the
+ * dead-items array so ambulkdelete sweeps the stale leaf and a
+ * later vacuum reclaims the LP.
+ *
+ * Otherwise, this is the classic-HOT case upstream has always
+ * handled here: a dead heap-only tuple with no leaf of its own
+ * should have been reached and removed as part of pruning its
+ * HOT chain. If it is not HotUpdated, it is a legitimate
+ * standalone dead heap-only tuple (e.g. an aborted update) and
+ * can be reclaimed; if it *is* HotUpdated, something is wrong --
+ * preserve it and error out rather than silently destroy the
+ * evidence.
+ */
+ HeapTupleHeaderAdvanceConflictHorizon(htup,
+ &prstate->latest_xid_removed);
+ if ((htup->t_infomask2 & HEAP_INDEXED_UPDATED) != 0 &&
+ HeapTupleHeaderGetNatts(htup) > 0)
+ heap_prune_record_dead_or_unused(prstate, offnum, true);
+ else if (likely(!HeapTupleHeaderIsHotUpdated(htup)))
heap_prune_record_unused(prstate, offnum, true);
- }
else
{
/*
- * This tuple should've been processed and removed as part of
- * a HOT chain, so something's wrong. To preserve evidence,
- * we don't dare to remove it. We cannot leave behind a DEAD
+ * This tuple should've been processed and removed as part of a
+ * HOT chain, so something's wrong. To preserve evidence, we
+ * don't dare to remove it. We cannot leave behind a DEAD
* tuple either, because that will cause VACUUM to error out.
* Throwing an error with a distinct error message seems like
* the least bad option.
@@ -1163,7 +1216,8 @@ heap_page_prune_and_freeze(PruneFreezeParams *params,
do_prune = prstate.nredirected > 0 ||
prstate.ndead > 0 ||
- prstate.nunused > 0;
+ prstate.nunused > 0 ||
+ prstate.nstubs > 0;
/*
* Even if we don't prune anything, if we found a new value for the
@@ -1264,7 +1318,8 @@ heap_page_prune_and_freeze(PruneFreezeParams *params,
heap_page_prune_execute(prstate.buffer, false,
prstate.redirected, prstate.nredirected,
prstate.nowdead, prstate.ndead,
- prstate.nowunused, prstate.nunused);
+ prstate.nowunused, prstate.nunused,
+ prstate.stubs, prstate.nstubs);
}
if (do_freeze)
@@ -1307,7 +1362,8 @@ heap_page_prune_and_freeze(PruneFreezeParams *params,
prstate.frozen, prstate.nfrozen,
prstate.redirected, prstate.nredirected,
prstate.nowdead, prstate.ndead,
- prstate.nowunused, prstate.nunused);
+ prstate.nowunused, prstate.nunused,
+ prstate.stubs, prstate.nstubs);
}
}
@@ -1448,6 +1504,101 @@ htsv_get_valid_status(int status)
return (HTSV_Result) status;
}
+/*
+ * heap_prune_chain_find_live
+ * Follow a HOT chain from 'start' to its first surviving member.
+ *
+ * Used when re-pruning a HOT/SIU chain that was collapsed by an earlier prune:
+ * the root and any entry-bearing dead members were turned into LP_REDIRECTs to
+ * what was then the first live tuple. If that tuple has since been HOT-updated
+ * again and died, the redirects must be re-pointed to the current first live
+ * tuple, or several redirects forwarding to one live tuple must agree on it.
+ * Both cases need the chain's current first surviving member.
+ *
+ * Walks t_ctid on this page starting at 'start', skipping DEAD members, and
+ * returns the offset of the first non-DEAD (surviving) member. Returns
+ * InvalidOffsetNumber if the chain dead-ends with no survivor or runs off the
+ * page. Reads only the page's pre-execute state, so it is correct regardless
+ * of the order in which sibling redirects are processed.
+ */
+static OffsetNumber
+heap_prune_chain_find_live(PruneState *prstate, OffsetNumber start)
+{
+ Page page = prstate->page;
+ OffsetNumber maxoff = PageGetMaxOffsetNumber(page);
+ OffsetNumber offnum = start;
+ OffsetNumber survivor = start; /* successor of the last DEAD member */
+ int loops = 0;
+
+ while (offnum >= FirstOffsetNumber && offnum <= maxoff)
+ {
+ ItemId lp = PageGetItemId(page, offnum);
+ HTSV_Result status;
+ HeapTupleHeader htup;
+
+ /* A redirect/dead/unused item cannot be a surviving chain member. */
+ if (!ItemIdIsNormal(lp))
+ return InvalidOffsetNumber;
+
+ htup = (HeapTupleHeader) PageGetItem(page, lp);
+
+ /*
+ * A collapse-survivor stub is an xid-free forwarding node, not a
+ * chain member; the page scan records it unchanged without computing
+ * visibility, so its htsv slot is unset. Step through it to its
+ * forward link rather than reading htsv, which would trip the
+ * validity assert.
+ */
+ if (HotIndexedHeaderIsStub(htup))
+ {
+ offnum = HotIndexedStubGetForward(htup);
+ if (++loops > maxoff)
+ return InvalidOffsetNumber;
+ continue;
+ }
+
+ status = htsv_get_valid_status(prstate->htsv[offnum]);
+ htup = (HeapTupleHeader) PageGetItem(page, lp);
+
+ if (status == HEAPTUPLE_DEAD)
+ {
+ /*
+ * A DEAD member is reclaimed/redirected, so the surviving tail
+ * starts at its successor. A DEAD member with no live successor
+ * means the whole chain is dead.
+ */
+ if (!HeapTupleHeaderIsHotUpdated(htup) ||
+ ItemPointerGetBlockNumber(&htup->t_ctid) != prstate->block)
+ return InvalidOffsetNumber;
+ offnum = ItemPointerGetOffsetNumber(&htup->t_ctid);
+ survivor = offnum;
+ }
+ else if (status == HEAPTUPLE_RECENTLY_DEAD)
+ {
+ /*
+ * RECENTLY_DEAD members belong to the surviving tail unless a
+ * DEAD member follows them (which would make them part of the
+ * dead prefix). Keep walking to find out, but do not advance the
+ * survivor; it stays at the successor of the last DEAD member.
+ */
+ if (!HeapTupleHeaderIsHotUpdated(htup) ||
+ ItemPointerGetBlockNumber(&htup->t_ctid) != prstate->block)
+ return survivor;
+ offnum = ItemPointerGetOffsetNumber(&htup->t_ctid);
+ }
+ else
+ {
+ /* LIVE (or in-progress): the surviving tail is settled. */
+ return survivor;
+ }
+
+ if (++loops > maxoff)
+ return InvalidOffsetNumber; /* defend against a corrupt cycle */
+ }
+
+ return InvalidOffsetNumber;
+}
+
/*
* Prune specified line pointer or a HOT chain originating at line pointer.
*
@@ -1518,6 +1669,36 @@ heap_prune_chain(OffsetNumber maxoff, OffsetNumber rootoffnum,
if (offnum > maxoff)
break;
+ /*
+ * Step transparently through a collapse-survivor stub. A redirect or
+ * an earlier stub may forward into a stub that replaced a dead
+ * mid-chain member; the stub was already recorded (as unchanged) by
+ * the page scan and is not itself a chain member, so follow its
+ * forward link rather than stopping at the processed check below.
+ */
+ {
+ ItemId slp = PageGetItemId(page, offnum);
+
+ if (ItemIdIsNormal(slp))
+ {
+ HeapTupleHeader shtup = (HeapTupleHeader) PageGetItem(page, slp);
+
+ if (HotIndexedHeaderIsStub(shtup))
+ {
+ /*
+ * A stub is xid-free, so the xmin/xmax linkage cannot be
+ * verified across it. Trust the stub's forward link and
+ * skip the prior-xmax check for the first member past it
+ * (otherwise the chain would be severed there, dropping
+ * its tail).
+ */
+ offnum = HotIndexedStubGetForward(shtup);
+ priorXmax = InvalidTransactionId;
+ continue;
+ }
+ }
+ }
+
/* If item is already processed, stop --- it must not be same chain */
if (prstate->processed[offnum])
break;
@@ -1624,13 +1805,27 @@ heap_prune_chain(OffsetNumber maxoff, OffsetNumber rootoffnum,
if (ItemIdIsRedirected(rootlp) && nchain < 2)
{
/*
- * We found a redirect item that doesn't point to a valid follow-on
- * item. This can happen if the loop in heap_page_prune_and_freeze()
- * caused us to visit the dead successor of a redirect item before
- * visiting the redirect item. We can clean up by setting the
- * redirect item to LP_DEAD state or LP_UNUSED if the caller
- * indicated.
+ * The walk could not get past the redirect: its target was either
+ * already processed by a sibling redirect's walk (several redirects
+ * of a collapsed HOT/SIU chain forward to the same live tuple) or has
+ * since died and been collapsed further. Re-point this redirect at
+ * the chain's current first surviving member so every entry that
+ * resolves through it still reaches the live tuple. If no survivor
+ * remains, the redirect is dangling and is reclaimed (LP_DEAD, or
+ * LP_UNUSED if the caller allows it).
*/
+ OffsetNumber target = ItemIdGetRedirect(rootlp);
+ OffsetNumber live = heap_prune_chain_find_live(prstate, target);
+
+ if (OffsetNumberIsValid(live))
+ {
+ if (live == target)
+ heap_prune_record_unchanged_lp_redirect(prstate, rootoffnum);
+ else
+ heap_prune_record_redirect(prstate, rootoffnum, live, false);
+ return;
+ }
+
heap_prune_record_dead_or_unused(prstate, rootoffnum, false);
return;
}
@@ -1656,24 +1851,143 @@ process_chain:
else if (ndeadchain == nchain)
{
/*
- * The entire chain is dead. Mark the root line pointer LP_DEAD, and
- * fully remove the other tuples in the chain.
+ * The entire chain is dead. No live tuple remains to forward to, so
+ * mark the root LP_DEAD (or LP_UNUSED if the caller allows it) and
+ * reclaim each member. A dead HOT-selectively-updated member may
+ * still have a stale btree leaf pointing at it: mark it LP_DEAD so
+ * the slot is pinned against reuse and added to the dead-items array,
+ * letting ambulkdelete sweep the leaf and a later vacuum reclaim the
+ * line pointer. Classic-HOT members carry no leaf of their own and
+ * go straight to LP_UNUSED.
*/
heap_prune_record_dead_or_unused(prstate, rootoffnum, ItemIdIsNormal(rootlp));
for (int i = 1; i < nchain; i++)
- heap_prune_record_unused(prstate, chainitems[i], true);
+ {
+ if (heap_prune_item_preserves_hot_indexed(page, chainitems[i]))
+ heap_prune_record_dead_or_unused(prstate, chainitems[i], true);
+ else
+ heap_prune_record_unused(prstate, chainitems[i], true);
+ }
}
else
{
/*
- * We found a DEAD tuple in the chain. Redirect the root line pointer
- * to the first non-DEAD tuple, and mark as unused each intermediate
- * item that we are able to remove from the chain.
+ * The chain has a dead prefix followed by a live remainder. Collapse
+ * it with PHOT-style key tuples so that the per-hop modified-attrs
+ * bitmaps survive for the bitmap-overlap read path.
+ *
+ * Walk the dead members from the live end backwards, accumulating in
+ * laterattrs the union of the modified-attrs bitmaps of the members
+ * that follow (the "later hops"). A dead key tuple -- one that
+ * carried its own index entries because it changed an indexed
+ * attribute at its hop (heap_prune_item_preserves_hot_indexed) -- is
+ * disposed of as follows:
+ *
+ * - If every attribute it changed was changed again by a later hop
+ * (its bitmap is a subset of laterattrs), every index entry pointing
+ * at it is superseded, so no live entry references it: reclaim it
+ * (LP_DEAD), which lets this vacuum's index pass sweep its now-stale
+ * leaves and a later pass free the line pointer. This loses no
+ * per-hop information for readers -- its attributes are already
+ * carried by the surviving later members the reader still crosses.
+ *
+ * - Otherwise it introduced an attribute not changed again, so a live
+ * entry still points at it: keep it as an xid-free stub forwarding to
+ * the next survivor, preserving its inline bitmap for the read path.
+ *
+ * Classic-HOT members carry no entry of their own and are reclaimed
+ * to LP_UNUSED; survivors forward past them. The root is redirected
+ * to the first survivor. Stubs carry no XIDs, so the page stays
+ * freeze-safe; because each forwards to the next survivor (not the
+ * live tuple), a reader crossing the collapsed prefix sees every
+ * surviving hop's bitmap, and stub->stub forwarding lets a later
+ * collapse extend the chain without re-pointing existing stubs.
*/
- heap_prune_record_redirect(prstate, rootoffnum, chainitems[ndeadchain],
- ItemIdIsNormal(rootlp));
- for (int i = 1; i < ndeadchain; i++)
- heap_prune_record_unused(prstate, chainitems[i], true);
+ OffsetNumber first_live = chainitems[ndeadchain];
+ OffsetNumber next_survivor = first_live;
+ OffsetNumber root_target;
+ int relnatts = RelationGetNumberOfAttributes(prstate->relation);
+ uint8 laterattrs[(MaxHeapAttributeNumber + 7) / 8];
+
+ /*
+ * laterattrs accumulates every surviving hop's modified attributes.
+ * Size it for the relation's current natts (the maximum); each
+ * contributing tuple's bitmap is located and OR-ed using that tuple's
+ * write-time natts (HotIndexedTupleBitmapNatts), since ADD COLUMN may
+ * have grown the relation since some hops were written.
+ */
+ memset(laterattrs, 0, HotIndexedBitmapBytes(relnatts));
+ for (int i = ndeadchain; i < nchain; i++)
+ {
+ ItemId lp = PageGetItemId(page, chainitems[i]);
+ HeapTupleHeader htup = (HeapTupleHeader) PageGetItem(page, lp);
+
+ if ((htup->t_infomask2 & HEAP_INDEXED_UPDATED) != 0)
+ {
+ int bmnatts = HotIndexedTupleBitmapNatts(htup);
+
+ /*
+ * A hop's write-time natts can never legitimately exceed the
+ * relation's current natts (natts only grows via ADD COLUMN).
+ * On a corrupt page a stub's unbounded stashed natts (or a
+ * corrupt live tuple's natts field) could otherwise overflow
+ * laterattrs, which is sized for relnatts; clamp defensively.
+ */
+ Assert(bmnatts <= relnatts);
+ if (bmnatts > relnatts)
+ bmnatts = relnatts;
+
+ HotIndexedBitmapUnion(laterattrs,
+ HotIndexedGetModifiedBitmap(htup,
+ ItemIdGetLength(lp),
+ bmnatts),
+ bmnatts);
+ }
+ }
+
+ /* dead prefix: reclaim superseded members, stub the rest */
+ for (int i = ndeadchain - 1; i >= 1; i--)
+ {
+ if (heap_prune_item_preserves_hot_indexed(page, chainitems[i]))
+ {
+ ItemId lp = PageGetItemId(page, chainitems[i]);
+ HeapTupleHeader htup = (HeapTupleHeader) PageGetItem(page, lp);
+ int bmnatts = HotIndexedTupleBitmapNatts(htup);
+ const uint8 *attrs;
+
+ /* See the comment above laterattrs' first use. */
+ Assert(bmnatts <= relnatts);
+ if (bmnatts > relnatts)
+ bmnatts = relnatts;
+ attrs = HotIndexedGetModifiedBitmap(htup,
+ ItemIdGetLength(lp),
+ bmnatts);
+
+ if (HotIndexedBitmapIsSubset(attrs, laterattrs, bmnatts))
+ heap_prune_record_dead_or_unused(prstate, chainitems[i], true);
+ else
+ {
+ heap_prune_record_stub(prstate, chainitems[i], next_survivor);
+ next_survivor = chainitems[i];
+ }
+ HotIndexedBitmapUnion(laterattrs, attrs, bmnatts);
+ }
+ else
+ heap_prune_record_unused(prstate, chainitems[i], true);
+ }
+
+ root_target = next_survivor;
+
+ /*
+ * root -> first survivor (skip a redundant no-op redirect on
+ * re-prune)
+ */
+ if (ItemIdIsRedirected(rootlp) &&
+ ItemIdGetRedirect(rootlp) == root_target)
+ heap_prune_record_unchanged_lp_redirect(prstate, rootoffnum);
+ else
+ heap_prune_record_redirect(prstate, rootoffnum, root_target,
+ ItemIdIsNormal(rootlp));
/* the rest of tuples in the chain are normal, unchanged tuples */
for (int i = ndeadchain; i < nchain; i++)
@@ -1718,6 +2032,34 @@ heap_prune_record_redirect(PruneState *prstate,
* separately as an unchanged tuple.
*/
+ /*
+ * If the redirect points at a HOT-selectively-updated live tuple, the
+ * page may still carry stale btree entries that resolve through this
+ * redirect to a tuple with a different key. Such entries are filtered by
+ * the read path's crossed-attribute bitmap, which requires fetching the
+ * heap tuple -- but an index-only scan trusts the visibility map and skips
+ * that fetch. So
+ * the page must not be reported all-visible/all-frozen while such a
+ * redirect exists; it becomes eligible again only once vacuum has swept
+ * the stale leaves and reclaimed the redirect.
+ */
+ if (rdoffnum >= FirstOffsetNumber &&
+ rdoffnum <= PageGetMaxOffsetNumber(prstate->page))
+ {
+ ItemId tlp = PageGetItemId(prstate->page, rdoffnum);
+
+ if (ItemIdIsNormal(tlp))
+ {
+ HeapTupleHeader thtup = (HeapTupleHeader) PageGetItem(prstate->page, tlp);
+
+ if ((thtup->t_infomask2 & HEAP_INDEXED_UPDATED) != 0)
+ {
+ prstate->set_all_visible = false;
+ prstate->set_all_frozen = false;
+ }
+ }
+ }
+
Assert(prstate->nredirected < MaxHeapTuplesPerPage);
prstate->redirected[prstate->nredirected * 2] = offnum;
prstate->redirected[prstate->nredirected * 2 + 1] = rdoffnum;
@@ -1735,6 +2077,65 @@ heap_prune_record_redirect(PruneState *prstate,
prstate->hastup = true;
}
+/*
+ * Record a line pointer to be rewritten in place as a HOT-selectively-updated
+ * collapse-survivor stub forwarding to 'forward'.
+ *
+ * The source must be a dead heap-only tuple that carried its own btree
+ * entries (a key tuple) and so cannot be reclaimed outright: a stale entry may
+ * still resolve through it. Rewriting it into an xid-free stub keeps the
+ * forward link and the tuple's inline modified-attrs bitmap (so the read path
+ * can judge staleness) while dropping its XIDs, which keeps the page
+ * freeze-safe.
+ */
+static void
+heap_prune_record_stub(PruneState *prstate,
+ OffsetNumber offnum, OffsetNumber forward)
+{
+ Assert(!prstate->processed[offnum]);
+ prstate->processed[offnum] = true;
+
+ /*
+ * As with a redirect to a HOT-selectively-updated tuple, the page must
+ * not be reported all-visible/all-frozen while a stub exists: an
+ * index-only scan would otherwise trust the VM and skip the recheck that
+ * filters a now-stale entry resolving through the stub. A stub's
+ * HEAP_XMIN_INVALID also makes it invisible to every snapshot, which an
+ * all-visible page must never contain. Eligibility returns once vacuum
+ * reclaims the stub.
+ */
+ prstate->set_all_visible = false;
+ prstate->set_all_frozen = false;
+
+ Assert(prstate->nstubs < MaxHeapTuplesPerPage);
+ prstate->stubs[prstate->nstubs * 2] = offnum;
+ prstate->stubs[prstate->nstubs * 2 + 1] = forward;
+ prstate->nstubs++;
+
+ /* The dead key tuple's storage is being discarded; count it removed. */
+ prstate->ndeleted++;
+
+ prstate->hastup = true;
+}
+
+/*
+ * Record an existing collapse-survivor stub that is to be left unchanged.
+ *
+ * Encountered when re-pruning a page that already holds stubs from an earlier
+ * collapse. The stub is preserved (its bitmap is still needed) and counts as
+ * a reason the page cannot be reported all-visible/all-frozen.
+ */
+static void
+heap_prune_record_unchanged_lp_stub(PruneState *prstate, OffsetNumber offnum)
+{
+ Assert(!prstate->processed[offnum]);
+ prstate->processed[offnum] = true;
+ prstate->hastup = true;
+
+ prstate->set_all_visible = false;
+ prstate->set_all_frozen = false;
+}
+
/* Record line pointer to be marked dead */
static void
heap_prune_record_dead(PruneState *prstate, OffsetNumber offnum,
@@ -1816,6 +2217,52 @@ heap_prune_record_unused(PruneState *prstate, OffsetNumber offnum, bool was_norm
prstate->ndeleted++;
}
+
+/*
+ * heap_prune_item_preserves_hot_indexed
+ * True iff the LP at `offnum` on `page` is a live HOT-indexed (HOT/SIU)
+ * heap-only tuple whose LP must be preserved rather than reclaimed to
+ * LP_UNUSED, because a not-yet-swept index entry may still point at it.
+ *
+ * A HOT-indexed update plants a new index entry pointing at the heap-only
+ * tuple's own TID. Classic HOT's invariant that mid-chain LPs have no
+ * external references therefore does not hold for such tuples: until
+ * ambulkdelete sweeps any stale index entry, a reader arriving via it must
+ * still find a walkable hop at the LP. Chain collapse converts dead members
+ * to LP_REDIRECT forwarders for exactly this reason; a live member like this
+ * one must simply not be reclaimed out from under such a reader.
+ *
+ * Excluded from preservation:
+ * - items that are not LP_NORMAL (REDIRECT, DEAD, UNUSED);
+ * - tuples without HEAP_INDEXED_UPDATED (classic HOT chain members never
+ * had a per-tuple index entry planted);
+ * - tuples with no attributes (defensive: not a real chain member);
+ * - aborted heap-only tuples (HEAP_XMIN_INVALID): never visible through any
+ * index entry, so reclaiming them is safe.
+ */
+static bool
+heap_prune_item_preserves_hot_indexed(Page page, OffsetNumber offnum)
+{
+ ItemId lp = PageGetItemId(page, offnum);
+ HeapTupleHeader htup;
+
+ if (!ItemIdIsNormal(lp))
+ return false;
+
+ htup = (HeapTupleHeader) PageGetItem(page, lp);
+
+ if ((htup->t_infomask2 & HEAP_INDEXED_UPDATED) == 0)
+ return false;
+ if (HeapTupleHeaderGetNatts(htup) == 0)
+ return false;
+ if ((htup->t_infomask & HEAP_XMIN_INVALID) != 0)
+ return false;
+
+ return true;
+}
+
+
+
/*
* Record an unused line pointer that is left unchanged.
*/
@@ -2049,8 +2496,44 @@ heap_prune_record_unchanged_lp_redirect(PruneState *prstate, OffsetNumber offnum
*/
Assert(!prstate->processed[offnum]);
prstate->processed[offnum] = true;
+
+ /*
+ * As in heap_prune_record_redirect: if this redirect forwards to a
+ * HOT-selectively-updated live tuple, the page may carry stale btree
+ * entries that resolve through it, so it must not be reported
+ * all-visible/all-frozen (an index-only scan would otherwise skip the
+ * crossed-attribute bitmap check). This must happen here too, not only when the
+ * redirect is first created, because a re-prune records an existing SIU
+ * redirect as unchanged.
+ */
+ {
+ ItemId lp = PageGetItemId(prstate->page, offnum);
+
+ if (ItemIdIsRedirected(lp))
+ {
+ OffsetNumber rdoffnum = ItemIdGetRedirect(lp);
+
+ if (rdoffnum >= FirstOffsetNumber &&
+ rdoffnum <= PageGetMaxOffsetNumber(prstate->page))
+ {
+ ItemId tlp = PageGetItemId(prstate->page, rdoffnum);
+
+ if (ItemIdIsNormal(tlp))
+ {
+ HeapTupleHeader thtup = (HeapTupleHeader) PageGetItem(prstate->page, tlp);
+
+ if ((thtup->t_infomask2 & HEAP_INDEXED_UPDATED) != 0)
+ {
+ prstate->set_all_visible = false;
+ prstate->set_all_frozen = false;
+ }
+ }
+ }
+ }
+ }
}
+
/*
* Perform the actual page changes needed by heap_page_prune_and_freeze().
*
@@ -2065,16 +2548,27 @@ void
heap_page_prune_execute(Buffer buffer, bool lp_truncate_only,
OffsetNumber *redirected, int nredirected,
OffsetNumber *nowdead, int ndead,
- OffsetNumber *nowunused, int nunused)
+ OffsetNumber *nowunused, int nunused,
+ OffsetNumber *stubs, int nstubs)
{
Page page = BufferGetPage(buffer);
OffsetNumber *offnum;
HeapTupleHeader htup PG_USED_FOR_ASSERTS_ONLY;
/* Shouldn't be called unless there's something to do */
- Assert(nredirected > 0 || ndead > 0 || nunused > 0);
+ Assert(nredirected > 0 || ndead > 0 || nunused > 0 || nstubs > 0);
- /* If 'lp_truncate_only', we can only remove already-dead line pointers */
+ /*
+ * If 'lp_truncate_only', we can only remove already-dead line pointers
+ * and cannot re-point redirects: repointing moves the tuple an index-only
+ * scan or a concurrent chain walk expects to find, which needs a cleanup
+ * lock (see the file header comment). No producer in this series calls
+ * with lp_truncate_only set and a nonzero nredirected -- vacuum's second
+ * pass (the lp_truncate_only caller) never re-points a redirect itself,
+ * leaving that to a later prune that holds a cleanup lock -- so keep
+ * upstream's stricter assertion rather than the weaker one this would
+ * otherwise need to justify.
+ */
Assert(!lp_truncate_only || (nredirected == 0 && ndead == 0));
/* Update all redirected line pointers */
@@ -2086,6 +2580,16 @@ heap_page_prune_execute(Buffer buffer, bool lp_truncate_only,
ItemId fromlp = PageGetItemId(page, fromoff);
ItemId tolp PG_USED_FOR_ASSERTS_ONLY;
+ /*
+ * A redundant redirect (the LP already redirects to tooff) is a
+ * harmless no-op. This arises when a HOT-indexed chain that was
+ * already collapsed is re-pruned and the root still resolves to the
+ * same target; skip it so the apply stays idempotent on both primary
+ * and replay.
+ */
+ if (ItemIdIsRedirected(fromlp) && ItemIdGetRedirect(fromlp) == tooff)
+ continue;
+
#ifdef USE_ASSERT_CHECKING
/*
@@ -2100,7 +2604,16 @@ heap_page_prune_execute(Buffer buffer, bool lp_truncate_only,
Assert(ItemIdHasStorage(fromlp) && ItemIdIsNormal(fromlp));
htup = (HeapTupleHeader) PageGetItem(page, fromlp);
- Assert(!HeapTupleHeaderIsHeapOnly(htup));
+
+ /*
+ * The redirect source is normally the non-heap-only chain root. A
+ * HOT/SIU chain collapse additionally redirects dead heap-only
+ * members that carried their own btree entry to the live tuple,
+ * so a heap-only redirect source is allowed when it is
+ * HOT-selectively-updated (HEAP_INDEXED_UPDATED).
+ */
+ Assert(!HeapTupleHeaderIsHeapOnly(htup) ||
+ (htup->t_infomask2 & HEAP_INDEXED_UPDATED) != 0);
}
else
{
@@ -2128,12 +2641,55 @@ heap_page_prune_execute(Buffer buffer, bool lp_truncate_only,
tolp = PageGetItemId(page, tooff);
Assert(ItemIdHasStorage(tolp) && ItemIdIsNormal(tolp));
htup = (HeapTupleHeader) PageGetItem(page, tolp);
+ /* A redirect targets the first surviving member: a heap-only tuple. */
Assert(HeapTupleHeaderIsHeapOnly(htup));
#endif
ItemIdSetRedirect(fromlp, tooff);
}
+ /*
+ * Rewrite collapse-survivor stubs in place. Each (offset, forward) pair
+ * names a dead key tuple to be turned into an xid-free forwarding stub:
+ * permanently invisible (HEAP_XMIN_INVALID|HEAP_XMAX_INVALID), flagged
+ * HEAP_INDEXED_UPDATED with natts == 0 so consumers recognise it as a
+ * stub rather than a tuple, heap-only preserved so it remains a valid
+ * redirect/forward target, and t_ctid.offnum set to the forward offset.
+ * The item's storage (including its inline modified-attrs bitmap in the
+ * final bytes) is left undisturbed, so the bitmap survives and need not
+ * be carried in WAL.
+ */
+ offnum = stubs;
+ for (int i = 0; i < nstubs; i++)
+ {
+ OffsetNumber off = *offnum++;
+ OffsetNumber forward = *offnum++;
+ ItemId lp = PageGetItemId(page, off);
+ HeapTupleHeader tup;
+ int bitmap_natts;
+
+ Assert(ItemIdIsNormal(lp));
+ tup = (HeapTupleHeader) PageGetItem(page, lp);
+
+ /*
+ * Preserve the tuple's write-time natts before we overwrite the natts
+ * field with the stub sentinel (0): the trailing modified-attrs bitmap
+ * was sized with it, and readers need it to locate the bitmap when the
+ * relation's current natts has since grown (ADD COLUMN). The stub's
+ * t_ctid offset half holds the forward link; the block half is unused
+ * for a stub, so stash the natts there. This runs identically on the
+ * primary and in redo (the pre-stub tuple is on both pages), so no WAL
+ * change is needed.
+ */
+ bitmap_natts = HeapTupleHeaderGetNatts(tup);
+
+ tup->t_infomask = HEAP_XMIN_INVALID | HEAP_XMAX_INVALID;
+ tup->t_infomask2 = HEAP_ONLY_TUPLE | HEAP_INDEXED_UPDATED;
+ HeapTupleHeaderSetNatts(tup, 0);
+ ItemPointerSetOffsetNumber(&tup->t_ctid, forward);
+ HotIndexedStubSetBitmapNatts(tup, bitmap_natts);
+ }
+
/* Update all now-dead line pointers */
offnum = nowdead;
for (int i = 0; i < ndead; i++)
@@ -2149,12 +2705,23 @@ heap_page_prune_execute(Buffer buffer, bool lp_truncate_only,
* an index. This should never be necessary with any individual
* heap-only tuple item, though. (It's not clear how much of a problem
* that would be, but there is no reason to allow it.)
+ *
+ * Exception: a HOT-indexed aborted orphan whose chain root is
+ * unreachable on this page is intentionally marked LP_DEAD by the
+ * heap-only-tuples loop in heap_page_prune_and_freeze (see the
+ * heap_prune_record_dead call there). The tuple is heap-only (it was
+ * created by an UPDATE) and carries HEAP_INDEXED_UPDATED; the
+ * adjacent btree leaf is still live, so we keep the slot pinned via
+ * LP_DEAD until ambulkdelete sweeps it. A subsequent vacuum reclaims
+ * the LP to LP_UNUSED.
*/
if (ItemIdHasStorage(lp))
{
Assert(ItemIdIsNormal(lp));
htup = (HeapTupleHeader) PageGetItem(page, lp);
- Assert(!HeapTupleHeaderIsHeapOnly(htup));
+ Assert(!HeapTupleHeaderIsHeapOnly(htup) ||
+ ((htup->t_infomask2 & HEAP_INDEXED_UPDATED) != 0 &&
+ HeapTupleHeaderGetNatts(htup) > 0));
}
else
{
@@ -2177,7 +2744,7 @@ heap_page_prune_execute(Buffer buffer, bool lp_truncate_only,
if (lp_truncate_only)
{
- /* Setting LP_DEAD to LP_UNUSED in vacuum's second pass */
+ /* Setting LP_DEAD to LP_UNUSED in vacuum's second pass. */
Assert(ItemIdIsDead(lp) && !ItemIdHasStorage(lp));
}
else
@@ -2188,7 +2755,8 @@ heap_page_prune_execute(Buffer buffer, bool lp_truncate_only,
* items to be made LP_UNUSED instead. This is only possible if
* the relation has no indexes. If there are any dead items, then
* mark_unused_now was not true and every item being marked
- * LP_UNUSED must refer to a heap-only tuple.
+ * LP_UNUSED must refer to a heap-only tuple whose chain has been
+ * pruned.
*/
if (ndead > 0)
{
@@ -2264,6 +2832,8 @@ page_verify_redirects(Page page)
Assert(ItemIdIsNormal(targitem));
Assert(ItemIdHasStorage(targitem));
htup = (HeapTupleHeader) PageGetItem(page, targitem);
+
+ /* A redirect targets the first surviving chain member: heap-only. */
Assert(HeapTupleHeaderIsHeapOnly(htup));
}
#endif
@@ -2566,7 +3136,8 @@ log_heap_prune_and_freeze(Relation relation, Buffer buffer,
HeapTupleFreeze *frozen, int nfrozen,
OffsetNumber *redirected, int nredirected,
OffsetNumber *dead, int ndead,
- OffsetNumber *unused, int nunused)
+ OffsetNumber *unused, int nunused,
+ OffsetNumber *stubs, int nstubs)
{
xl_heap_prune xlrec;
XLogRecPtr recptr;
@@ -2581,8 +3152,10 @@ log_heap_prune_and_freeze(Relation relation, Buffer buffer,
xlhp_prune_items redirect_items;
xlhp_prune_items dead_items;
xlhp_prune_items unused_items;
+ xlhp_prune_items stub_items;
OffsetNumber frz_offsets[MaxHeapTuplesPerPage];
- bool do_prune = nredirected > 0 || ndead > 0 || nunused > 0;
+ bool do_prune = nredirected > 0 || ndead > 0 || nunused > 0 ||
+ nstubs > 0;
bool do_set_vm = vmflags & VISIBILITYMAP_VALID_BITS;
bool heap_fpi_allowed = true;
@@ -2670,6 +3243,16 @@ log_heap_prune_and_freeze(Relation relation, Buffer buffer,
XLogRegisterBufData(0, unused,
sizeof(OffsetNumber) * nunused);
}
+ if (nstubs > 0)
+ {
+ xlrec.flags |= XLHP_HAS_HOT_INDEXED_STUBS;
+
+ stub_items.ntargets = nstubs;
+ XLogRegisterBufData(0, &stub_items,
+ offsetof(xlhp_prune_items, data));
+ XLogRegisterBufData(0, stubs,
+ sizeof(OffsetNumber[2]) * nstubs);
+ }
if (nfrozen > 0)
XLogRegisterBufData(0, frz_offsets,
sizeof(OffsetNumber) * nfrozen);
@@ -2692,8 +3275,17 @@ log_heap_prune_and_freeze(Relation relation, Buffer buffer,
xlrec.flags |= XLHP_CLEANUP_LOCK;
else
{
- Assert(nredirected == 0 && ndead == 0);
- /* also, any items in 'unused' must've been LP_DEAD previously */
+ /*
+ * Without a cleanup lock we can only remove already-dead line
+ * pointers and re-point redirects. The latter happens when vacuum's
+ * second pass reclaims a collapsed HOT-indexed chain and re-points
+ * the root redirect at first_live: that change is made under an
+ * exclusive lock and preserves the chain's reachability (every walker
+ * still reaches first_live), so no cleanup lock is needed -- the same
+ * basis on which this pass already reclaims dead line pointers to
+ * LP_UNUSED.
+ */
+ Assert(ndead == 0);
}
XLogRegisterData(&xlrec, SizeOfHeapPrune);
if (TransactionIdIsValid(conflict_xid))
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 39395aed0d5..dab4b5b1f82 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -131,6 +131,7 @@
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/hot_indexed.h"
#include "access/htup_details.h"
#include "access/multixact.h"
#include "access/tidstore.h"
@@ -445,7 +446,8 @@ static bool lazy_vacuum_all_indexes(LVRelState *vacrel);
static void lazy_vacuum_heap_rel(LVRelState *vacrel);
static void lazy_vacuum_heap_page(LVRelState *vacrel, BlockNumber blkno,
Buffer buffer, OffsetNumber *deadoffsets,
- int num_offsets, Buffer vmbuffer);
+ int num_offsets, Buffer vmbuffer,
+ bool got_cleanup_lock);
static bool lazy_check_wraparound_failsafe(LVRelState *vacrel);
static void lazy_cleanup_all_indexes(LVRelState *vacrel);
static IndexBulkDeleteResult *lazy_vacuum_one_index(Relation indrel,
@@ -1972,6 +1974,7 @@ lazy_scan_new_or_empty(LVRelState *vacrel, Buffer buf, BlockNumber blkno,
NULL, 0,
NULL, 0,
NULL, 0,
+ NULL, 0,
NULL, 0);
END_CRIT_SECTION();
@@ -2686,6 +2689,7 @@ lazy_vacuum_heap_rel(LVRelState *vacrel)
Size freespace;
OffsetNumber offsets[MaxOffsetNumber];
int num_offsets;
+ bool got_cleanup_lock;
vacuum_delay_point(false);
@@ -2708,10 +2712,20 @@ lazy_vacuum_heap_rel(LVRelState *vacrel)
*/
visibilitymap_pin(vacrel->rel, blkno, &vmbuffer);
- /* We need a non-cleanup exclusive lock to mark dead_items unused */
- LockBuffer(buf, BUFFER_LOCK_EXCLUSIVE);
+ /*
+ * Setting dead items unused needs only an exclusive lock. We still
+ * prefer a cleanup lock here, as the first pass does, and fall back to
+ * an exclusive lock if one is not immediately available. This pass
+ * only turns LP_DEAD items into LP_UNUSED; it does NOT reclaim a
+ * collapsed HOT-indexed chain's stubs or re-point its redirects --
+ * that chain-structure rewrite (which moves TIDs concurrent scans
+ * follow, and so needs a cleanup lock) is left to a later prune.
+ */
+ got_cleanup_lock = ConditionalLockBufferForCleanup(buf);
+ if (!got_cleanup_lock)
+ LockBuffer(buf, BUFFER_LOCK_EXCLUSIVE);
lazy_vacuum_heap_page(vacrel, blkno, buf, offsets,
- num_offsets, vmbuffer);
+ num_offsets, vmbuffer, got_cleanup_lock);
/* Now that we've vacuumed the page, record its available space */
page = BufferGetPage(buf);
@@ -2757,7 +2771,7 @@ lazy_vacuum_heap_rel(LVRelState *vacrel)
static void
lazy_vacuum_heap_page(LVRelState *vacrel, BlockNumber blkno, Buffer buffer,
OffsetNumber *deadoffsets, int num_offsets,
- Buffer vmbuffer)
+ Buffer vmbuffer, bool got_cleanup_lock)
{
Page page = BufferGetPage(buffer);
OffsetNumber unused[MaxHeapTuplesPerPage];
@@ -2783,7 +2797,9 @@ lazy_vacuum_heap_page(LVRelState *vacrel, BlockNumber blkno, Buffer buffer,
* and mark the page all-visible within the same critical section,
* enabling both changes to be emitted in a single WAL record. Since the
* visibility checks may perform I/O and allocate memory, they must be
- * done outside the critical section.
+ * done outside the critical section. A deferred reclaim leaves a
+ * not-yet-removed member on the page, so skip the check when anything was
+ * deferred.
*/
if (heap_page_would_be_all_visible(vacrel->rel, buffer,
vacrel->vistest, true,
@@ -2815,13 +2831,12 @@ lazy_vacuum_heap_page(LVRelState *vacrel, BlockNumber blkno, Buffer buffer,
itemid = PageGetItemId(page, toff);
+ /* A reclaimable item is a classic LP_DEAD line pointer. */
Assert(ItemIdIsDead(itemid) && !ItemIdHasStorage(itemid));
ItemIdSetUnused(itemid);
unused[nunused++] = toff;
}
- Assert(nunused > 0);
-
/* Attempt to truncate line pointer array now */
PageTruncateLinePointerArray(page);
@@ -2851,12 +2866,13 @@ lazy_vacuum_heap_page(LVRelState *vacrel, BlockNumber blkno, Buffer buffer,
vmflags != 0 ? vmbuffer : InvalidBuffer,
vmflags,
conflict_xid,
- false, /* no cleanup lock required */
+ false, /* no cleanup lock required: see below */
PRUNE_VACUUM_CLEANUP,
NULL, 0, /* frozen */
NULL, 0, /* redirected */
NULL, 0, /* dead */
- unused, nunused);
+ unused, nunused,
+ NULL, 0); /* stubs */
}
END_CRIT_SECTION();
@@ -3647,10 +3663,44 @@ heap_page_would_be_all_visible(Relation rel, Buffer buf,
*logging_offnum = offnum;
itemid = PageGetItemId(page, offnum);
- /* Unused or redirect line pointers are of no interest */
- if (!ItemIdIsUsed(itemid) || ItemIdIsRedirected(itemid))
+ /* Unused line pointers are of no interest. */
+ if (!ItemIdIsUsed(itemid))
continue;
+ /*
+ * Plain redirects are of no interest (the chain member they point at
+ * is inspected separately) -- except a redirect that forwards to a
+ * HOT-selectively-updated live tuple. Such a redirect may still be
+ * reached by a stale index entry whose key the live tuple no longer
+ * holds; if the page were marked all-visible an index-only scan would
+ * trust the VM, skip the heap fetch, and surface that stale key.
+ * Keep the page not-all-visible until the stale leaves are swept and
+ * the redirect reclaimed. This mirrors the guard in
+ * heap_prune_record_redirect, applied here because VACUUM's second
+ * pass can set all-visible after reclaiming other items on the page.
+ */
+ if (ItemIdIsRedirected(itemid))
+ {
+ OffsetNumber rdoff = ItemIdGetRedirect(itemid);
+
+ if (rdoff >= FirstOffsetNumber && rdoff <= maxoff)
+ {
+ ItemId tlp = PageGetItemId(page, rdoff);
+
+ if (ItemIdIsNormal(tlp))
+ {
+ HeapTupleHeader thtup = (HeapTupleHeader) PageGetItem(page, tlp);
+
+ if ((thtup->t_infomask2 & HEAP_INDEXED_UPDATED) != 0)
+ {
+ *all_frozen = all_visible = false;
+ break;
+ }
+ }
+ }
+ continue;
+ }
+
ItemPointerSet(&(tuple.t_self), blockno, offnum);
/*
@@ -3676,6 +3726,24 @@ heap_page_would_be_all_visible(Relation rel, Buffer buf,
tuple.t_len = ItemIdGetLength(itemid);
tuple.t_tableOid = RelationGetRelid(rel);
+ /*
+ * A HOT-indexed collapse-survivor stub is an LP_NORMAL item that is
+ * not a real tuple: it forwards through the chain and carries a
+ * preserved modified-attrs bitmap that a reader arriving via a stale
+ * leaf must still cross. A page holding one must stay not-all-visible
+ * so index-only scans heap-fetch through the chain, exactly like the
+ * redirect-to-SIU case above. A stub's header is frozen-invalid
+ * (HEAP_XMIN_INVALID), so the visibility check below would also class
+ * it not-all-visible -- but recognize it explicitly here rather than
+ * relying on that side effect, so the guard cannot silently lapse if
+ * the stub encoding ever changes.
+ */
+ if (HotIndexedHeaderIsStub(tuple.t_data))
+ {
+ *all_frozen = all_visible = false;
+ break;
+ }
+
/* Visibility checks may do IO or allocate memory */
Assert(CritSectionCount == 0);
switch (HeapTupleSatisfiesVacuumHorizon(&tuple, buf, &dead_after))
diff --git a/src/backend/access/rmgrdesc/heapdesc.c b/src/backend/access/rmgrdesc/heapdesc.c
index 75ae6f9d375..97f925df161 100644
--- a/src/backend/access/rmgrdesc/heapdesc.c
+++ b/src/backend/access/rmgrdesc/heapdesc.c
@@ -108,7 +108,8 @@ heap_xlog_deserialize_prune_and_freeze(char *cursor, uint16 flags,
OffsetNumber **frz_offsets,
int *nredirected, OffsetNumber **redirected,
int *ndead, OffsetNumber **nowdead,
- int *nunused, OffsetNumber **nowunused)
+ int *nunused, OffsetNumber **nowunused,
+ int *nstubs, OffsetNumber **stubs)
{
if (flags & XLHP_HAS_FREEZE_PLANS)
{
@@ -178,6 +179,23 @@ heap_xlog_deserialize_prune_and_freeze(char *cursor, uint16 flags,
*nowunused = NULL;
}
+ if (flags & XLHP_HAS_HOT_INDEXED_STUBS)
+ {
+ xlhp_prune_items *subrecord = (xlhp_prune_items *) cursor;
+
+ *nstubs = subrecord->ntargets;
+ Assert(*nstubs > 0);
+ *stubs = &subrecord->data[0];
+
+ cursor += offsetof(xlhp_prune_items, data);
+ cursor += sizeof(OffsetNumber[2]) * *nstubs;
+ }
+ else
+ {
+ *nstubs = 0;
+ *stubs = NULL;
+ }
+
*frz_offsets = (OffsetNumber *) cursor;
}
@@ -305,6 +323,8 @@ heap2_desc(StringInfo buf, XLogReaderState *record)
int nredirected;
int nunused;
int ndead;
+ int nstubs;
+ OffsetNumber *stubs;
int nplans;
xlhp_freeze_plan *plans;
OffsetNumber *frz_offsets;
@@ -315,10 +335,11 @@ heap2_desc(StringInfo buf, XLogReaderState *record)
&nplans, &plans, &frz_offsets,
&nredirected, &redirected,
&ndead, &nowdead,
- &nunused, &nowunused);
+ &nunused, &nowunused,
+ &nstubs, &stubs);
- appendStringInfo(buf, ", nplans: %u, nredirected: %u, ndead: %u, nunused: %u",
- nplans, nredirected, ndead, nunused);
+ appendStringInfo(buf, ", nplans: %u, nredirected: %u, ndead: %u, nunused: %u, nstubs: %u",
+ nplans, nredirected, ndead, nunused, nstubs);
if (nplans > 0)
{
@@ -347,6 +368,13 @@ heap2_desc(StringInfo buf, XLogReaderState *record)
array_desc(buf, nowunused, sizeof(OffsetNumber), nunused,
&offset_elem_desc, NULL);
}
+
+ if (nstubs > 0)
+ {
+ appendStringInfoString(buf, ", stubs:");
+ array_desc(buf, stubs, sizeof(OffsetNumber) * 2,
+ nstubs, &redirect_elem_desc, NULL);
+ }
}
}
else if (info == XLOG_HEAP2_MULTI_INSERT)
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 23676ae71e9..64ae9733103 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -490,7 +490,8 @@ extern void heap_page_prune_and_freeze(PruneFreezeParams *params,
extern void heap_page_prune_execute(Buffer buffer, bool lp_truncate_only,
OffsetNumber *redirected, int nredirected,
OffsetNumber *nowdead, int ndead,
- OffsetNumber *nowunused, int nunused);
+ OffsetNumber *nowunused, int nunused,
+ OffsetNumber *stubs, int nstubs);
extern void heap_get_root_tuples(Page page, OffsetNumber *root_offsets);
extern void log_heap_prune_and_freeze(Relation relation, Buffer buffer,
Buffer vmbuffer, uint8 vmflags,
@@ -500,7 +501,8 @@ extern void log_heap_prune_and_freeze(Relation relation, Buffer buffer,
HeapTupleFreeze *frozen, int nfrozen,
OffsetNumber *redirected, int nredirected,
OffsetNumber *dead, int ndead,
- OffsetNumber *unused, int nunused);
+ OffsetNumber *unused, int nunused,
+ OffsetNumber *stubs, int nstubs);
/* in heap/heapam.c */
diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h
index fdca7d821c8..8997a505006 100644
--- a/src/include/access/heapam_xlog.h
+++ b/src/include/access/heapam_xlog.h
@@ -273,6 +273,10 @@ typedef struct xl_heap_update
* uint16 nunused
* OffsetNumber nowunused[nunused]
*
+ * xlhp_prune_items
+ * uint16 nstubs
+ * OffsetNumber stubs[2 * nstubs]
+ *
* OffsetNumber frz_offsets[sum([plan.ntuples for plan in plans])]
*-----------------------------------------------------------------------------
*
@@ -341,6 +345,18 @@ typedef struct xl_heap_prune
#define XLHP_VM_ALL_VISIBLE (1 << 8)
#define XLHP_VM_ALL_FROZEN (1 << 9)
+/*
+ * Indicates that an xlhp_prune_items sub-record with HOT-selectively-updated
+ * collapse-survivor stubs is present. Each pair (offset, forward) names a
+ * line pointer to be rewritten in place into an xid-free forwarding stub
+ * (HEAP_XMIN_INVALID|HEAP_XMAX_INVALID, HEAP_ONLY_TUPLE|HEAP_INDEXED_UPDATED,
+ * natts==0) whose t_ctid.offnum is set to the forward offset. The stub's
+ * modified-attrs bitmap is already present in the item on the page (it is the
+ * pre-prune tuple's inline bitmap, left undisturbed), so it is not carried in
+ * the WAL.
+ */
+#define XLHP_HAS_HOT_INDEXED_STUBS (1 << 10)
+
/*
* xlhp_freeze_plan describes how to freeze a group of one or more heap tuples
* (appears in xl_heap_prune's xlhp_freeze_plans sub-record)
@@ -494,6 +510,7 @@ extern void heap_xlog_deserialize_prune_and_freeze(char *cursor, uint16 flags,
OffsetNumber **frz_offsets,
int *nredirected, OffsetNumber **redirected,
int *ndead, OffsetNumber **nowdead,
- int *nunused, OffsetNumber **nowunused);
+ int *nunused, OffsetNumber **nowunused,
+ int *nstubs, OffsetNumber **stubs);
#endif /* HEAPAM_XLOG_H */
--
2.50.1
[text/x-patch] v57-0006-Teach-amcheck-to-recognize-HOT-indexed-chains-an.patch (18.0K, ../[email protected]/7-v57-0006-Teach-amcheck-to-recognize-HOT-indexed-chains-an.patch)
download | inline diff:
From 0b7866fd77507be1f7e9b0d91e77af92e8b5d64e Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Wed, 17 Jun 2026 21:31:13 -0400
Subject: [PATCH v57 6/9] Teach amcheck to recognize HOT-indexed chains and
collapse stubs
verify_heapam must not flag the HOT-indexed artifacts as corruption: a live
HEAP_INDEXED_UPDATED heap-only tuple whose mid-chain line pointer is preserved
because an index entry still points at it, an xid-free collapse-survivor stub,
and more than one LP_REDIRECT forwarding to the same live tuple are all
legitimate. Recognize them and continue checking the rest of the chain.
Cover this with an amcheck regression test, and add a pg_upgrade test that
carries a relation with HOT-indexed chains, an ABA-cycled indexed column, an
out-of-line indexed column, and VACUUM-collapsed stubs across an upgrade,
verifying the data, verify_heapam, bt_index_check, and the chain scans on the
new cluster.
Authored-by: Greg Burd <[email protected]>
---
contrib/amcheck/expected/check_heap.out | 40 +++++++
contrib/amcheck/sql/check_heap.sql | 37 +++++++
contrib/amcheck/verify_heapam.c | 89 ++++++++++++++--
src/backend/access/heap/README.HOT-INDEXED | 10 ++
src/bin/pg_upgrade/meson.build | 1 +
src/bin/pg_upgrade/t/009_hot_indexed.pl | 118 +++++++++++++++++++++
6 files changed, 285 insertions(+), 10 deletions(-)
create mode 100644 src/bin/pg_upgrade/t/009_hot_indexed.pl
diff --git a/contrib/amcheck/expected/check_heap.out b/contrib/amcheck/expected/check_heap.out
index 979e5e84e72..b8dee2bb71b 100644
--- a/contrib/amcheck/expected/check_heap.out
+++ b/contrib/amcheck/expected/check_heap.out
@@ -231,6 +231,46 @@ SELECT * FROM verify_heapam('test_foreign_table',
endblock := NULL);
ERROR: cannot check relation "test_foreign_table"
DETAIL: This operation is not supported for foreign tables.
+-- HOT-indexed (HOT/SIU) on-page artifacts:
+--
+-- A HOT-indexed UPDATE keeps the new tuple on the same page as a heap-only
+-- tuple marked HEAP_INDEXED_UPDATED and plants index entries pointing at its
+-- own TID. Pruning a chain of such updates collapses dead members to
+-- LP_REDIRECT forwarders and preserves the LP of a live HOT-indexed member
+-- whose index entries may not yet be swept. verify_heapam must treat all of
+-- these as legitimate. This scenario exercises them and asserts that
+-- verify_heapam reports zero corruption against legitimate HOT-indexed
+-- activity.
+CREATE TABLE hot_indexed_check (id int PRIMARY KEY, c1 int, c2 int, c3 int)
+ WITH (fillfactor = 70);
+CREATE INDEX hot_indexed_check_c1 ON hot_indexed_check (c1);
+CREATE INDEX hot_indexed_check_c2 ON hot_indexed_check (c2);
+INSERT INTO hot_indexed_check
+ SELECT g, g, g, g FROM generate_series(1, 200) g;
+-- Single-step UPDATEs: each row gets one HOT-indexed update. Each
+-- successful HOT-indexed update keeps its new tuple on-page and inserts an
+-- entry only into the index whose attribute changed.
+UPDATE hot_indexed_check SET c1 = c1 + 1000;
+-- Multi-step UPDATEs: drive several successive HOT-indexed updates against
+-- the same rows so prune sees a chain of dead intermediates and collapses
+-- them to LP_REDIRECT forwarders. An explicit VACUUM runs the prune path
+-- and exercises chain collapse.
+UPDATE hot_indexed_check SET c2 = c2 + 1 WHERE id <= 50;
+UPDATE hot_indexed_check SET c2 = c2 + 1 WHERE id <= 50;
+UPDATE hot_indexed_check SET c2 = c2 + 1 WHERE id <= 50;
+VACUUM (INDEX_CLEANUP off) hot_indexed_check;
+-- verify_heapam must not report any corruption against legitimate HOT-
+-- indexed artifacts. Selecting the corrupting message makes any
+-- regression unmistakable in the regress diff.
+SELECT blkno, offnum, attnum, msg
+ FROM verify_heapam('hot_indexed_check',
+ startblock := NULL,
+ endblock := NULL);
+ blkno | offnum | attnum | msg
+-------+--------+--------+-----
+(0 rows)
+
+DROP TABLE hot_indexed_check;
-- cleanup
DROP TABLE heaptest;
DROP TABLESPACE regress_test_stats_tblspc;
diff --git a/contrib/amcheck/sql/check_heap.sql b/contrib/amcheck/sql/check_heap.sql
index 1745bae634e..c0ba2635180 100644
--- a/contrib/amcheck/sql/check_heap.sql
+++ b/contrib/amcheck/sql/check_heap.sql
@@ -138,6 +138,43 @@ SELECT * FROM verify_heapam('test_foreign_table',
startblock := NULL,
endblock := NULL);
+-- HOT-indexed (HOT/SIU) on-page artifacts:
+--
+-- A HOT-indexed UPDATE keeps the new tuple on the same page as a heap-only
+-- tuple marked HEAP_INDEXED_UPDATED and plants index entries pointing at its
+-- own TID. Pruning a chain of such updates collapses dead members to
+-- LP_REDIRECT forwarders and preserves the LP of a live HOT-indexed member
+-- whose index entries may not yet be swept. verify_heapam must treat all of
+-- these as legitimate. This scenario exercises them and asserts that
+-- verify_heapam reports zero corruption against legitimate HOT-indexed
+-- activity.
+CREATE TABLE hot_indexed_check (id int PRIMARY KEY, c1 int, c2 int, c3 int)
+ WITH (fillfactor = 70);
+CREATE INDEX hot_indexed_check_c1 ON hot_indexed_check (c1);
+CREATE INDEX hot_indexed_check_c2 ON hot_indexed_check (c2);
+INSERT INTO hot_indexed_check
+ SELECT g, g, g, g FROM generate_series(1, 200) g;
+-- Single-step UPDATEs: each row gets one HOT-indexed update. Each
+-- successful HOT-indexed update keeps its new tuple on-page and inserts an
+-- entry only into the index whose attribute changed.
+UPDATE hot_indexed_check SET c1 = c1 + 1000;
+-- Multi-step UPDATEs: drive several successive HOT-indexed updates against
+-- the same rows so prune sees a chain of dead intermediates and collapses
+-- them to LP_REDIRECT forwarders. An explicit VACUUM runs the prune path
+-- and exercises chain collapse.
+UPDATE hot_indexed_check SET c2 = c2 + 1 WHERE id <= 50;
+UPDATE hot_indexed_check SET c2 = c2 + 1 WHERE id <= 50;
+UPDATE hot_indexed_check SET c2 = c2 + 1 WHERE id <= 50;
+VACUUM (INDEX_CLEANUP off) hot_indexed_check;
+-- verify_heapam must not report any corruption against legitimate HOT-
+-- indexed artifacts. Selecting the corrupting message makes any
+-- regression unmistakable in the regress diff.
+SELECT blkno, offnum, attnum, msg
+ FROM verify_heapam('hot_indexed_check',
+ startblock := NULL,
+ endblock := NULL);
+DROP TABLE hot_indexed_check;
+
-- cleanup
DROP TABLE heaptest;
DROP TABLESPACE regress_test_stats_tblspc;
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 20ff58aa782..93a3bc318a2 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -13,6 +13,7 @@
#include "access/detoast.h"
#include "access/genam.h"
#include "access/heaptoast.h"
+#include "access/hot_indexed.h"
#include "access/multixact.h"
#include "access/relation.h"
#include "access/table.h"
@@ -522,9 +523,12 @@ verify_heapam(PG_FUNCTION_ARGS)
*/
if (ItemIdIsRedirected(ctx.itemid))
{
- OffsetNumber rdoffnum = ItemIdGetRedirect(ctx.itemid);
+ OffsetNumber rdoffnum;
ItemId rditem;
+ /* Resolve the redirect's target offset. */
+ rdoffnum = ItemIdGetRedirect(ctx.itemid);
+
if (rdoffnum < FirstOffsetNumber)
{
report_corruption(&ctx,
@@ -615,18 +619,47 @@ verify_heapam(PG_FUNCTION_ARGS)
ctx.tuphdr = (HeapTupleHeader) PageGetItem(ctx.page, ctx.itemid);
ctx.natts = HeapTupleHeaderGetNatts(ctx.tuphdr);
- /* Ok, ready to check this next tuple */
- check_tuple(&ctx,
- &xmin_commit_status_ok[ctx.offnum],
- &xmin_commit_status[ctx.offnum]);
+ /*
+ * A HOT-selectively-updated collapse-survivor stub is an
+ * LP_NORMAL item that is not a real tuple: HEAP_INDEXED_UPDATED
+ * with natts == 0, permanently invisible (HEAP_XMIN_INVALID),
+ * carrying a forward link and a modified-attrs bitmap. The
+ * per-tuple checks assume a real tuple and would misreport it, so
+ * skip them; the update-chain pass below still records its
+ * forward edge and treats it like a redirect (a forwarding node).
+ */
+ if (!HotIndexedHeaderIsStub(ctx.tuphdr))
+ check_tuple(&ctx,
+ &xmin_commit_status_ok[ctx.offnum],
+ &xmin_commit_status[ctx.offnum]);
/*
* If the CTID field of this tuple seems to point to another tuple
* on the same page, record that tuple as the successor of this
- * one.
+ * one. A collapse-survivor stub stores its forward link in the
+ * t_ctid offset only (the block half is repurposed to hold the
+ * stub's write-time natts), so resolve its successor via the stub
+ * accessor; the forward target is always on the same page.
*/
- nextblkno = ItemPointerGetBlockNumber(&(ctx.tuphdr)->t_ctid);
- nextoffnum = ItemPointerGetOffsetNumber(&(ctx.tuphdr)->t_ctid);
+ if (HotIndexedHeaderIsStub(ctx.tuphdr))
+ {
+ nextblkno = ctx.blkno;
+ nextoffnum = HotIndexedStubGetForward(ctx.tuphdr);
+ if (nextoffnum == ctx.offnum ||
+ nextoffnum < FirstOffsetNumber || nextoffnum > maxoff)
+ {
+ report_corruption(&ctx,
+ psprintf("HOT-indexed stub forward link to item at offset %d is out of range or self-referential (valid range %d..%d)",
+ nextoffnum,
+ FirstOffsetNumber, maxoff));
+ continue;
+ }
+ }
+ else
+ {
+ nextblkno = ItemPointerGetBlockNumber(&(ctx.tuphdr)->t_ctid);
+ nextoffnum = ItemPointerGetOffsetNumber(&(ctx.tuphdr)->t_ctid);
+ }
if (nextblkno == ctx.blkno && nextoffnum != ctx.offnum &&
nextoffnum >= FirstOffsetNumber && nextoffnum <= maxoff)
successor[ctx.offnum] = nextoffnum;
@@ -675,7 +708,7 @@ verify_heapam(PG_FUNCTION_ARGS)
*/
Assert(ItemIdIsNormal(next_lp));
- /* Can only redirect to a HOT tuple. */
+ /* A redirect targets the first surviving chain member. */
next_htup = (HeapTupleHeader) PageGetItem(ctx.page, next_lp);
if (!HeapTupleHeaderIsHeapOnly(next_htup))
{
@@ -687,6 +720,19 @@ verify_heapam(PG_FUNCTION_ARGS)
/* HOT chains should not intersect. */
if (predecessor[nextoffnum] != InvalidOffsetNumber)
{
+ /*
+ * In the HOT/SIU model several redirects legitimately
+ * forward to the same live tuple: when a chain collapses,
+ * the root and each entry-bearing dead member become a
+ * redirect to first_live so every stale btree entry still
+ * resolves there (the read path then rechecks the leaf
+ * key). Multiple predecessors are therefore expected
+ * when the target is HOT-selectively-updated; keep the
+ * first predecessor and do not report it as corruption.
+ */
+ if ((next_htup->t_infomask2 & HEAP_INDEXED_UPDATED) != 0)
+ continue;
+
report_corruption(&ctx,
psprintf("redirect line pointer points to offset %d, but offset %d also points there",
nextoffnum, predecessor[nextoffnum]));
@@ -701,6 +747,30 @@ verify_heapam(PG_FUNCTION_ARGS)
continue;
}
+ /*
+ * A collapse-survivor stub forwards like a redirect: it is not a
+ * real tuple, so don't apply the tuple-to-tuple update-chain
+ * checks, but do record the predecessor edge to its target so the
+ * live tuple it ultimately forwards to is not mistaken for a
+ * chain root. Its target must be heap-only (another stub or the
+ * live heap-only tuple).
+ */
+ curr_htup = (HeapTupleHeader) PageGetItem(ctx.page, curr_lp);
+ if (HotIndexedHeaderIsStub(curr_htup))
+ {
+ if (ItemIdIsNormal(next_lp))
+ {
+ next_htup = (HeapTupleHeader) PageGetItem(ctx.page, next_lp);
+ if (!HeapTupleHeaderIsHeapOnly(next_htup))
+ report_corruption(&ctx,
+ psprintf("HOT-indexed stub forwards to a non-heap-only tuple at offset %d",
+ nextoffnum));
+ else if (predecessor[nextoffnum] == InvalidOffsetNumber)
+ predecessor[nextoffnum] = ctx.offnum;
+ }
+ continue;
+ }
+
/*
* If the next line pointer is a redirect, or if it's a tuple but
* the XMAX of this tuple doesn't match the XMIN of the next
@@ -709,7 +779,6 @@ verify_heapam(PG_FUNCTION_ARGS)
*/
if (ItemIdIsRedirected(next_lp))
continue;
- curr_htup = (HeapTupleHeader) PageGetItem(ctx.page, curr_lp);
curr_xmax = HeapTupleHeaderGetUpdateXid(curr_htup);
next_htup = (HeapTupleHeader) PageGetItem(ctx.page, next_lp);
next_xmin = HeapTupleHeaderGetXmin(next_htup);
diff --git a/src/backend/access/heap/README.HOT-INDEXED b/src/backend/access/heap/README.HOT-INDEXED
index ab4f8bc1881..1f41b0fffe8 100644
--- a/src/backend/access/heap/README.HOT-INDEXED
+++ b/src/backend/access/heap/README.HOT-INDEXED
@@ -244,6 +244,16 @@ stale entries (enforced in heap_prune_record_redirect, the stub recorders, and
heap_page_would_be_all_visible).
+amcheck and statistics
+----------------------
+
+verify_heapam treats the HOT-indexed artifacts as legitimate: a live
+HEAP_INDEXED_UPDATED heap-only tuple whose line pointer is preserved, and
+multiple LP_REDIRECTs forwarding to one live tuple.
+
+Statistics: pg_stat_all_tables.n_tup_hot_indexed_upd counts HOT-indexed
+
+
Appendices
----------
diff --git a/src/bin/pg_upgrade/meson.build b/src/bin/pg_upgrade/meson.build
index ffbf6ae8d75..0a6fa0dcff2 100644
--- a/src/bin/pg_upgrade/meson.build
+++ b/src/bin/pg_upgrade/meson.build
@@ -69,6 +69,7 @@ tests += {
't/006_transfer_modes.pl',
't/007_multixact_conversion.pl',
't/008_extension_control_path.pl',
+ 't/009_hot_indexed.pl',
],
'deps': [test_ext],
'test_kwargs': {'priority': 40}, # pg_upgrade tests are slow
diff --git a/src/bin/pg_upgrade/t/009_hot_indexed.pl b/src/bin/pg_upgrade/t/009_hot_indexed.pl
new file mode 100644
index 00000000000..b71129ae03d
--- /dev/null
+++ b/src/bin/pg_upgrade/t/009_hot_indexed.pl
@@ -0,0 +1,118 @@
+# Copyright (c) 2026, PostgreSQL Global Development Group
+
+# pg_upgrade must preserve HOT-indexed on-disk state. A relation that has
+# accumulated HOT-indexed chains -- including a value cycled away and back
+# (ABA), an out-of-line (TOAST) indexed column, and chains collapsed to
+# xid-free forwarding stubs by VACUUM -- must come through an upgrade with its
+# data intact, its indexes structurally sound, and its chains still scanning
+# correctly. pg_upgrade transfers heap and index files verbatim, so this is
+# really a check that the new-state bits (HEAP_INDEXED_UPDATED, collapse stubs)
+# are not rejected by pg_upgrade's checks and stay correct on the new cluster.
+
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $mode = $ENV{PG_TEST_PG_UPGRADE_MODE} || '--copy';
+
+my $oldnode = PostgreSQL::Test::Cluster->new('old_node');
+$oldnode->init;
+$oldnode->start;
+
+# Build a relation with several secondary indexes so single-column updates
+# stay HOT-indexed, then exercise the cases that produce interesting on-disk
+# state.
+$oldnode->safe_psql('postgres', q{
+ CREATE EXTENSION amcheck;
+ CREATE TABLE hi (id int PRIMARY KEY, k int, v int, big text)
+ WITH (fillfactor = 50);
+ ALTER TABLE hi ALTER COLUMN big SET STORAGE EXTERNAL;
+ CREATE INDEX hi_k ON hi (k);
+ CREATE INDEX hi_v ON hi (v);
+ CREATE INDEX hi_big ON hi (big);
+ INSERT INTO hi SELECT g, g, g * 10, repeat(chr(64 + g), 2000)
+ FROM generate_series(1, 20) g;
+});
+
+# Interleave updates of different indexed columns on the same rows. A member
+# that changed a column not changed again by a later hop survives VACUUM as a
+# collapse stub; the rest are reclaimed. Row 1 additionally cycles k away and
+# back (ABA), and row 2 rewrites its toasted indexed column.
+$oldnode->safe_psql('postgres', q{
+ UPDATE hi SET k = k + 100 WHERE id <= 10; -- changes k
+ UPDATE hi SET v = v + 1 WHERE id <= 10; -- changes v (survives as stub)
+ UPDATE hi SET k = k - 100 WHERE id <= 10; -- k back to original (ABA)
+ UPDATE hi SET big = repeat('Z', 2000) WHERE id = 2;
+});
+# Collapse dead chain members to stubs.
+$oldnode->safe_psql('postgres', 'VACUUM (INDEX_CLEANUP off) hi');
+
+# The pre-upgrade state must already be self-consistent.
+is( $oldnode->safe_psql('postgres',
+ q{SELECT count(*) FROM verify_heapam('hi')}),
+ '0', 'pre-upgrade heap is consistent');
+
+# Snapshot the data we will compare after the upgrade.
+my $expect = $oldnode->safe_psql('postgres',
+ q{SELECT id, k, v, length(big) FROM hi ORDER BY id});
+
+$oldnode->stop;
+
+# New cluster, same version.
+my $newnode = PostgreSQL::Test::Cluster->new('new_node');
+$newnode->init;
+
+my $oldbindir = $oldnode->config_data('--bindir');
+my $newbindir = $newnode->config_data('--bindir');
+
+# Run pg_upgrade from a writable directory (matches 002_pg_upgrade).
+chdir ${PostgreSQL::Test::Utils::tmp_check};
+
+command_ok(
+ [
+ 'pg_upgrade', '--no-sync',
+ '--old-datadir' => $oldnode->data_dir,
+ '--new-datadir' => $newnode->data_dir,
+ '--old-bindir' => $oldbindir,
+ '--new-bindir' => $newbindir,
+ '--socketdir' => $newnode->host,
+ '--old-port' => $oldnode->port,
+ '--new-port' => $newnode->port,
+ $mode,
+ ],
+ 'run of pg_upgrade for HOT-indexed relation');
+
+$newnode->start;
+
+# Data survived intact.
+my $got = $newnode->safe_psql('postgres',
+ q{SELECT id, k, v, length(big) FROM hi ORDER BY id});
+is($got, $expect, 'HOT-indexed table data preserved across pg_upgrade');
+
+# Heap and indexes are structurally sound on the new cluster.
+is( $newnode->safe_psql('postgres',
+ q{SELECT count(*) FROM verify_heapam('hi')}),
+ '0', 'post-upgrade heap is consistent (collapse stubs recognised)');
+is( $newnode->safe_psql('postgres', q{
+ SELECT count(*) FROM (
+ SELECT bt_index_check(c.oid)
+ FROM pg_class c JOIN pg_index i ON i.indexrelid = c.oid
+ WHERE i.indrelid = 'hi'::regclass) s}),
+ '4', 'post-upgrade indexes pass bt_index_check');
+
+# The ABA chain on row 1 still scans correctly through a forced index scan:
+# k=1 returns exactly the one live row, and its superseded value is gone.
+is( $newnode->safe_psql('postgres', q{
+ SET enable_seqscan = off; SET enable_bitmapscan = off;
+ SELECT count(*) FROM hi WHERE k = 1}),
+ '1', 'post-upgrade index scan returns the ABA row once');
+is( $newnode->safe_psql('postgres', q{
+ SET enable_seqscan = off; SET enable_bitmapscan = off;
+ SELECT count(*) FROM hi WHERE k = 101}),
+ '0', 'post-upgrade index scan drops the superseded value');
+
+$newnode->stop;
+
+done_testing();
--
2.50.1
[text/x-patch] v57-0007-Add-HOT-indexed-statistics-and-the-comprehensive.patch (168.0K, ../[email protected]/8-v57-0007-Add-HOT-indexed-statistics-and-the-comprehensive.patch)
download | inline diff:
From 59e64fdf0882b4bdee15db18ac319f7e3a991546 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Wed, 17 Jun 2026 21:38:11 -0400
Subject: [PATCH v57 7/9] Add HOT-indexed statistics and the comprehensive test
suite
Expose the HOT-indexed activity counters maintained by the write path:
pg_stat_all_tables.n_tup_hot_indexed_upd, the per-index
n_tup_hot_indexed_upd_matched / n_tup_hot_indexed_upd_skipped counters in
pg_stat_all_indexes, and pg_relation_hot_indexed_stats() reporting per-relation
HOT-indexed chain composition. Document them in monitoring.sgml and the
README.
With statistics, prune/collapse, and amcheck recognition all in place, add the
full feature test suite, which uses those facilities to verify behavior:
- hot_indexed_updates (regression): eligibility and classification; selective
maintenance across multiple/composite indexes; the crossed-attribute read
path for equality, range, and inequality scans; a key cycled away and back
(ABA), including across two distinct live rows; TOASTed indexed columns;
partial-index predicate flips (key and non-key predicate columns);
trigger-modified indexed columns; exclusion-constraint tables; partitioned
tables; non-btree access methods (hash, GIN, GiST); a UNIQUE index on a type
where image equality differs from operator equality; CREATE INDEX / REINDEX
and DROP INDEX over live chains; prune reclamation, stub mixes, and
re-collapse across partial VACUUMs; the never-all-visible guard; and DDL
after a chain exists (ADD COLUMN crossing a bitmap-size boundary, DROP
COLUMN).
- hot_indexed_adversarial (isolation): concurrent UPDATE / VACUUM / prune and
index scans, key cycling, aborts, and reader consistency across a concurrent
collapse.
- 054_hot_indexed_recovery (recovery): WAL replay of the chain and its collapse
under wal_consistency_checking.
- pg_surgery handling of HOT-indexed tuples and collapse-survivor stubs.
Authored-by: Greg Burd <[email protected]>
---
contrib/pg_surgery/Makefile | 1 +
contrib/pg_surgery/expected/heap_surgery.out | 83 +
contrib/pg_surgery/heap_surgery.c | 16 +
contrib/pg_surgery/sql/heap_surgery.sql | 53 +
doc/src/sgml/monitoring.sgml | 57 +
src/backend/access/heap/Makefile | 1 +
src/backend/access/heap/README.HOT-INDEXED | 3 +
src/backend/access/heap/hot_indexed_stats.c | 188 ++
src/backend/access/heap/meson.build | 1 +
src/backend/catalog/system_views.sql | 4 +
src/backend/utils/adt/pgstatfuncs.c | 12 +
src/include/catalog/pg_proc.dat | 28 +
.../expected/hot_indexed_adversarial.out | 139 ++
src/test/isolation/isolation_schedule | 1 +
.../specs/hot_indexed_adversarial.spec | 123 ++
src/test/recovery/Makefile | 3 +-
src/test/recovery/meson.build | 1 +
.../recovery/t/055_hot_indexed_recovery.pl | 149 ++
.../regress/expected/hot_indexed_updates.out | 1719 +++++++++++++++++
src/test/regress/expected/rules.out | 12 +
src/test/regress/parallel_schedule | 1 +
src/test/regress/sql/hot_indexed_updates.sql | 1167 +++++++++++
22 files changed, 3761 insertions(+), 1 deletion(-)
create mode 100644 src/backend/access/heap/hot_indexed_stats.c
create mode 100644 src/test/isolation/expected/hot_indexed_adversarial.out
create mode 100644 src/test/isolation/specs/hot_indexed_adversarial.spec
create mode 100644 src/test/recovery/t/055_hot_indexed_recovery.pl
create mode 100644 src/test/regress/expected/hot_indexed_updates.out
create mode 100644 src/test/regress/sql/hot_indexed_updates.sql
diff --git a/contrib/pg_surgery/Makefile b/contrib/pg_surgery/Makefile
index a66776c4c41..da752a81147 100644
--- a/contrib/pg_surgery/Makefile
+++ b/contrib/pg_surgery/Makefile
@@ -10,6 +10,7 @@ DATA = pg_surgery--1.0.sql
PGFILEDESC = "pg_surgery - perform surgery on a damaged relation"
REGRESS = heap_surgery
+EXTRA_INSTALL = contrib/pageinspect
ifdef USE_PGXS
PG_CONFIG = pg_config
diff --git a/contrib/pg_surgery/expected/heap_surgery.out b/contrib/pg_surgery/expected/heap_surgery.out
index df7d13b0908..1a5fef5b2bb 100644
--- a/contrib/pg_surgery/expected/heap_surgery.out
+++ b/contrib/pg_surgery/expected/heap_surgery.out
@@ -175,6 +175,89 @@ DETAIL: This operation is not supported for views.
select heap_force_freeze('vw'::regclass, ARRAY['(0, 1)']::tid[]);
ERROR: cannot operate on relation "vw"
DETAIL: This operation is not supported for views.
+-- A HOT/SIU chain collapse turns the chain root and each dead entry-bearing
+-- member into an LP_REDIRECT to the live tuple. pg_surgery operates on real
+-- tuples and must leave the live row reachable after such a collapse.
+create extension pageinspect;
+create table htomb (id int primary key, a int, b int) with (fillfactor = 50);
+create index htomb_a on htomb(a);
+insert into htomb values (1, 10, 20);
+-- Two HOT-indexed updates on an indexed attr, then prune: the dead mid-chain
+-- versions collapse to LP_REDIRECTs to the live tuple. INDEX_CLEANUP off keeps
+-- the stale btree leaves (and hence the redirects) in place.
+update htomb set a = 11 where id = 1;
+update htomb set a = 12 where id = 1;
+vacuum (index_cleanup off) htomb;
+select n_hot_indexed > 0 as made_hot_indexed
+ from pg_relation_hot_indexed_stats('htomb');
+ made_hot_indexed
+------------------
+ t
+(1 row)
+
+-- the live row is intact and reachable after the collapse
+select id, a, b from htomb;
+ id | a | b
+----+----+----
+ 1 | 12 | 20
+(1 row)
+
+drop table htomb;
+-- A collapse that keeps a *stub* (an xid-free forwarding LP_NORMAL item with
+-- natts == 0), not just redirects: update two different indexed columns so the
+-- first dead member's changed-attr bitmap is not subsumed by later hops and is
+-- preserved as a stub. pg_surgery must skip such a stub -- forcing a
+-- freeze/kill would overwrite its t_ctid forward link and corrupt the chain.
+create table hstub (id int primary key, a int, b int) with (fillfactor = 50);
+create index hstub_a on hstub(a);
+create index hstub_b on hstub(b);
+insert into hstub values (1, 10, 100);
+update hstub set a = 11 where id = 1; -- changes a
+update hstub set a = 12 where id = 1; -- changes a again (supersedes)
+update hstub set b = 101 where id = 1; -- changes b -> first hop's {a} kept as stub
+vacuum (index_cleanup off) hstub;
+-- Locate the stub: an LP_NORMAL item carrying HEAP_INDEXED_UPDATED (0x0800 in
+-- t_infomask2) with zero live attributes (t_infomask2 natts bits == 0).
+select lp as stub_off
+ from heap_page_items(get_raw_page('hstub', 0))
+ where lp_flags = 1 -- LP_NORMAL
+ and (t_infomask2 & 2048) <> 0 -- HEAP_INDEXED_UPDATED
+ and (t_infomask2 & 2047) = 0 -- natts == 0 (stub sentinel)
+ \gset
+-- Force kill/freeze on the stub's tid: both must be refused with a NOTICE.
+select heap_force_kill('hstub'::regclass, ARRAY[('(0,' || :'stub_off' || ')')]::tid[]);
+NOTICE: skipping tid (0, 3) for relation "hstub" because it is a HOT-indexed collapse stub
+ heap_force_kill
+-----------------
+
+(1 row)
+
+select heap_force_freeze('hstub'::regclass, ARRAY[('(0,' || :'stub_off' || ')')]::tid[]);
+NOTICE: skipping tid (0, 3) for relation "hstub" because it is a HOT-indexed collapse stub
+ heap_force_freeze
+-------------------
+
+(1 row)
+
+-- The chain is untouched: the live row is still reachable through each index.
+set enable_seqscan = off;
+set enable_bitmapscan = off;
+select id, a, b from hstub where a = 12;
+ id | a | b
+----+----+-----
+ 1 | 12 | 101
+(1 row)
+
+select id, a, b from hstub where b = 101;
+ id | a | b
+----+----+-----
+ 1 | 12 | 101
+(1 row)
+
+reset enable_bitmapscan;
+reset enable_seqscan;
+drop table hstub;
+drop extension pageinspect;
-- cleanup.
drop view vw;
drop extension pg_surgery;
diff --git a/contrib/pg_surgery/heap_surgery.c b/contrib/pg_surgery/heap_surgery.c
index b8ce1095782..7fc653cc334 100644
--- a/contrib/pg_surgery/heap_surgery.c
+++ b/contrib/pg_surgery/heap_surgery.c
@@ -13,6 +13,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/hot_indexed.h"
#include "access/relation.h"
#include "access/visibilitymap.h"
#include "access/xloginsert.h"
@@ -226,6 +227,21 @@ heap_force_common(FunctionCallInfo fcinfo, HeapTupleForceOption heap_force_opt)
blkno, offno, RelationGetRelationName(rel))));
continue;
}
+ else if (HotIndexedHeaderIsStub((HeapTupleHeader) PageGetItem(page, itemid)))
+ {
+ /*
+ * A HOT-indexed collapse-survivor stub is an xid-free
+ * forwarding node, not a real tuple: its t_ctid carries the
+ * chain forward link and the write-time natts, and it has no
+ * attribute data. Forcing a kill or freeze would overwrite
+ * t_ctid and clear its xact bits, breaking the chain walk and
+ * corrupting the heap. Skip it, as we do for redirects.
+ */
+ ereport(NOTICE,
+ (errmsg("skipping tid (%u, %u) for relation \"%s\" because it is a HOT-indexed collapse stub",
+ blkno, offno, RelationGetRelationName(rel))));
+ continue;
+ }
/* Mark it for processing. */
Assert(offno <= MaxHeapTuplesPerPage);
diff --git a/contrib/pg_surgery/sql/heap_surgery.sql b/contrib/pg_surgery/sql/heap_surgery.sql
index 6526b27535d..be4fed2af65 100644
--- a/contrib/pg_surgery/sql/heap_surgery.sql
+++ b/contrib/pg_surgery/sql/heap_surgery.sql
@@ -83,6 +83,59 @@ create view vw as select 1;
select heap_force_kill('vw'::regclass, ARRAY['(0, 1)']::tid[]);
select heap_force_freeze('vw'::regclass, ARRAY['(0, 1)']::tid[]);
+-- A HOT/SIU chain collapse turns the chain root and each dead entry-bearing
+-- member into an LP_REDIRECT to the live tuple. pg_surgery operates on real
+-- tuples and must leave the live row reachable after such a collapse.
+create extension pageinspect;
+create table htomb (id int primary key, a int, b int) with (fillfactor = 50);
+create index htomb_a on htomb(a);
+insert into htomb values (1, 10, 20);
+-- Two HOT-indexed updates on an indexed attr, then prune: the dead mid-chain
+-- versions collapse to LP_REDIRECTs to the live tuple. INDEX_CLEANUP off keeps
+-- the stale btree leaves (and hence the redirects) in place.
+update htomb set a = 11 where id = 1;
+update htomb set a = 12 where id = 1;
+vacuum (index_cleanup off) htomb;
+select n_hot_indexed > 0 as made_hot_indexed
+ from pg_relation_hot_indexed_stats('htomb');
+-- the live row is intact and reachable after the collapse
+select id, a, b from htomb;
+drop table htomb;
+
+-- A collapse that keeps a *stub* (an xid-free forwarding LP_NORMAL item with
+-- natts == 0), not just redirects: update two different indexed columns so the
+-- first dead member's changed-attr bitmap is not subsumed by later hops and is
+-- preserved as a stub. pg_surgery must skip such a stub -- forcing a
+-- freeze/kill would overwrite its t_ctid forward link and corrupt the chain.
+create table hstub (id int primary key, a int, b int) with (fillfactor = 50);
+create index hstub_a on hstub(a);
+create index hstub_b on hstub(b);
+insert into hstub values (1, 10, 100);
+update hstub set a = 11 where id = 1; -- changes a
+update hstub set a = 12 where id = 1; -- changes a again (supersedes)
+update hstub set b = 101 where id = 1; -- changes b -> first hop's {a} kept as stub
+vacuum (index_cleanup off) hstub;
+-- Locate the stub: an LP_NORMAL item carrying HEAP_INDEXED_UPDATED (0x0800 in
+-- t_infomask2) with zero live attributes (t_infomask2 natts bits == 0).
+select lp as stub_off
+ from heap_page_items(get_raw_page('hstub', 0))
+ where lp_flags = 1 -- LP_NORMAL
+ and (t_infomask2 & 2048) <> 0 -- HEAP_INDEXED_UPDATED
+ and (t_infomask2 & 2047) = 0 -- natts == 0 (stub sentinel)
+ \gset
+-- Force kill/freeze on the stub's tid: both must be refused with a NOTICE.
+select heap_force_kill('hstub'::regclass, ARRAY[('(0,' || :'stub_off' || ')')]::tid[]);
+select heap_force_freeze('hstub'::regclass, ARRAY[('(0,' || :'stub_off' || ')')]::tid[]);
+-- The chain is untouched: the live row is still reachable through each index.
+set enable_seqscan = off;
+set enable_bitmapscan = off;
+select id, a, b from hstub where a = 12;
+select id, a, b from hstub where b = 101;
+reset enable_bitmapscan;
+reset enable_seqscan;
+drop table hstub;
+drop extension pageinspect;
+
-- cleanup.
drop view vw;
drop extension pg_surgery;
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..f34aa648b59 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4457,6 +4457,19 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>n_tup_hot_indexed_upd</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of rows updated using the HOT-indexed path: the successor
+ version stays on the same page as a heap-only tuple even though it
+ changed one or more indexed columns, and only the affected indexes
+ receive a new entry. Every such update is also counted in
+ <structfield>n_tup_hot_upd</structfield>.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>n_tup_newpage_upd</structfield> <type>bigint</type>
@@ -4927,6 +4940,27 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>n_tup_hot_indexed_upd_matched</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of HOT-indexed updates that inserted a new entry into this
+ index (the update changed one of this index's attributes)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>n_tup_hot_indexed_upd_skipped</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of HOT-indexed updates that skipped this index (the update
+ changed no attribute of this index, so its existing entry continues
+ to resolve the HOT chain)
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>stats_reset</structfield> <type>timestamp with time zone</type>
@@ -5608,6 +5642,29 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</thead>
<tbody>
+ <row>
+ <entry id="pg-relation-hot-indexed-stats" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_relation_hot_indexed_stats</primary>
+ </indexterm>
+ <function>pg_relation_hot_indexed_stats</function> ( <parameter>relation</parameter> <type>regclass</type> )
+ <returnvalue>record</returnvalue>
+ ( <parameter>n_hot_indexed</parameter> <type>bigint</type>,
+ <parameter>n_chains</parameter> <type>bigint</type>,
+ <parameter>avg_chain_len</parameter> <type>double precision</type>,
+ <parameter>max_chain_len</parameter> <type>bigint</type> )
+ </para>
+ <para>
+ Reports HOT-indexed structural statistics for a table by scanning
+ every page under <literal>AccessShareLock</literal>:
+ <parameter>n_hot_indexed</parameter> is the number of live
+ HOT-indexed tuple versions present, and
+ <parameter>n_chains</parameter>, <parameter>avg_chain_len</parameter>
+ and <parameter>max_chain_len</parameter> describe the HOT-indexed
+ chains. Intended for diagnostics.
+ </para></entry>
+ </row>
+
<row>
<!-- See also the entry for this in func.sgml -->
<entry role="func_table_entry"><para role="func_signature">
diff --git a/src/backend/access/heap/Makefile b/src/backend/access/heap/Makefile
index 1d27ccb916e..dfaf3350736 100644
--- a/src/backend/access/heap/Makefile
+++ b/src/backend/access/heap/Makefile
@@ -20,6 +20,7 @@ OBJS = \
heapam_xlog.o \
heaptoast.o \
hio.o \
+ hot_indexed_stats.o \
pruneheap.o \
rewriteheap.o \
vacuumlazy.o \
diff --git a/src/backend/access/heap/README.HOT-INDEXED b/src/backend/access/heap/README.HOT-INDEXED
index 1f41b0fffe8..2e5e5f4a081 100644
--- a/src/backend/access/heap/README.HOT-INDEXED
+++ b/src/backend/access/heap/README.HOT-INDEXED
@@ -252,6 +252,9 @@ HEAP_INDEXED_UPDATED heap-only tuple whose line pointer is preserved, and
multiple LP_REDIRECTs forwarding to one live tuple.
Statistics: pg_stat_all_tables.n_tup_hot_indexed_upd counts HOT-indexed
+updates; pg_stat_all_indexes.n_tup_hot_indexed_upd_matched / _skipped count
+per-index recheck outcomes; and pg_relation_hot_indexed_stats() reports
+per-relation HOT-indexed chain counts.
Appendices
diff --git a/src/backend/access/heap/hot_indexed_stats.c b/src/backend/access/heap/hot_indexed_stats.c
new file mode 100644
index 00000000000..895a29654b5
--- /dev/null
+++ b/src/backend/access/heap/hot_indexed_stats.c
@@ -0,0 +1,188 @@
+/*-------------------------------------------------------------------------
+ *
+ * hot_indexed_stats.c
+ * SQL-callable diagnostic that walks every page of a heap relation and
+ * reports hot-indexed-related structural statistics.
+ *
+ * These numbers complement the running pgstat counters
+ * (n_tup_hot_indexed_upd in pg_stat_all_tables): they answer "what is on disk
+ * right now?" rather than "how often did hot-indexed fire during the stats
+ * window?".
+ *
+ * Portions Copyright (c) 1996-2026, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/access/heap/hot_indexed_stats.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/heapam.h"
+#include "access/hot_indexed.h"
+#include "access/htup_details.h"
+#include "catalog/objectaddress.h"
+#include "catalog/pg_type.h"
+#include "fmgr.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "storage/bufmgr.h"
+#include "storage/bufpage.h"
+#include "storage/itemptr.h"
+#include "utils/acl.h"
+#include "utils/builtins.h"
+#include "utils/rel.h"
+
+/*
+ * pg_relation_hot_indexed_stats(regclass) -> record
+ *
+ * Walks every block of the relation's main fork and counts:
+ * n_hot_indexed -- live HOT-indexed versions (HEAP_INDEXED_UPDATED, natts>0,
+ * carrying an inline-trailing modified-attrs bitmap)
+ * n_chains -- LP_REDIRECT items, i.e. HOT chain roots. Matches
+ * the number of distinct HOT chains that have survived
+ * the most recent prune. Root-not-redirect chains
+ * (length 1) are not counted here because they are
+ * indistinguishable from a non-chain tuple.
+ * avg_chain_len -- mean length across chains rooted at an LP_REDIRECT,
+ * derived by walking each redirect target to the end
+ * of its HEAP_HOT_UPDATED chain.
+ * max_chain_len -- longest chain observed.
+ *
+ * The caller must hold SELECT privilege on the relation, like other
+ * relation-inspection functions; it takes only AccessShareLock and short-term
+ * buffer share locks while scanning.
+ */
+Datum
+pg_relation_hot_indexed_stats(PG_FUNCTION_ARGS)
+{
+ Oid relid = PG_GETARG_OID(0);
+ Relation rel;
+ AclResult aclresult;
+ BlockNumber nblocks;
+ BlockNumber blk;
+ int64 n_hot_indexed = 0;
+ int64 n_chains = 0;
+ int64 sum_chain_len = 0;
+ int64 max_chain_len = 0;
+ TupleDesc tupdesc;
+ Datum values[4];
+ bool nulls[4] = {0};
+ HeapTuple resulttup;
+
+ rel = relation_open(relid, AccessShareLock);
+ if (rel->rd_rel->relkind != RELKIND_RELATION &&
+ rel->rd_rel->relkind != RELKIND_MATVIEW &&
+ rel->rd_rel->relkind != RELKIND_TOASTVALUE)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not a table, materialized view, or TOAST table",
+ RelationGetRelationName(rel))));
+
+ /* Caller must be able to read the relation. */
+ aclresult = pg_class_aclcheck(relid, GetUserId(), ACL_SELECT);
+ if (aclresult != ACLCHECK_OK)
+ aclcheck_error(aclresult,
+ get_relkind_objtype(rel->rd_rel->relkind),
+ RelationGetRelationName(rel));
+
+ nblocks = RelationGetNumberOfBlocks(rel);
+
+ for (blk = 0; blk < nblocks; blk++)
+ {
+ Buffer buf;
+ Page page;
+ OffsetNumber off;
+ OffsetNumber maxoff;
+
+ CHECK_FOR_INTERRUPTS();
+
+ buf = ReadBufferExtended(rel, MAIN_FORKNUM, blk, RBM_NORMAL, NULL);
+ LockBuffer(buf, BUFFER_LOCK_SHARE);
+ page = BufferGetPage(buf);
+
+ if (PageIsNew(page) || PageIsEmpty(page))
+ {
+ UnlockReleaseBuffer(buf);
+ continue;
+ }
+
+ maxoff = PageGetMaxOffsetNumber(page);
+ for (off = FirstOffsetNumber; off <= maxoff; off = OffsetNumberNext(off))
+ {
+ ItemId lp = PageGetItemId(page, off);
+
+ if (!ItemIdIsUsed(lp))
+ continue;
+
+ if (ItemIdIsRedirected(lp))
+ {
+ /* Walk the chain starting at the redirect target. */
+ OffsetNumber cur = ItemIdGetRedirect(lp);
+ int64 len = 0;
+
+ /*
+ * Walk the same-page HOT chain. Bound the loop by the page's
+ * item count so a corrupt cyclic t_ctid cannot spin forever
+ * under the buffer lock, and check for interrupts each step.
+ */
+ while (cur >= FirstOffsetNumber && cur <= maxoff && len < maxoff)
+ {
+ ItemId chain_lp = PageGetItemId(page, cur);
+ HeapTupleHeader thdr;
+
+ CHECK_FOR_INTERRUPTS();
+
+ if (!ItemIdIsNormal(chain_lp))
+ break;
+ thdr = (HeapTupleHeader) PageGetItem(page, chain_lp);
+ len++;
+ if (!(thdr->t_infomask2 & HEAP_HOT_UPDATED))
+ break;
+ /* HOT chains stay on one page; stop if the link leaves it. */
+ if (ItemPointerGetBlockNumber(&thdr->t_ctid) != blk)
+ break;
+ cur = ItemPointerGetOffsetNumber(&thdr->t_ctid);
+ }
+ if (len > 0)
+ {
+ n_chains++;
+ sum_chain_len += len;
+ if (len > max_chain_len)
+ max_chain_len = len;
+ }
+ }
+ else if (ItemIdIsNormal(lp))
+ {
+ HeapTupleHeader thdr = (HeapTupleHeader) PageGetItem(page, lp);
+
+ if (!HotIndexedHeaderIsStub(thdr) &&
+ (thdr->t_infomask2 & HEAP_INDEXED_UPDATED) != 0)
+ n_hot_indexed++;
+ }
+ }
+
+ UnlockReleaseBuffer(buf);
+ }
+
+ relation_close(rel, AccessShareLock);
+
+ tupdesc = CreateTemplateTupleDesc(4);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "n_hot_indexed", INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "n_chains", INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "avg_chain_len", FLOAT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "max_chain_len", INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ tupdesc = BlessTupleDesc(tupdesc);
+
+ values[0] = Int64GetDatum(n_hot_indexed);
+ values[1] = Int64GetDatum(n_chains);
+ if (n_chains > 0)
+ values[2] = Float8GetDatum(((double) sum_chain_len) / (double) n_chains);
+ else
+ values[2] = Float8GetDatum(0.0);
+ values[3] = Int64GetDatum(max_chain_len);
+
+ resulttup = heap_form_tuple(tupdesc, values, nulls);
+ PG_RETURN_DATUM(HeapTupleGetDatum(resulttup));
+}
diff --git a/src/backend/access/heap/meson.build b/src/backend/access/heap/meson.build
index 00ec07d7f30..b5c2a8d5cb6 100644
--- a/src/backend/access/heap/meson.build
+++ b/src/backend/access/heap/meson.build
@@ -8,6 +8,7 @@ backend_sources += files(
'heapam_xlog.c',
'heaptoast.c',
'hio.c',
+ 'hot_indexed_stats.c',
'pruneheap.c',
'rewriteheap.c',
'vacuumlazy.c',
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 6c1c5545cb5..02c8a049a32 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -730,6 +730,7 @@ CREATE VIEW pg_stat_all_tables AS
pg_stat_get_tuples_updated(C.oid) AS n_tup_upd,
pg_stat_get_tuples_deleted(C.oid) AS n_tup_del,
pg_stat_get_tuples_hot_updated(C.oid) AS n_tup_hot_upd,
+ pg_stat_get_tuples_hot_indexed_updated(C.oid) AS n_tup_hot_indexed_upd,
pg_stat_get_tuples_newpage_updated(C.oid) AS n_tup_newpage_upd,
pg_stat_get_live_tuples(C.oid) AS n_live_tup,
pg_stat_get_dead_tuples(C.oid) AS n_dead_tup,
@@ -768,6 +769,7 @@ CREATE VIEW pg_stat_xact_all_tables AS
pg_stat_get_xact_tuples_updated(C.oid) AS n_tup_upd,
pg_stat_get_xact_tuples_deleted(C.oid) AS n_tup_del,
pg_stat_get_xact_tuples_hot_updated(C.oid) AS n_tup_hot_upd,
+ pg_stat_get_xact_tuples_hot_indexed_updated(C.oid) AS n_tup_hot_indexed_upd,
pg_stat_get_xact_tuples_newpage_updated(C.oid) AS n_tup_newpage_upd
FROM pg_class C LEFT JOIN
pg_index I ON C.oid = I.indrelid
@@ -869,6 +871,8 @@ CREATE VIEW pg_stat_all_indexes AS
pg_stat_get_lastscan(I.oid) AS last_idx_scan,
pg_stat_get_tuples_returned(I.oid) AS idx_tup_read,
pg_stat_get_tuples_fetched(I.oid) AS idx_tup_fetch,
+ pg_stat_get_tuples_hot_indexed_updated_skipped(I.oid) AS n_tup_hot_indexed_upd_skipped,
+ pg_stat_get_tuples_hot_indexed_updated_matched(I.oid) AS n_tup_hot_indexed_upd_matched,
pg_stat_get_stat_reset_time(I.oid) AS stats_reset
FROM pg_class C JOIN
pg_index X ON C.oid = X.indrelid JOIN
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..552f3540cde 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -93,6 +93,15 @@ PG_STAT_GET_RELENTRY_INT64(tuples_fetched)
/* pg_stat_get_tuples_hot_updated */
PG_STAT_GET_RELENTRY_INT64(tuples_hot_updated)
+/* pg_stat_get_tuples_hot_indexed_updated */
+PG_STAT_GET_RELENTRY_INT64(tuples_hot_indexed_updated)
+
+/* pg_stat_get_tuples_hot_indexed_updated_skipped */
+PG_STAT_GET_RELENTRY_INT64(tuples_hot_indexed_upd_skipped)
+
+/* pg_stat_get_tuples_hot_indexed_updated_matched */
+PG_STAT_GET_RELENTRY_INT64(tuples_hot_indexed_upd_matched)
+
/* pg_stat_get_tuples_newpage_updated */
PG_STAT_GET_RELENTRY_INT64(tuples_newpage_updated)
@@ -1888,6 +1897,9 @@ PG_STAT_GET_XACT_RELENTRY_INT64(tuples_fetched)
/* pg_stat_get_xact_tuples_hot_updated */
PG_STAT_GET_XACT_RELENTRY_INT64(tuples_hot_updated)
+/* pg_stat_get_xact_tuples_hot_indexed_updated */
+PG_STAT_GET_XACT_RELENTRY_INT64(tuples_hot_indexed_updated)
+
/* pg_stat_get_xact_tuples_newpage_updated */
PG_STAT_GET_XACT_RELENTRY_INT64(tuples_newpage_updated)
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..175fa88eb57 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5594,6 +5594,29 @@
proname => 'pg_stat_get_tuples_hot_updated', provolatile => 's',
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_tuples_hot_updated' },
+{ oid => '9953',
+ descr => 'statistics: number of tuples updated via HOT-indexed (Selective Index Update)',
+ proname => 'pg_stat_get_tuples_hot_indexed_updated', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_tuples_hot_indexed_updated' },
+{ oid => '9956',
+ descr => 'statistics: number of HOT-indexed updates that skipped this index',
+ proname => 'pg_stat_get_tuples_hot_indexed_updated_skipped', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_tuples_hot_indexed_upd_skipped' },
+{ oid => '9957',
+ descr => 'statistics: number of HOT-indexed updates that inserted into this index',
+ proname => 'pg_stat_get_tuples_hot_indexed_updated_matched', provolatile => 's',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_tuples_hot_indexed_upd_matched' },
+{ oid => '9955',
+ descr => 'HOT-indexed structural stats: HOT-indexed versions and chain lengths',
+ proname => 'pg_relation_hot_indexed_stats', provolatile => 'v',
+ proparallel => 'r', prorettype => 'record', proargtypes => 'regclass',
+ proallargtypes => '{regclass,int8,int8,float8,int8}',
+ proargmodes => '{i,o,o,o,o}',
+ proargnames => '{relation,n_hot_indexed,n_chains,avg_chain_len,max_chain_len}',
+ prosrc => 'pg_relation_hot_indexed_stats' },
{ oid => '6217',
descr => 'statistics: number of tuples updated onto a new page',
proname => 'pg_stat_get_tuples_newpage_updated', provolatile => 's',
@@ -6179,6 +6202,11 @@
proname => 'pg_stat_get_xact_tuples_hot_updated', provolatile => 'v',
proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
prosrc => 'pg_stat_get_xact_tuples_hot_updated' },
+{ oid => '9954',
+ descr => 'statistics: number of HOT-indexed tuple updates in current transaction',
+ proname => 'pg_stat_get_xact_tuples_hot_indexed_updated', provolatile => 'v',
+ proparallel => 'r', prorettype => 'int8', proargtypes => 'oid',
+ prosrc => 'pg_stat_get_xact_tuples_hot_indexed_updated' },
{ oid => '6218',
descr => 'statistics: number of tuples updated onto a new page in current transaction',
proname => 'pg_stat_get_xact_tuples_newpage_updated', provolatile => 'v',
diff --git a/src/test/isolation/expected/hot_indexed_adversarial.out b/src/test/isolation/expected/hot_indexed_adversarial.out
new file mode 100644
index 00000000000..22f62801513
--- /dev/null
+++ b/src/test/isolation/expected/hot_indexed_adversarial.out
@@ -0,0 +1,139 @@
+Parsed test spec with 6 sessions
+
+starting permutation: s2_noseq s1_cycle s2_eq10 s2_eq20
+step s2_noseq: SET enable_seqscan = off;
+step s1_cycle: UPDATE hia SET k = 20 WHERE id = 1; UPDATE hia SET k = 10 WHERE id = 1;
+step s2_eq10: SELECT id, k FROM hia WHERE k = 10;
+id| k
+--+--
+ 1|10
+(1 row)
+
+step s2_eq20: SELECT id, k FROM hia WHERE k = 20;
+id|k
+--+-
+(0 rows)
+
+
+starting permutation: s2_noseq s1_cycle s2_range
+step s2_noseq: SET enable_seqscan = off;
+step s1_cycle: UPDATE hia SET k = 20 WHERE id = 1; UPDATE hia SET k = 10 WHERE id = 1;
+step s2_range: SELECT id, k FROM hia WHERE k >= 5 ORDER BY k;
+id| k
+--+--
+ 1|10
+(1 row)
+
+
+starting permutation: s2_noseq s1_begin s1_upd20 s1_abort s2_eq20 s2_eq10
+step s2_noseq: SET enable_seqscan = off;
+step s1_begin: BEGIN;
+step s1_upd20: UPDATE hia SET k = 20 WHERE id = 1;
+step s1_abort: ROLLBACK;
+step s2_eq20: SELECT id, k FROM hia WHERE k = 20;
+id|k
+--+-
+(0 rows)
+
+step s2_eq10: SELECT id, k FROM hia WHERE k = 10;
+id| k
+--+--
+ 1|10
+(1 row)
+
+
+starting permutation: s1_begin s1_uupd20 s2_ins10 s1_commit
+step s1_begin: BEGIN;
+step s1_uupd20: UPDATE hiu SET k = 20 WHERE id = 1;
+step s2_ins10: INSERT INTO hiu VALUES (2, 10, repeat('y', 40)); <waiting ...>
+step s1_commit: COMMIT;
+step s2_ins10: <... completed>
+
+starting permutation: s1_begin s1_uupd20 s1_commit s2_ins10
+step s1_begin: BEGIN;
+step s1_uupd20: UPDATE hiu SET k = 20 WHERE id = 1;
+step s1_commit: COMMIT;
+step s2_ins10: INSERT INTO hiu VALUES (2, 10, repeat('y', 40));
+
+starting permutation: s1_begin s1_uupd20 s1_commit s2_ins20
+step s1_begin: BEGIN;
+step s1_uupd20: UPDATE hiu SET k = 20 WHERE id = 1;
+step s1_commit: COMMIT;
+step s2_ins20: INSERT INTO hiu VALUES (3, 20, repeat('z', 40));
+ERROR: duplicate key value violates unique constraint "hiu_k"
+
+starting permutation: s3_begin s3_eq10 s2_to30 s2_scan30 s3_eq10 s3_commit
+step s3_begin: BEGIN ISOLATION LEVEL REPEATABLE READ; SET enable_seqscan = off;
+step s3_eq10: SELECT id, k FROM hia WHERE k = 10;
+id| k
+--+--
+ 1|10
+(1 row)
+
+step s2_to30: UPDATE hia SET k = 20 WHERE id = 1; UPDATE hia SET k = 30 WHERE id = 1;
+step s2_scan30: SET enable_seqscan = off; SELECT id, k FROM hia WHERE k = 30;
+id| k
+--+--
+ 1|30
+(1 row)
+
+step s3_eq10: SELECT id, k FROM hia WHERE k = 10;
+id| k
+--+--
+ 1|10
+(1 row)
+
+step s3_commit: COMMIT;
+
+starting permutation: b1_begin b1_snap b2_update b2_vacuum b1_snap b1_commit b3_seq
+step b1_begin: BEGIN;
+step b1_snap: SELECT id, v FROM hib WHERE v = 400;
+id| v
+--+---
+ 1|400
+(1 row)
+
+step b2_update: UPDATE hib SET v = 500 WHERE id = 1;
+step b2_vacuum: VACUUM (INDEX_CLEANUP off) hib;
+step b1_snap: SELECT id, v FROM hib WHERE v = 400;
+id|v
+--+-
+(0 rows)
+
+step b1_commit: COMMIT;
+step b3_seq: SELECT id, v FROM hib ORDER BY id;
+id| v
+--+---
+ 1|500
+ 2| 20
+ 3| 30
+ 4| 40
+ 5| 50
+(5 rows)
+
+
+starting permutation: b1_begin b2_update b1_snap b2_vacuum b1_snap b1_commit b3_seq
+step b1_begin: BEGIN;
+step b2_update: UPDATE hib SET v = 500 WHERE id = 1;
+step b1_snap: SELECT id, v FROM hib WHERE v = 400;
+id|v
+--+-
+(0 rows)
+
+step b2_vacuum: VACUUM (INDEX_CLEANUP off) hib;
+step b1_snap: SELECT id, v FROM hib WHERE v = 400;
+id|v
+--+-
+(0 rows)
+
+step b1_commit: COMMIT;
+step b3_seq: SELECT id, v FROM hib ORDER BY id;
+id| v
+--+---
+ 1|500
+ 2| 20
+ 3| 30
+ 4| 40
+ 5| 50
+(5 rows)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index b8ebe92553c..e9afb48199e 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -128,3 +128,4 @@ test: matview-write-skew
test: lock-nowait
test: for-portion-of
test: ddl-dependency-locking
+test: hot_indexed_adversarial
diff --git a/src/test/isolation/specs/hot_indexed_adversarial.spec b/src/test/isolation/specs/hot_indexed_adversarial.spec
new file mode 100644
index 00000000000..64705bf37e0
--- /dev/null
+++ b/src/test/isolation/specs/hot_indexed_adversarial.spec
@@ -0,0 +1,123 @@
+# Adversarial correctness tests for HOT-indexed (SIU) updates.
+#
+# Each permutation pins a case that the mid-chain-pointing invariant must
+# satisfy: an index entry points at the heap-only version whose key it
+# matched, and a chain walk that crosses a HOT-indexed hop drops the arriving
+# entry when the crossed-attribute bitmap overlaps the index's key columns
+# (no key comparison). These are
+# exactly the cases that historically broke write-amplification-reduction
+# designs.
+
+setup
+{
+ CREATE TABLE hia (id int PRIMARY KEY, k int, pad text) WITH (fillfactor = 40);
+ CREATE INDEX hia_k ON hia(k);
+ CREATE TABLE hiu (id int PRIMARY KEY, k int, pad text) WITH (fillfactor = 40);
+ CREATE UNIQUE INDEX hiu_k ON hiu(k);
+ INSERT INTO hia VALUES (1, 10, repeat('x', 40));
+ INSERT INTO hiu VALUES (1, 10, repeat('x', 40));
+
+ -- Table for the concurrent-collapse reader-consistency case (7).
+ CREATE TABLE hib (id int PRIMARY KEY, v int, pad text) WITH (fillfactor = 50);
+ CREATE INDEX hib_v_idx ON hib(v);
+ INSERT INTO hib SELECT g, g * 10, repeat('x', 50) FROM generate_series(1, 5) g;
+ UPDATE hib SET v = 100 WHERE id = 1;
+ UPDATE hib SET v = 200 WHERE id = 1;
+ UPDATE hib SET v = 300 WHERE id = 1;
+ UPDATE hib SET v = 400 WHERE id = 1;
+}
+
+teardown
+{
+ DROP TABLE hia;
+ DROP TABLE hiu;
+ DROP TABLE hib;
+}
+
+session s1
+step s1_begin { BEGIN; }
+# Cycle the indexed key away and back: 10 -> 20 -> 10. The original 10 leaf
+# and the freshly-inserted 10 leaf both resolve to the live tuple; the chain
+# walk must drop the stale one so a lookup returns the row exactly once.
+step s1_cycle { UPDATE hia SET k = 20 WHERE id = 1; UPDATE hia SET k = 10 WHERE id = 1; }
+# A single HOT-indexed update on hia, used by the abort and reader cases.
+step s1_upd20 { UPDATE hia SET k = 20 WHERE id = 1; }
+# A HOT-indexed update on the UNIQUE-indexed table, freeing key 10 for k=20.
+step s1_uupd20 { UPDATE hiu SET k = 20 WHERE id = 1; }
+step s1_commit { COMMIT; }
+step s1_abort { ROLLBACK; }
+
+session s2
+# Index-only lookups (forced) that exercise the stale-leaf recheck.
+step s2_noseq { SET enable_seqscan = off; }
+step s2_eq10 { SELECT id, k FROM hia WHERE k = 10; }
+step s2_eq20 { SELECT id, k FROM hia WHERE k = 20; }
+step s2_range { SELECT id, k FROM hia WHERE k >= 5 ORDER BY k; }
+# Concurrent unique insert of the key s1 is freeing (10) and of the key s1 is
+# taking (20); _bt_check_unique must filter the stale 10 leaf but still
+# conflict on the live key.
+step s2_ins10 { INSERT INTO hiu VALUES (2, 10, repeat('y', 40)); }
+step s2_ins20 { INSERT INTO hiu VALUES (3, 20, repeat('z', 40)); }
+# Move the indexed key well away from 10 (two HOT-indexed hops) and force an
+# index scan on the new key. That scan reaches the live tuple through the
+# stale 10 leaf and may try to kill it for bloat reclaim.
+step s2_to30 { UPDATE hia SET k = 20 WHERE id = 1; UPDATE hia SET k = 30 WHERE id = 1; }
+step s2_scan30 { SET enable_seqscan = off; SELECT id, k FROM hia WHERE k = 30; }
+
+# Reader holding an older REPEATABLE READ snapshot that still sees k=10.
+session s3
+step s3_begin { BEGIN ISOLATION LEVEL REPEATABLE READ; SET enable_seqscan = off; }
+step s3_eq10 { SELECT id, k FROM hia WHERE k = 10; }
+step s3_commit { COMMIT; }
+
+session b1
+step b1_begin { BEGIN; }
+# Reader takes a snapshot and reads the chain via the secondary index.
+step b1_snap { SELECT id, v FROM hib WHERE v = 400; }
+step b1_commit { COMMIT; }
+
+session b2
+# Force prune/collapse: another HOT-indexed update plus a VACUUM that
+# collapses the dead chain members to LP_REDIRECT forwarders.
+step b2_update { UPDATE hib SET v = 500 WHERE id = 1; }
+step b2_vacuum { VACUUM (INDEX_CLEANUP off) hib; }
+
+session b3
+step b3_seq { SELECT id, v FROM hib ORDER BY id; }
+
+# 1. a->b->a cycle: exactly one row for the cycled-back key, none for the
+# transient key.
+permutation s2_noseq s1_cycle s2_eq10 s2_eq20
+
+# 2. Range scan returns the live row exactly once across the stale+fresh
+# leaves left by the cycle.
+permutation s2_noseq s1_cycle s2_range
+
+# 3. Abort of a HOT-indexed update: the new key must not surface and the old
+# key must still resolve to the (restored) live tuple.
+permutation s2_noseq s1_begin s1_upd20 s1_abort s2_eq20 s2_eq10
+
+# 4. Concurrent unique insert while a HOT-indexed update is in flight. An
+# insert of the key s1 is freeing (10) must wait on the uncommitted updater,
+# then succeed once it commits (the stale 10 leaf is filtered).
+permutation s1_begin s1_uupd20 s2_ins10 s1_commit
+
+# 5. After the update commits, the freed key (10) inserts cleanly and the
+# taken key (20) conflicts -- the live leaf is honoured, the stale one is not.
+permutation s1_begin s1_uupd20 s1_commit s2_ins10
+permutation s1_begin s1_uupd20 s1_commit s2_ins20
+
+# 6. Snapshot safety of stale-leaf reclaim. An older REPEATABLE READ reader
+# still sees k=10; a concurrent session then moves the key to 30 and runs an
+# index scan that reaches the live tuple through the stale 10 leaf and may
+# try to reclaim it. The reclaim is gated on the skipped versions being
+# dead to all transactions, which s3's held snapshot prevents, so s3 must
+# still find the row by k=10 after the scan.
+permutation s3_begin s3_eq10 s2_to30 s2_scan30 s3_eq10 s3_commit
+
+# 7. Reader consistency across a concurrent prune/collapse. s1's index scan,
+# crossing the collapsed chain after the VACUUM, must still return the row
+# via the crossed-attribute bitmap; the query must not error and the row count
+# must be consistent.
+permutation b1_begin b1_snap b2_update b2_vacuum b1_snap b1_commit b3_seq
+permutation b1_begin b2_update b1_snap b2_vacuum b1_snap b1_commit b3_seq
diff --git a/src/test/recovery/Makefile b/src/test/recovery/Makefile
index d41aaaf8ae1..2736caa1a1b 100644
--- a/src/test/recovery/Makefile
+++ b/src/test/recovery/Makefile
@@ -9,7 +9,8 @@
#
#-------------------------------------------------------------------------
-EXTRA_INSTALL=contrib/pg_prewarm \
+EXTRA_INSTALL=contrib/amcheck \
+ contrib/pg_prewarm \
contrib/pg_stat_statements \
contrib/test_decoding \
src/test/modules/injection_points
diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build
index ad0d85f4189..8dbcda35775 100644
--- a/src/test/recovery/meson.build
+++ b/src/test/recovery/meson.build
@@ -63,6 +63,7 @@ tests += {
't/052_checkpoint_segment_missing.pl',
't/053_standby_login_event_trigger.pl',
't/054_unlogged_sequence_promotion.pl',
+ 't/055_hot_indexed_recovery.pl',
],
},
}
diff --git a/src/test/recovery/t/055_hot_indexed_recovery.pl b/src/test/recovery/t/055_hot_indexed_recovery.pl
new file mode 100644
index 00000000000..b6f077b3159
--- /dev/null
+++ b/src/test/recovery/t/055_hot_indexed_recovery.pl
@@ -0,0 +1,149 @@
+
+# Copyright (c) 2026, PostgreSQL Global Development Group
+
+# Crash-recovery coverage for HOT-indexed (HOT/SIU) chains.
+#
+# Build a HOT-indexed chain by repeatedly UPDATEing a single row,
+# changing one indexed (non-PK) column each time. Force a prune so the
+# dead chain members collapse to LP_REDIRECT forwarders (with the live
+# HOT-indexed version visible via pg_relation_hot_indexed_stats as
+# n_hot_indexed > 0). Crash-recover the primary with stop('immediate')
+# so the collapsed chain comes back from WAL or from the FPI. After
+# restart, verify:
+#
+# 1. an index lookup walking the chain returns the live tuple,
+# 2. pg_amcheck (verify_heapam) reports no errors on the relation,
+# 3. VACUUM reclaims the collapsed chain (n_hot_indexed drops to 0).
+#
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $node = PostgreSQL::Test::Cluster->new('primary');
+$node->init;
+# Disable autovacuum to keep the chain shape stable up to the explicit
+# prune we trigger below.
+$node->append_conf('postgresql.conf', q{autovacuum = off
+wal_consistency_checking = 'all'});
+$node->start;
+
+# amcheck (verify_heapam) is shipped as a contrib extension; we use it
+# from SQL after the crash-restart cycle.
+$node->safe_psql('postgres', q{CREATE EXTENSION amcheck});
+
+# Wide-ish table: PK + four indexed columns plus a non-indexed payload
+# so HOT-indexed updates have width to amortise. fillfactor = 50 keeps
+# free space on-page for HOT-indexed continuations.
+$node->safe_psql('postgres', q{
+ CREATE TABLE hi_recov (
+ id int PRIMARY KEY,
+ c1 int,
+ c2 int,
+ c3 int,
+ c4 int,
+ payload text
+ ) WITH (fillfactor = 50);
+ CREATE INDEX hi_recov_c1 ON hi_recov(c1);
+ CREATE INDEX hi_recov_c2 ON hi_recov(c2);
+ CREATE INDEX hi_recov_c3 ON hi_recov(c3);
+ CREATE INDEX hi_recov_c4 ON hi_recov(c4);
+ INSERT INTO hi_recov VALUES (1, 100, 200, 300, 400, 'payload');
+});
+
+# Build a HOT-indexed chain: five UPDATEs, each touching one indexed
+# column. Every UPDATE keeps the new version on-page and plants a fresh
+# index entry because c1 is indexed and changed. Use a SQL transaction-
+# range loop so each UPDATE is its own xact (xmin/xmax distinct).
+for my $i (1 .. 5)
+{
+ my $newval = 100 + $i;
+ $node->safe_psql('postgres',
+ "UPDATE hi_recov SET c1 = $newval WHERE id = 1");
+}
+
+my $pre_prune = $node->safe_psql('postgres',
+ q{SELECT n_hot_indexed FROM pg_relation_hot_indexed_stats('hi_recov')});
+cmp_ok($pre_prune, '>', 0,
+ 'HOT-indexed chain has at least one live HOT-indexed version before prune');
+
+# Force a prune. The chain has dead heap-only members from the early
+# UPDATEs (their xmins are now committed and below the snapshot horizon).
+# A SELECT under default isolation visits the page; under
+# default_statistics_target etc. that's not enough on its own to trigger
+# prune. The reliable way to drive opportunistic prune is a query that
+# exercises the heap_page_prune_opt path, which fires from an indexscan
+# that finds the page non-all-visible. Use a sequential scan plus a
+# subsequent UPDATE that itself looks for free space (heap_update calls
+# heap_page_prune_opt).
+$node->safe_psql('postgres', q{
+ SET enable_indexscan = off;
+ SELECT count(*) FROM hi_recov;
+ UPDATE hi_recov SET payload = 'pruned' WHERE id = 1;
+});
+
+# Read the chain state after the prune: the live HOT-indexed version
+# remains while dead members collapse to LP_REDIRECT forwarders.
+my $post_prune = $node->safe_psql('postgres',
+ q{SELECT n_hot_indexed FROM pg_relation_hot_indexed_stats('hi_recov')});
+cmp_ok($post_prune, '>', 0,
+ 'live HOT-indexed version survives opportunistic prune');
+
+# Crash-restart. stop('immediate') is the standard "kill -9" simulation
+# used elsewhere in src/test/recovery/. We intentionally do NOT issue a
+# CHECKPOINT first: that would advance the redo point past the HOT-indexed
+# UPDATE/prune records and leave nothing for recovery to replay. Crashing
+# without a checkpoint forces startup redo to reconstruct the collapsed
+# chain from WAL, and wal_consistency_checking = 'all' (set above) compares
+# each replayed page against its full-page image, catching any divergence
+# between the write path and the redo path.
+$node->stop('immediate');
+$node->start;
+
+# 1. Chain walk via the indexed column on the live row returns the
+# correct (and only the correct) tuple. c1 = 105 was the last
+# UPDATE, so the live tuple has c1 = 105 and c2..c4 unchanged.
+my $live = $node->safe_psql('postgres', q{
+ SET enable_seqscan = off;
+ SELECT id, c1, c2, c3, c4, payload FROM hi_recov WHERE c1 = 105;
+});
+is($live, "1|105|200|300|400|pruned",
+ 'index lookup on chain returns the post-prune live tuple');
+
+# Older c1 values are not reachable: every stale btree entry that
+# chain-resolves across a HOT/SIU hop must be dropped by the read-side
+# crossed-attribute bitmap.
+my $stale_count = $node->safe_psql('postgres',
+ q{SELECT count(*) FROM hi_recov WHERE c1 = 100});
+is($stale_count, '0',
+ 'stale btree entries are filtered by the crossed-attribute bitmap');
+
+# 2. verify_heapam reports no errors on the relation (skip_option =
+# 'all-frozen' is the default; we want to scan everything).
+my $heapcheck = $node->safe_psql('postgres', q{
+ SELECT count(*) FROM verify_heapam('hi_recov',
+ skip := 'none',
+ check_toast := false);
+});
+is($heapcheck, '0',
+ 'verify_heapam reports zero errors after crash recovery');
+
+# 3. Reclamation: after the live row is deleted, two VACUUM (FREEZE)
+# passes drive prune to revisit the page and reclaim the now-fully-dead
+# collapsed chain (the first removes the dead row's index entries and
+# reduces its LP; the second reclaims the unreferenced members and
+# re-points the redirect). After that, n_hot_indexed must be zero.
+$node->safe_psql('postgres', q{DELETE FROM hi_recov WHERE id = 1});
+$node->safe_psql('postgres',
+ q{VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) hi_recov});
+$node->safe_psql('postgres',
+ q{VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) hi_recov});
+my $final = $node->safe_psql('postgres',
+ q{SELECT n_hot_indexed FROM pg_relation_hot_indexed_stats('hi_recov')});
+is($final, '0',
+ 'two VACUUM (FREEZE) passes after DELETE reclaim the chain post-recovery');
+
+$node->stop;
+
+done_testing();
diff --git a/src/test/regress/expected/hot_indexed_updates.out b/src/test/regress/expected/hot_indexed_updates.out
new file mode 100644
index 00000000000..9030f7f6851
--- /dev/null
+++ b/src/test/regress/expected/hot_indexed_updates.out
@@ -0,0 +1,1719 @@
+--
+-- HOT_INDEXED_UPDATES
+-- Test HOT-indexed update (hot-indexed), aka HOT-indexed, behaviour
+--
+-- Every UPDATE in this file modifies at least one non-summarizing
+-- indexed attribute. On a pre-hot-indexed server all of these would be
+-- non-HOT; on the hot-indexed branch each eligible update stays on-page and
+-- inserts into only the indexes whose attributes actually changed.
+--
+-- We verify four things:
+-- (A) pg_stat counters: HOT and hot-indexed counts increment as expected
+-- (B) index lookups return the new value and not the stale value
+-- for EQUALITY queries (the read-side staleness test drops a
+-- leaf whose covered attribute changed on the way to the live tuple)
+-- (C) pg_relation_hot_indexed_stats reports the HOT-indexed versions we expect
+-- (D) **RANGE/INEQUALITY** queries return the correct number of
+-- tuples -- this is the class of bugs where a stale btree
+-- entry's key is still reachable via a looser scan key; the
+-- crossed-attribute bitmap drops the stale arrival because the index's
+-- attribute changed between that leaf's target and the live tuple
+--
+CREATE EXTENSION IF NOT EXISTS pageinspect;
+CREATE OR REPLACE FUNCTION get_hot_count(rel_name text)
+RETURNS TABLE (updates BIGINT, hot BIGINT) AS $$
+DECLARE rel_oid oid;
+BEGIN
+ rel_oid := rel_name::regclass::oid;
+ updates := COALESCE(pg_stat_get_tuples_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_updated(rel_oid), 0);
+ hot := COALESCE(pg_stat_get_tuples_hot_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_hot_updated(rel_oid), 0);
+ RETURN NEXT;
+END;
+$$ LANGUAGE plpgsql;
+CREATE OR REPLACE FUNCTION get_hi_count(rel_name text)
+RETURNS TABLE (updates BIGINT, hot BIGINT, hot_idx BIGINT) AS $$
+DECLARE rel_oid oid;
+BEGIN
+ rel_oid := rel_name::regclass::oid;
+ updates := COALESCE(pg_stat_get_tuples_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_updated(rel_oid), 0);
+ hot := COALESCE(pg_stat_get_tuples_hot_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_hot_updated(rel_oid), 0);
+ hot_idx := COALESCE(pg_stat_get_tuples_hot_indexed_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_hot_indexed_updated(rel_oid), 0);
+ RETURN NEXT;
+END;
+$$ LANGUAGE plpgsql;
+-- ---------------------------------------------------------------------------
+-- 1. Basic hot-indexed: modifying an indexed column stays HOT and counts as hot-indexed
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_basic (
+ id int PRIMARY KEY,
+ indexed_col int,
+ non_indexed_col text
+) WITH (fillfactor = 50);
+CREATE INDEX hi_basic_idx ON hi_basic(indexed_col);
+INSERT INTO hi_basic VALUES (1, 100, 'initial');
+-- Pre-hot-indexed this would be non-HOT. Under hot-indexed it's HOT-indexed; both the
+-- HOT counter and the hot-indexed counter advance.
+UPDATE hi_basic SET indexed_col = 150 WHERE id = 1;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT * FROM get_hi_count('hi_basic');
+ updates | hot | hot_idx
+---------+-----+---------
+ 1 | 1 | 1
+(1 row)
+
+-- The new value is reachable via the index.
+SET enable_seqscan = off;
+EXPLAIN (COSTS OFF) SELECT id, indexed_col FROM hi_basic WHERE indexed_col = 150;
+ QUERY PLAN
+-----------------------------------------
+ Bitmap Heap Scan on hi_basic
+ Recheck Cond: (indexed_col = 150)
+ -> Bitmap Index Scan on hi_basic_idx
+ Index Cond: (indexed_col = 150)
+(4 rows)
+
+SELECT id, indexed_col FROM hi_basic WHERE indexed_col = 150;
+ id | indexed_col
+----+-------------
+ 1 | 150
+(1 row)
+
+-- The old value is not reachable through this index: the stale btree
+-- entry (indexed_col=100) walks to the current tuple via the hot-indexed hop,
+-- nodeIndexscan re-evaluates `indexed_col = 100` against the current
+-- tuple (indexed_col=150), and the row is correctly dropped. This is
+-- the equality-lookup case the crossed-attribute bitmap handles.
+EXPLAIN (COSTS OFF) SELECT id FROM hi_basic WHERE indexed_col = 100;
+ QUERY PLAN
+-----------------------------------------
+ Bitmap Heap Scan on hi_basic
+ Recheck Cond: (indexed_col = 100)
+ -> Bitmap Index Scan on hi_basic_idx
+ Index Cond: (indexed_col = 100)
+(4 rows)
+
+SELECT id FROM hi_basic WHERE indexed_col = 100;
+ id
+----
+(0 rows)
+
+RESET enable_seqscan;
+-- pg_relation_hot_indexed_stats sees one HOT-indexed version, zero HOT redirects (the
+-- chain has not yet been pruned so no LP_REDIRECT exists).
+SELECT n_hot_indexed, n_chains, avg_chain_len, max_chain_len
+FROM pg_relation_hot_indexed_stats('hi_basic');
+ n_hot_indexed | n_chains | avg_chain_len | max_chain_len
+---------------+----------+---------------+---------------
+ 1 | 0 | 0 | 0
+(1 row)
+
+DROP TABLE hi_basic;
+-- ---------------------------------------------------------------------------
+-- 2. RANGE/INEQUALITY correctness after hot-indexed on an indexed column
+--
+-- This is the test class that catches the hot-indexed false-dup bug: a stale
+-- btree entry whose key value still satisfies the range predicate,
+-- reachable via the hot-indexed chain hop.
+--
+-- To exercise the bug we must force an IndexScan plan (the
+-- IndexOnlyScan path permissively drops every hot-indexed-reachable index-only
+-- hit; the BitmapHeapScan path dedups by TID). We include a payload
+-- column not present in the PK so the planner must heap-fetch.
+--
+-- The read-side crossed-attribute bitmap makes the IndexScan return the correct
+-- count of 1: the stale entry ('1','5') chain-walks to the live tuple across
+-- the b-changing hop, and because the PK covers b the overlap is non-empty, so
+-- the stale leaf is dropped. The fresh entry ('1','15') points directly at the
+-- live tuple (no hop after it) and is kept. The ORDER BY likewise returns the
+-- single live row.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_range (
+ a int,
+ b int,
+ payload text,
+ PRIMARY KEY (a, b)
+) WITH (fillfactor = 50);
+INSERT INTO hi_range VALUES (1, 5, 'hi');
+-- hot-indexed update on the second PK column: stale btree entry ('1','5')
+-- remains, new entry ('1','15') inserted. The stale entry points at
+-- the chain root; the fresh entry points directly at the new
+-- heap-only tuple.
+UPDATE hi_range SET b = 15 WHERE a = 1 AND b = 5;
+SET enable_seqscan = off;
+SET enable_bitmapscan = off;
+-- IndexScan: payload IS NOT NULL forces heap fetch, no IndexOnlyScan.
+-- The stale ('1','5') leaf is dropped by the crossed-attribute bitmap, so this
+-- returns 1.
+EXPLAIN (COSTS OFF)
+SELECT count(*) FROM hi_range WHERE a = 1 AND b < 100 AND payload IS NOT NULL;
+ QUERY PLAN
+--------------------------------------------------
+ Aggregate
+ -> Index Scan using hi_range_pkey on hi_range
+ Index Cond: ((a = 1) AND (b < 100))
+ Filter: (payload IS NOT NULL)
+(4 rows)
+
+SELECT count(*) FROM hi_range WHERE a = 1 AND b < 100 AND payload IS NOT NULL;
+ count
+-------
+ 1
+(1 row)
+
+SELECT a, b FROM hi_range WHERE a = 1 AND payload IS NOT NULL ORDER BY b;
+ a | b
+---+----
+ 1 | 15
+(1 row)
+
+-- IndexOnlyScan: the page holds a preserved HOT-indexed member so it is never all-visible; IOS
+-- performs the heap fetch and the crossed-attribute bitmap drops the stale ('1','5')
+-- leaf, so count = 1.
+EXPLAIN (COSTS OFF) SELECT count(*) FROM hi_range WHERE a = 1 AND b < 100;
+ QUERY PLAN
+-------------------------------------------------------
+ Aggregate
+ -> Index Only Scan using hi_range_pkey on hi_range
+ Index Cond: ((a = 1) AND (b < 100))
+(3 rows)
+
+SELECT count(*) FROM hi_range WHERE a = 1 AND b < 100;
+ count
+-------
+ 1
+(1 row)
+
+-- BitmapHeapScan: TID dedup collapses the stale and fresh hits.
+SET enable_indexscan = off;
+SET enable_indexonlyscan = off;
+RESET enable_bitmapscan;
+EXPLAIN (COSTS OFF) SELECT count(*) FROM hi_range WHERE a = 1 AND b < 100;
+ QUERY PLAN
+---------------------------------------------------
+ Aggregate
+ -> Bitmap Heap Scan on hi_range
+ Recheck Cond: ((a = 1) AND (b < 100))
+ -> Bitmap Index Scan on hi_range_pkey
+ Index Cond: ((a = 1) AND (b < 100))
+(5 rows)
+
+SELECT count(*) FROM hi_range WHERE a = 1 AND b < 100;
+ count
+-------
+ 1
+(1 row)
+
+RESET enable_indexscan;
+RESET enable_indexonlyscan;
+-- SeqScan: reads the heap directly, sees exactly one live tuple.
+RESET enable_seqscan;
+SET enable_indexscan = off;
+SET enable_indexonlyscan = off;
+SET enable_bitmapscan = off;
+EXPLAIN (COSTS OFF) SELECT count(*) FROM hi_range WHERE a = 1 AND b < 100;
+ QUERY PLAN
+-----------------------------------------
+ Aggregate
+ -> Seq Scan on hi_range
+ Filter: ((b < 100) AND (a = 1))
+(3 rows)
+
+SELECT count(*) FROM hi_range WHERE a = 1 AND b < 100;
+ count
+-------
+ 1
+(1 row)
+
+RESET enable_indexscan;
+RESET enable_indexonlyscan;
+RESET enable_bitmapscan;
+-- Same shape on a secondary (non-PK) btree: another hot-indexed update on b.
+CREATE INDEX hi_range_b_idx ON hi_range(b);
+UPDATE hi_range SET b = 25 WHERE a = 1 AND b = 15;
+SET enable_seqscan = off;
+SET enable_bitmapscan = off;
+-- IndexScan path on the secondary index; same fix applies.
+SELECT count(*) FROM hi_range WHERE b BETWEEN 0 AND 100 AND payload IS NOT NULL;
+ count
+-------
+ 1
+(1 row)
+
+RESET enable_seqscan;
+RESET enable_bitmapscan;
+DROP TABLE hi_range;
+-- ---------------------------------------------------------------------------
+-- 3. All-or-none on a multi-indexed table: hot-indexed only touches indexes
+-- whose attributes changed
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_multi (
+ id int PRIMARY KEY,
+ col_a int,
+ col_b int,
+ col_c int,
+ non_indexed text
+) WITH (fillfactor = 50);
+CREATE INDEX hi_multi_a_idx ON hi_multi(col_a);
+CREATE INDEX hi_multi_b_idx ON hi_multi(col_b);
+CREATE INDEX hi_multi_c_idx ON hi_multi(col_c);
+INSERT INTO hi_multi VALUES (1, 10, 20, 30, 'initial');
+-- col_a only: under hot-indexed this is HOT-indexed, and only hi_multi_a_idx
+-- gets a new entry. hi_multi_b_idx / hi_multi_c_idx keep pointing
+-- at the chain root.
+UPDATE hi_multi SET col_a = 15 WHERE id = 1;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT * FROM get_hi_count('hi_multi');
+ updates | hot | hot_idx
+---------+-----+---------
+ 1 | 1 | 1
+(1 row)
+
+-- Lookups on all three indexes return the row.
+SET enable_seqscan = off;
+SELECT id FROM hi_multi WHERE col_a = 15;
+ id
+----
+ 1
+(1 row)
+
+SELECT id FROM hi_multi WHERE col_b = 20;
+ id
+----
+ 1
+(1 row)
+
+SELECT id FROM hi_multi WHERE col_c = 30;
+ id
+----
+ 1
+(1 row)
+
+-- Old col_a value is unreachable by equality (stale entry dropped by the
+-- read-side crossed-attribute bitmap).
+SELECT id FROM hi_multi WHERE col_a = 10;
+ id
+----
+(0 rows)
+
+RESET enable_seqscan;
+DROP TABLE hi_multi;
+-- ---------------------------------------------------------------------------
+-- 4. Multi-column btree: hot-indexed on part of a composite key
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_composite (
+ id int PRIMARY KEY,
+ col_a int,
+ col_b int,
+ data text
+) WITH (fillfactor = 50);
+CREATE INDEX hi_composite_ab_idx ON hi_composite(col_a, col_b);
+INSERT INTO hi_composite VALUES (1, 10, 20, 'data');
+-- col_a is part of the composite key: hot-indexed.
+UPDATE hi_composite SET col_a = 15;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT * FROM get_hi_count('hi_composite');
+ updates | hot | hot_idx
+---------+-----+---------
+ 1 | 1 | 1
+(1 row)
+
+-- Reset and then update col_b (also part of the key).
+UPDATE hi_composite SET col_a = 10;
+UPDATE hi_composite SET col_b = 25;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT * FROM get_hi_count('hi_composite');
+ updates | hot | hot_idx
+---------+-----+---------
+ 3 | 3 | 3
+(1 row)
+
+DROP TABLE hi_composite;
+-- ---------------------------------------------------------------------------
+-- 5. Partial index: status transition out-of-predicate
+--
+-- 'status' is a partial-index predicate column. A change to a predicate
+-- column can flip a row in or out of the index, which the read-side key
+-- recheck cannot detect, so HeapUpdateHotAllowable conservatively disqualifies
+-- HOT-indexed for any predicate-column change (even this out-of-predicate ->
+-- out-of-predicate case). The update is therefore non-HOT, and the partial
+-- index correctly stays empty for these non-'active' rows.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_partial (
+ id int PRIMARY KEY,
+ status text,
+ data text
+) WITH (fillfactor = 50);
+CREATE INDEX hi_partial_active_idx ON hi_partial(status) WHERE status = 'active';
+INSERT INTO hi_partial VALUES (1, 'active', 'data1');
+INSERT INTO hi_partial VALUES (2, 'inactive', 'data2');
+INSERT INTO hi_partial VALUES (3, 'deleted', 'data3');
+-- out -> out transition on the predicate column: HOT-indexed keeps it on-page,
+-- and the partial index gets no entry (the row satisfies the predicate neither
+-- before nor after the update).
+UPDATE hi_partial SET status = 'deleted' WHERE id = 2;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT * FROM get_hi_count('hi_partial');
+ updates | hot | hot_idx
+---------+-----+---------
+ 1 | 1 | 1
+(1 row)
+
+-- The partial index still correctly answers "active" queries.
+SELECT id, status FROM hi_partial WHERE status = 'active';
+ id | status
+----+--------
+ 1 | active
+(1 row)
+
+DROP TABLE hi_partial;
+-- ---------------------------------------------------------------------------
+-- 6. Partition: hot-indexed inside one partition
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_part (
+ id int,
+ partition_key int,
+ indexed_col int,
+ data text,
+ PRIMARY KEY (id, partition_key)
+) PARTITION BY RANGE (partition_key);
+CREATE TABLE hi_part_1 PARTITION OF hi_part
+ FOR VALUES FROM (1) TO (100) WITH (fillfactor = 50);
+CREATE INDEX hi_part_idx ON hi_part(indexed_col);
+INSERT INTO hi_part VALUES (1, 50, 100, 'data');
+UPDATE hi_part SET indexed_col = 150 WHERE id = 1;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT * FROM get_hi_count('hi_part_1');
+ updates | hot | hot_idx
+---------+-----+---------
+ 1 | 1 | 1
+(1 row)
+
+SET enable_seqscan = off;
+SELECT id FROM hi_part WHERE indexed_col = 150;
+ id
+----
+ 1
+(1 row)
+
+SELECT id FROM hi_part WHERE indexed_col = 100;
+ id
+----
+(0 rows)
+
+RESET enable_seqscan;
+DROP TABLE hi_part CASCADE;
+-- ---------------------------------------------------------------------------
+-- 7. Trigger modifies indexed column: hot-indexed, not non-HOT
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_trigger (
+ id int PRIMARY KEY,
+ triggered_col int,
+ data text
+) WITH (fillfactor = 50);
+CREATE INDEX hi_trigger_idx ON hi_trigger(triggered_col);
+CREATE OR REPLACE FUNCTION hi_trigger_bump()
+RETURNS TRIGGER AS $$
+BEGIN
+ NEW.triggered_col = NEW.triggered_col + 1;
+ RETURN NEW;
+END;
+$$ LANGUAGE plpgsql;
+CREATE TRIGGER before_update_bump
+ BEFORE UPDATE ON hi_trigger
+ FOR EACH ROW
+ EXECUTE FUNCTION hi_trigger_bump();
+INSERT INTO hi_trigger VALUES (1, 100, 'initial');
+-- UPDATE's SET clause doesn't touch the indexed column, but the
+-- trigger modifies it via heap_modify_tuple. hot-indexed must detect this
+-- and keep the tuple on-page (HEAP_INDEXED_UPDATED) plus a new btree entry.
+UPDATE hi_trigger SET data = 'updated' WHERE id = 1;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT * FROM get_hi_count('hi_trigger');
+ updates | hot | hot_idx
+---------+-----+---------
+ 1 | 1 | 1
+(1 row)
+
+SELECT triggered_col FROM hi_trigger WHERE id = 1;
+ triggered_col
+---------------
+ 101
+(1 row)
+
+-- New value reachable.
+SET enable_seqscan = off;
+SELECT id FROM hi_trigger WHERE triggered_col = 101;
+ id
+----
+ 1
+(1 row)
+
+SELECT id FROM hi_trigger WHERE triggered_col = 100;
+ id
+----
+(0 rows)
+
+RESET enable_seqscan;
+DROP TABLE hi_trigger CASCADE;
+DROP FUNCTION hi_trigger_bump();
+-- ---------------------------------------------------------------------------
+-- 8. JSONB expression index: HOT-indexed is not yet supported on expression
+-- indexes, so the update falls back to a non-HOT update (hot_idx = 0).
+-- Reads stay correct.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_jsonb (
+ id int PRIMARY KEY,
+ data jsonb
+) WITH (fillfactor = 50);
+CREATE INDEX hi_jsonb_name_idx ON hi_jsonb ((data->>'name'));
+INSERT INTO hi_jsonb VALUES (1, '{"name":"Alice","age":30}');
+-- Changing the indexed expression's value (name): expression indexes are not
+-- yet supported, so this is a non-HOT update.
+UPDATE hi_jsonb SET data = jsonb_set(data, '{name}', '"Alice2"') WHERE id = 1;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT * FROM get_hi_count('hi_jsonb');
+ updates | hot | hot_idx
+---------+-----+---------
+ 1 | 0 | 0
+(1 row)
+
+SET enable_seqscan = off;
+SELECT id FROM hi_jsonb WHERE data->>'name' = 'Alice2';
+ id
+----
+ 1
+(1 row)
+
+SELECT id FROM hi_jsonb WHERE data->>'name' = 'Alice';
+ id
+----
+(0 rows)
+
+RESET enable_seqscan;
+DROP TABLE hi_jsonb;
+-- ---------------------------------------------------------------------------
+-- 9. GIN index with changed extracted keys: hot-indexed
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_gin (
+ id int PRIMARY KEY,
+ tags text[]
+) WITH (fillfactor = 50);
+CREATE INDEX hi_gin_tags_idx ON hi_gin USING gin (tags);
+INSERT INTO hi_gin VALUES (1, ARRAY['tag1', 'tag2']);
+-- Adding a tag yields a different extracted-key set: hot-indexed.
+UPDATE hi_gin SET tags = ARRAY['tag1', 'tag2', 'tag5'] WHERE id = 1;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT * FROM get_hi_count('hi_gin');
+ updates | hot | hot_idx
+---------+-----+---------
+ 1 | 1 | 1
+(1 row)
+
+SET enable_seqscan = off;
+SELECT id FROM hi_gin WHERE tags @> ARRAY['tag5'];
+ id
+----
+ 1
+(1 row)
+
+RESET enable_seqscan;
+DROP TABLE hi_gin;
+-- ---------------------------------------------------------------------------
+-- 10. Per-index HOT-indexed counters: skipped vs matched
+--
+-- A table with two independent secondary indexes. An UPDATE touches a
+-- column covered by only one of them; the HOT-indexed path must insert
+-- into that one index and skip the other. pg_stat_all_indexes reports
+-- matched>0 on the updated index and skipped>0 on the untouched index.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hotidx_perindex (
+ id int PRIMARY KEY,
+ a int,
+ b int
+) WITH (fillfactor = 50);
+CREATE INDEX hotidx_perindex_a ON hotidx_perindex(a);
+CREATE INDEX hotidx_perindex_b ON hotidx_perindex(b);
+INSERT INTO hotidx_perindex VALUES (1, 100, 200);
+-- Modify only column a. HOT-indexed inserts into hotidx_perindex_a and
+-- skips hotidx_perindex_b (primary key indrelid is the table itself and
+-- also unchanged, so it counts as skipped too).
+UPDATE hotidx_perindex SET a = 101 WHERE id = 1;
+-- Force flush of pending stats to the shared entry.
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT indexrelname,
+ n_tup_hot_indexed_upd_matched AS matched,
+ n_tup_hot_indexed_upd_skipped AS skipped
+ FROM pg_stat_all_indexes
+ WHERE relname = 'hotidx_perindex'
+ ORDER BY indexrelname;
+ indexrelname | matched | skipped
+----------------------+---------+---------
+ hotidx_perindex_a | 1 | 0
+ hotidx_perindex_b | 0 | 1
+ hotidx_perindex_pkey | 0 | 1
+(3 rows)
+
+-- A second UPDATE touching only b inverts the assignment.
+UPDATE hotidx_perindex SET b = 201 WHERE id = 1;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT indexrelname,
+ n_tup_hot_indexed_upd_matched AS matched,
+ n_tup_hot_indexed_upd_skipped AS skipped
+ FROM pg_stat_all_indexes
+ WHERE relname = 'hotidx_perindex'
+ ORDER BY indexrelname;
+ indexrelname | matched | skipped
+----------------------+---------+---------
+ hotidx_perindex_a | 1 | 1
+ hotidx_perindex_b | 1 | 1
+ hotidx_perindex_pkey | 0 | 2
+(3 rows)
+
+-- Invariant: matched + skipped == owning table's n_tup_hot_indexed_upd.
+SELECT indexrelname,
+ n_tup_hot_indexed_upd_matched + n_tup_hot_indexed_upd_skipped AS total,
+ (SELECT n_tup_hot_indexed_upd FROM pg_stat_all_tables
+ WHERE relname = 'hotidx_perindex') AS table_hot_idx_upd
+ FROM pg_stat_all_indexes
+ WHERE relname = 'hotidx_perindex'
+ ORDER BY indexrelname;
+ indexrelname | total | table_hot_idx_upd
+----------------------+-------+-------------------
+ hotidx_perindex_a | 2 | 2
+ hotidx_perindex_b | 2 | 2
+ hotidx_perindex_pkey | 2 | 2
+(3 rows)
+
+-- Boolean assertion of the same invariant. This is the canonical form
+-- reviewers asked for: every index entry is either matched (the index
+-- got a fresh insert this UPDATE) or skipped (HOT-indexed correctly
+-- avoided an insert because the index's attrs did not change). If the
+-- two counters drift apart from the table-level n_tup_hot_indexed_upd we
+-- have either lost a per-index increment or double-counted one.
+SELECT bool_and((n_tup_hot_indexed_upd_matched + n_tup_hot_indexed_upd_skipped) =
+ (SELECT n_tup_hot_indexed_upd FROM pg_stat_all_tables
+ WHERE relname = 'hotidx_perindex'))
+ AS perindex_invariant_holds
+ FROM pg_stat_all_indexes
+ WHERE relname = 'hotidx_perindex';
+ perindex_invariant_holds
+--------------------------
+ t
+(1 row)
+
+DROP TABLE hotidx_perindex;
+-- ---------------------------------------------------------------------------
+-- 11. Long hot-loop UPDATE stays compact and HOT-indexed
+--
+-- A long run of HOT-indexed UPDATEs to a single row stays compact: prune
+-- collapses each dead version to a redirect to the live tuple and reuses its
+-- slot, so the heap stays bounded and the chain does not grow unbounded.
+-- Every UPDATE that changes the indexed column (and leaves another index,
+-- here the PK, unchanged) takes the HOT-indexed path.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_chaincap (
+ id int PRIMARY KEY,
+ a int
+) WITH (fillfactor = 10);
+CREATE INDEX hi_chaincap_a_idx ON hi_chaincap(a);
+INSERT INTO hi_chaincap VALUES (1, 0);
+DO $$
+DECLARE
+ i int;
+BEGIN
+ FOR i IN 1 .. 200 LOOP
+ UPDATE hi_chaincap SET a = i WHERE id = 1;
+ END LOOP;
+END $$;
+-- After 200 UPDATEs the row's value is 200.
+SELECT a FROM hi_chaincap WHERE id = 1;
+ a
+-----
+ 200
+(1 row)
+
+-- Every UPDATE took the HOT-indexed path (the PK index is unchanged, so it is
+-- skipped), so n_tup_hot_indexed_upd advanced.
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT hot_idx > 0 AS hot_indexed_fired
+ FROM get_hi_count('hi_chaincap');
+ hot_indexed_fired
+-------------------
+ t
+(1 row)
+
+-- The heap stayed compact: prune+collapse reclaimed the dead versions, so the
+-- single live row stays within a couple of pages. pg_relation_size reflects
+-- the table's actual current size regardless of vacuum/analyze stats, unlike
+-- pg_class.relpages (which is only updated by VACUUM/ANALYZE and would be
+-- trivially <= 1 on this never-vacuumed table even if pruning had failed and
+-- the heap had ballooned).
+SELECT pg_relation_size('hi_chaincap') <= 8192 * 2 AS heap_stayed_compact;
+ heap_stayed_compact
+---------------------
+ t
+(1 row)
+
+DROP TABLE hi_chaincap;
+-- ---------------------------------------------------------------------------
+-- 12. Reclamation of a collapsed HOT-indexed chain by prune
+--
+-- A dead HOT-indexed chain member is preserved at prune time (its stale leaf
+-- may still exist) and the chain collapses to an LP_REDIRECT forwarder; the
+-- index cleanup pass then sweeps the stale leaf, and a later VACUUM reclaims
+-- the now-unreferenced member and re-points the redirect once it falls below
+-- the global xmin horizon. We assert only that the chain collapses
+-- (n_chains = 0): that is the deterministic proof the mechanism worked. The
+-- final member's reclaim (n_hot_indexed reaching 0) additionally requires the
+-- deleted tuple to fall below the global xmin horizon, which a snapshot held
+-- elsewhere in the running regression cluster can delay indefinitely -- so we
+-- do not assert an exact n_hot_indexed here (autovacuum_enabled is off only to
+-- keep our own VACUUMs from being skipped on a stolen lock).
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_reclaim (
+ id int PRIMARY KEY,
+ a int
+) WITH (fillfactor = 50, autovacuum_enabled = false);
+CREATE INDEX hi_reclaim_a_idx ON hi_reclaim(a);
+INSERT INTO hi_reclaim VALUES (1, 100);
+-- Generate a collapsed chain via a HOT-indexed update.
+UPDATE hi_reclaim SET a = 200 WHERE id = 1;
+SELECT n_hot_indexed >= 1 AS hot_indexed_present_before_reclaim
+ FROM pg_relation_hot_indexed_stats('hi_reclaim');
+ hot_indexed_present_before_reclaim
+------------------------------------
+ t
+(1 row)
+
+-- Delete the live tuple. The first VACUUM collapses the dead chain and sweeps
+-- the stale leaf; a later VACUUM reclaims the now-unreferenced member once it
+-- falls below the global xmin horizon. Assert only the deterministic
+-- collapse (n_chains = 0); the member-reclaim count is horizon-dependent and
+-- not something a regress test can pin down.
+DELETE FROM hi_reclaim WHERE id = 1;
+VACUUM hi_reclaim;
+VACUUM hi_reclaim;
+SELECT n_chains = 0 AS chain_collapsed_after_reclaim
+ FROM pg_relation_hot_indexed_stats('hi_reclaim');
+ chain_collapsed_after_reclaim
+-------------------------------
+ t
+(1 row)
+
+DROP TABLE hi_reclaim;
+-- ---------------------------------------------------------------------------
+-- 13. Page with a preserved HOT-indexed member is never marked all-visible
+--
+-- pruneheap deliberately leaves PD_ALL_VISIBLE clear on any page that still
+-- carries a preserved HOT-indexed member: an index-only scan must heap-fetch
+-- through the chain so the read-side crossed-attribute bitmap can filter stale btree
+-- entries.
+--
+-- We force the freeze path with VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) and
+-- then read pd_flags via pageinspect.page_header. The page must still carry
+-- a HOT-indexed member (n_hot_indexed > 0) AND must not have PD_ALL_VISIBLE
+-- (0x0004).
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_vm (
+ id int PRIMARY KEY,
+ a int
+) WITH (fillfactor = 50, autovacuum_enabled = false);
+CREATE INDEX hi_vm_a_idx ON hi_vm(a);
+INSERT INTO hi_vm VALUES (1, 1);
+-- Two HOT-indexed updates leave a multi-hop chain, so a preserved HOT-indexed
+-- member remains on the page after prune, which is what this test needs.
+UPDATE hi_vm SET a = 2 WHERE id = 1;
+UPDATE hi_vm SET a = 3 WHERE id = 1;
+-- Force the all-visible bit decision: VACUUM with DISABLE_PAGE_SKIPPING
+-- considers every page; FREEZE pushes hint bits hard. After this, any
+-- page bearing a preserved HOT-indexed member must still report all_visible = 0.
+VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) hi_vm;
+SELECT n_hot_indexed >= 1 AS hot_indexed_present
+ FROM pg_relation_hot_indexed_stats('hi_vm');
+ hot_indexed_present
+---------------------
+ t
+(1 row)
+
+-- PD_ALL_VISIBLE = 0x0004. Must be 0 on a page with a preserved member.
+SELECT (flags & 4) = 0 AS not_marked_all_visible
+ FROM page_header(get_raw_page('hi_vm', 0));
+ not_marked_all_visible
+------------------------
+ t
+(1 row)
+
+DROP TABLE hi_vm;
+-- ---------------------------------------------------------------------------
+-- 14. Cycle-key dedup: column rename a -> b -> a stays correct
+--
+-- A rename does not rewrite heap or index entries; it only updates the
+-- catalog. The relcache invalidation must trigger a fresh attribute
+-- bitmap and the HOT-indexed predicate must compare attribute *numbers*,
+-- not attribute *names*. After two renames that net to identity, every
+-- subsequent UPDATE must continue to drive the HOT-indexed path.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_cycle (
+ id int PRIMARY KEY,
+ a int
+) WITH (fillfactor = 50);
+CREATE INDEX hi_cycle_a_idx ON hi_cycle(a);
+INSERT INTO hi_cycle VALUES (1, 100);
+-- Cycle the column name and confirm both intermediate forms drive HOT-indexed.
+ALTER TABLE hi_cycle RENAME COLUMN a TO b;
+UPDATE hi_cycle SET b = 200 WHERE id = 1;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT hot_idx > 0 AS hot_indexed_after_first_rename
+ FROM get_hi_count('hi_cycle');
+ hot_indexed_after_first_rename
+--------------------------------
+ t
+(1 row)
+
+ALTER TABLE hi_cycle RENAME COLUMN b TO a;
+UPDATE hi_cycle SET a = 300 WHERE id = 1;
+-- Lookup via the index returns the current value, not any of the
+-- pre-rename values.
+SET enable_seqscan = off;
+SELECT id, a FROM hi_cycle WHERE a = 300;
+ id | a
+----+-----
+ 1 | 300
+(1 row)
+
+SELECT id FROM hi_cycle WHERE a = 100;
+ id
+----
+(0 rows)
+
+SELECT id FROM hi_cycle WHERE a = 200;
+ id
+----
+(0 rows)
+
+RESET enable_seqscan;
+DROP TABLE hi_cycle;
+-- ---------------------------------------------------------------------------
+-- 15. Summarizing-only column UPDATE produces CLASSIC, not INDEXED
+--
+-- HeapUpdateHotAllowable returns HEAP_HEAP_ONLY_UPDATE when every
+-- modified indexed attribute is covered only by summarizing indexes.
+-- A BRIN-only column is the canonical case: the BRIN index gets a
+-- new summary entry via aminsert, but no per-update btree entry is
+-- needed and HOT-indexed does not fire. The signal is
+-- n_tup_hot_upd > 0 with n_tup_hot_indexed_upd unchanged.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_brin (
+ id int PRIMARY KEY,
+ bcol int
+) WITH (fillfactor = 50);
+CREATE INDEX hi_brin_idx ON hi_brin USING brin(bcol);
+INSERT INTO hi_brin VALUES (1, 100);
+-- Capture the HOT-indexed counter before, drive a BRIN-only update,
+-- and assert that classic HOT advanced while HOT-indexed did not.
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT hot_idx AS hot_idx_before FROM get_hi_count('hi_brin') \gset
+UPDATE hi_brin SET bcol = 200 WHERE id = 1;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT (hot - 0) > 0 AS classic_hot_fired,
+ hot_idx = :hot_idx_before AS hot_indexed_did_not_fire
+ FROM get_hi_count('hi_brin');
+ classic_hot_fired | hot_indexed_did_not_fire
+-------------------+--------------------------
+ t | t
+(1 row)
+
+-- The BRIN index sees the new value via aminsert.
+SELECT bcol FROM hi_brin WHERE id = 1;
+ bcol
+------
+ 200
+(1 row)
+
+DROP TABLE hi_brin;
+-- ---------------------------------------------------------------------------
+-- 16. UNIQUE index on a type where image equality != operator equality
+--
+-- numeric 1.0 and 1.00 are equal under the btree opclass but have
+-- different on-disk images. A HOT-indexed update 1.0 -> 1.00 inserts a
+-- fresh leaf carrying the live image and leaves a stale leaf for 1.0
+-- (the hop's modified-attrs bitmap marks k changed, since modified-column
+-- detection is image-based). A later INSERT of a value equal under the
+-- opclass must still be detected as a duplicate: the unique check reaches
+-- the live tuple through the fresh leaf, which points directly at it (no hop
+-- after it, so the overlap is empty and the leaf is a genuine conflict); the
+-- stale 1.0 leaf is skipped because the k-changing hop overlaps the unique
+-- index's attribute.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_unum (k numeric UNIQUE, j int) WITH (fillfactor = 50);
+CREATE INDEX hi_unum_j ON hi_unum(j); -- 2nd indexed attr, kept fixed
+INSERT INTO hi_unum VALUES (1.0, 100);
+UPDATE hi_unum SET k = 1.00 WHERE j = 100; -- HOT-indexed: 1.0 -> 1.00
+SELECT n_hot_indexed > 0 AS made_hot_indexed
+ FROM pg_relation_hot_indexed_stats('hi_unum');
+ made_hot_indexed
+------------------
+ t
+(1 row)
+
+-- A numerically-equal insert must conflict (the fresh leaf catches it):
+INSERT INTO hi_unum VALUES (1.0, 1); -- expect duplicate key error
+ERROR: duplicate key value violates unique constraint "hi_unum_k_key"
+DETAIL: Key (k)=(1.0) already exists.
+-- A genuinely different value is accepted:
+INSERT INTO hi_unum VALUES (2.0, 2);
+SELECT k, j FROM hi_unum ORDER BY j;
+ k | j
+------+-----
+ 2.0 | 2
+ 1.00 | 100
+(2 rows)
+
+DROP TABLE hi_unum;
+-- ---------------------------------------------------------------------------
+-- 17. CREATE INDEX and REINDEX over live HOT-indexed chains
+--
+-- A freshly built or rebuilt index must reflect current values, never a
+-- stale chain member: the build scans live tuples only and points each
+-- HOT-indexed live tuple's entry at its own TID, so the new entries have no
+-- hop after them and the crossed-attribute bitmap keeps them.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_reindex (id int PRIMARY KEY, a int, b int) WITH (fillfactor = 50);
+CREATE INDEX hi_reindex_a ON hi_reindex(a);
+INSERT INTO hi_reindex SELECT g, g, g FROM generate_series(1, 6) g;
+UPDATE hi_reindex SET a = a + 100; -- HOT-indexed on a
+UPDATE hi_reindex SET a = a + 100; -- again -> longer chains
+SELECT n_hot_indexed > 0 AS made_hot_indexed
+ FROM pg_relation_hot_indexed_stats('hi_reindex');
+ made_hot_indexed
+------------------
+ t
+(1 row)
+
+-- Build a NEW index and REINDEX the existing one over the live chains.
+CREATE INDEX hi_reindex_b ON hi_reindex(b);
+REINDEX INDEX hi_reindex_a;
+SET enable_seqscan = off;
+SELECT id, a FROM hi_reindex WHERE a = 204; -- current value -> id 4
+ id | a
+----+-----
+ 4 | 204
+(1 row)
+
+SELECT count(*) FROM hi_reindex WHERE a = 4; -- obsolete value -> 0
+ count
+-------
+ 0
+(1 row)
+
+SELECT id FROM hi_reindex WHERE b = 2; -- via freshly built index -> 2
+ id
+----
+ 2
+(1 row)
+
+RESET enable_seqscan;
+DROP TABLE hi_reindex;
+-- ---------------------------------------------------------------------------
+-- 18. DROP every index over live HOT-indexed chains, then VACUUM
+--
+-- After all indexes are dropped, heap pages may still carry preserved
+-- HOT-indexed members left by earlier updates. VACUUM of such a no-index
+-- relation must complete without error, and reads must stay correct via the
+-- redirect forwarders.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_dropidx (id int PRIMARY KEY, a int) WITH (fillfactor = 50, autovacuum_enabled = false);
+CREATE INDEX hi_dropidx_a ON hi_dropidx(a);
+INSERT INTO hi_dropidx SELECT g, g FROM generate_series(1, 6) g;
+UPDATE hi_dropidx SET a = a + 100; -- HOT-indexed on a
+UPDATE hi_dropidx SET a = a + 100; -- again -> longer chains
+SELECT n_hot_indexed > 0 AS made_hot_indexed
+ FROM pg_relation_hot_indexed_stats('hi_dropidx');
+ made_hot_indexed
+------------------
+ t
+(1 row)
+
+-- Drop every index, leaving preserved HOT-indexed members with no index to sweep.
+DROP INDEX hi_dropidx_a;
+ALTER TABLE hi_dropidx DROP CONSTRAINT hi_dropidx_pkey;
+-- Must not crash on the no-index path; two passes exercise the second-pass
+-- reclaim guard as well.
+VACUUM hi_dropidx;
+VACUUM hi_dropidx;
+-- Reads remain correct after the indexes are gone.
+SELECT id, a FROM hi_dropidx ORDER BY id;
+ id | a
+----+-----
+ 1 | 201
+ 2 | 202
+ 3 | 203
+ 4 | 204
+ 5 | 205
+ 6 | 206
+(6 rows)
+
+DROP TABLE hi_dropidx;
+-- ---------------------------------------------------------------------------
+-- 19. Re-collapse of a data-redirect chain across partial VACUUMs
+--
+-- A chain that collapses to a HOT-indexed data redirect, is vacuumed with
+-- INDEX_CLEANUP off (so the stale leaves and the redirect survive), then
+-- receives further HOT-indexed updates that re-collapse the chain and
+-- re-point the redirect at a new live tuple, must not leave the redirect
+-- dangling. A subsequent full VACUUM must complete without error, leave the
+-- heap consistent (verify_heapam reports nothing), and reads must stay
+-- correct. (Regression: an earlier revision crashed reclaiming a mid-chain
+-- member while a data redirect still pointed past it.)
+-- ---------------------------------------------------------------------------
+CREATE EXTENSION IF NOT EXISTS amcheck;
+CREATE TABLE hi_recollapse (id int PRIMARY KEY, a int) WITH (fillfactor = 50, autovacuum_enabled = false);
+CREATE INDEX hi_recollapse_a ON hi_recollapse(a);
+INSERT INTO hi_recollapse VALUES (1, 1);
+-- First chain: two HOT-indexed updates, then prune to a data redirect while
+-- leaving the stale btree leaves in place (INDEX_CLEANUP off).
+UPDATE hi_recollapse SET a = 2 WHERE id = 1;
+UPDATE hi_recollapse SET a = 3 WHERE id = 1;
+VACUUM (INDEX_CLEANUP off) hi_recollapse;
+-- Re-collapse: more HOT-indexed updates extend the chain past the redirect
+-- target; the next prune re-points the data redirect at the new first live
+-- tuple and extends its union.
+UPDATE hi_recollapse SET a = 4 WHERE id = 1;
+UPDATE hi_recollapse SET a = 5 WHERE id = 1;
+VACUUM (INDEX_CLEANUP off) hi_recollapse;
+-- Full vacuum now reclaims the dead chain; the re-pointed redirect must not
+-- dangle. Two passes also exercise the redirect re-point second pass.
+VACUUM hi_recollapse;
+VACUUM hi_recollapse;
+-- Heap must be structurally consistent (no rows == no corruption).
+SELECT * FROM verify_heapam('hi_recollapse');
+ blkno | offnum | attnum | msg
+-------+--------+--------+-----
+(0 rows)
+
+SET enable_seqscan = off;
+SELECT id, a FROM hi_recollapse WHERE a = 5; -- current value -> id 1
+ id | a
+----+---
+ 1 | 5
+(1 row)
+
+SELECT count(*) FROM hi_recollapse WHERE a = 3; -- obsolete value -> 0
+ count
+-------
+ 0
+(1 row)
+
+RESET enable_seqscan;
+SELECT id, a FROM hi_recollapse ORDER BY id;
+ id | a
+----+---
+ 1 | 5
+(1 row)
+
+DROP TABLE hi_recollapse;
+-- ---------------------------------------------------------------------------
+-- 20. Index deletion over an entry that points at a data-redirect root
+--
+-- A data redirect is an LP_REDIRECT that carries a bitmap, so it reports
+-- lp_len > 0 (ItemIdHasStorage true) even though it is not a normal tuple.
+-- index_delete_check_htid must treat it as a redirect, not read its blob as a
+-- HeapTupleHeader. Reproduce: collapse a chain root to a data redirect while
+-- keeping the stale leaf that points at it (INDEX_CLEANUP off), then insert
+-- many duplicates of the stale key so btree bottom-up deletion runs
+-- heap_index_delete_tuples over that stale entry.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_iddel (id int, a int) WITH (fillfactor = 50, autovacuum_enabled = false);
+CREATE INDEX hi_iddel_a ON hi_iddel(a);
+INSERT INTO hi_iddel VALUES (1, 1);
+UPDATE hi_iddel SET a = a + 1 WHERE id = 1; -- HOT-indexed
+UPDATE hi_iddel SET a = a + 1 WHERE id = 1; -- multi-hop chain
+VACUUM (INDEX_CLEANUP off) hi_iddel; -- root -> data redirect, keep stale a=1 leaf
+-- Many duplicates of the stale key fill the leaf and trigger bottom-up
+-- deletion, which feeds the stale a=1 entry (htid -> the data-redirect root)
+-- to heap_index_delete_tuples. Must not crash or misread the blob.
+INSERT INTO hi_iddel SELECT g, 1 FROM generate_series(2, 3000) g;
+VACUUM hi_iddel;
+SELECT * FROM verify_heapam('hi_iddel');
+ blkno | offnum | attnum | msg
+-------+--------+--------+-----
+(0 rows)
+
+SET enable_seqscan = off;
+SELECT id, a FROM hi_iddel WHERE id = 1; -- current value -> a = 3
+ id | a
+----+---
+ 1 | 3
+(1 row)
+
+RESET enable_seqscan;
+DROP TABLE hi_iddel;
+-- ---------------------------------------------------------------------------
+-- 21. A change to a column covered by a non-btree index AM is HOT-indexed
+--
+-- A HOT-indexed update leaves a stale pre-update leaf that the read side
+-- filters via the crossed-attribute bitmap, which is access-method agnostic.
+-- A column covered by a non-btree index (here a GiST index on a point column)
+-- is therefore HOT-indexed like any other, and the GiST index still returns
+-- correct results across the chain. A change to a btree-only column on the
+-- same table is likewise HOT-indexed.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_nonbtree (id int PRIMARY KEY, tag int, p point)
+ WITH (fillfactor = 10);
+CREATE INDEX hi_nonbtree_tag ON hi_nonbtree (tag); -- btree index
+CREATE INDEX hi_nonbtree_p ON hi_nonbtree USING gist (p); -- GiST, non-btree
+INSERT INTO hi_nonbtree SELECT g, g, point(g, g)
+ FROM generate_series(1, 200) g;
+-- Change the GiST-covered column first: HOT-indexed (hot_idx = 200).
+UPDATE hi_nonbtree SET p = point(p[0] + 1000, p[1] + 1000);
+SELECT hot_idx AS gist_col_hot_indexed FROM get_hi_count('hi_nonbtree');
+ gist_col_hot_indexed
+----------------------
+ 200
+(1 row)
+
+-- The GiST index must return correct results: the old positions are gone and
+-- every row is found at its new position (no stale leaf surfaces an old key).
+SET enable_seqscan = off;
+SELECT count(*) AS at_old_positions
+ FROM hi_nonbtree WHERE p <@ box(point(0, 0), point(300, 300));
+ at_old_positions
+------------------
+ 0
+(1 row)
+
+SELECT count(*) AS at_new_positions
+ FROM hi_nonbtree WHERE p <@ box(point(1000, 1000), point(1300, 1300));
+ at_new_positions
+------------------
+ 200
+(1 row)
+
+RESET enable_seqscan;
+-- Changing the btree-only column (p unchanged) stays HOT-indexed.
+UPDATE hi_nonbtree SET tag = tag + 1000;
+SELECT hot_idx > 0 AS btree_col_is_hot_indexed FROM get_hi_count('hi_nonbtree');
+ btree_col_is_hot_indexed
+--------------------------
+ t
+(1 row)
+
+DROP TABLE hi_nonbtree;
+-- ---------------------------------------------------------------------------
+-- 22. ABA on a unique key across two distinct live rows: a key cycled away
+-- and back must still collide with another row that holds it. The stale
+-- leaves left by the cycle must not let a genuine duplicate slip past the
+-- uniqueness check -- the read-side recheck compares the live key, not just
+-- a changed-attribute bitmap.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_aba (k int, v int) WITH (fillfactor = 50);
+CREATE UNIQUE INDEX hi_aba_k ON hi_aba (k);
+CREATE INDEX hi_aba_v ON hi_aba (v);
+INSERT INTO hi_aba VALUES (1, 10), (2, 20);
+-- Cycle row1's unique key 1 -> 3 -> 1 (v unchanged, so each step is
+-- HOT-indexed and leaves stale entries in hi_aba_k).
+UPDATE hi_aba SET k = 3 WHERE v = 10;
+UPDATE hi_aba SET k = 1 WHERE v = 10;
+SELECT hot_idx > 0 AS cycled_hot_indexed FROM get_hi_count('hi_aba');
+ cycled_hot_indexed
+--------------------
+ t
+(1 row)
+
+-- row1 is live at k = 1 again. Moving row2 onto k = 1 must raise a unique
+-- violation despite the stale '1' leaves from the cycle.
+UPDATE hi_aba SET k = 1 WHERE v = 20;
+ERROR: duplicate key value violates unique constraint "hi_aba_k"
+DETAIL: Key (k)=(1) already exists.
+DROP TABLE hi_aba;
+-- ---------------------------------------------------------------------------
+-- 23. Partial index whose predicate references a non-key column. Flipping the
+-- row out of the predicate while leaving the indexed key unchanged is
+-- HOT-indexed: the predicate column is part of the index's attribute set, so
+-- the crossed-attribute bitmap drops the now-stale partial-index entry on read
+-- (no value recheck is involved).
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_partpred (id int PRIMARY KEY, k int, active boolean)
+ WITH (fillfactor = 50);
+CREATE INDEX hi_partpred_k ON hi_partpred (k) WHERE active;
+INSERT INTO hi_partpred VALUES (1, 100, true);
+-- Flip the predicate column 'active' true -> false; the index key k is
+-- unchanged. The row no longer satisfies the predicate, so its partial-index
+-- entry must be removed, not left pointing into the chain.
+UPDATE hi_partpred SET active = false WHERE id = 1;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT hot, hot_idx FROM get_hi_count('hi_partpred');
+ hot | hot_idx
+-----+---------
+ 1 | 1
+(1 row)
+
+-- The partial index must not surface the row now that active = false.
+-- A query whose qual exactly matches the partial predicate uses the index
+-- without re-filtering 'active' on the heap, so a stale entry would surface.
+SET enable_seqscan = off;
+SET enable_bitmapscan = off;
+EXPLAIN (COSTS OFF) SELECT id FROM hi_partpred WHERE active;
+ QUERY PLAN
+-----------------------------------------------
+ Index Scan using hi_partpred_k on hi_partpred
+(1 row)
+
+SELECT id FROM hi_partpred WHERE k = 100 AND active;
+ id
+----
+(0 rows)
+
+SELECT id FROM hi_partpred WHERE active;
+ id
+----
+(0 rows)
+
+RESET enable_bitmapscan;
+RESET enable_seqscan;
+DROP TABLE hi_partpred;
+-- ---------------------------------------------------------------------------
+-- 24. Reclaim + stub mix. Repeated updates of column a followed by an update
+-- of column b build a chain whose prune reclaims the members whose change was
+-- superseded (a changed again) and keeps stubs for those that were not, so a
+-- root redirect ends up pointing at a stub and a later walk crosses mid-chain
+-- stubs. Reads through each index and amcheck must stay correct across the
+-- collapse, and a second round must walk the existing stubs without severing
+-- the chain.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_stubmix (id int PRIMARY KEY, a int, b int) WITH (fillfactor = 50, autovacuum_enabled = false);
+CREATE INDEX hi_stubmix_a ON hi_stubmix (a);
+CREATE INDEX hi_stubmix_b ON hi_stubmix (b);
+INSERT INTO hi_stubmix VALUES (1, 10, 100);
+UPDATE hi_stubmix SET a = 11 WHERE id = 1; -- changes a
+UPDATE hi_stubmix SET a = 12 WHERE id = 1; -- changes a again (supersedes)
+UPDATE hi_stubmix SET b = 101 WHERE id = 1; -- changes b
+VACUUM hi_stubmix;
+SET enable_seqscan = off;
+SET enable_bitmapscan = off;
+SELECT id, a, b FROM hi_stubmix WHERE a = 12; -- current a
+ id | a | b
+----+----+-----
+ 1 | 12 | 101
+(1 row)
+
+SELECT id, a, b FROM hi_stubmix WHERE b = 101; -- current b
+ id | a | b
+----+----+-----
+ 1 | 12 | 101
+(1 row)
+
+SELECT id FROM hi_stubmix WHERE a = 10; -- stale a: 0 rows
+ id
+----
+(0 rows)
+
+RESET enable_bitmapscan;
+RESET enable_seqscan;
+SELECT * FROM verify_heapam('hi_stubmix'); -- no corruption across stubs
+ blkno | offnum | attnum | msg
+-------+--------+--------+-----
+(0 rows)
+
+-- A second round must walk the existing stubs (no priorXmax sever).
+UPDATE hi_stubmix SET a = 13 WHERE id = 1;
+VACUUM hi_stubmix;
+SELECT id, a, b FROM hi_stubmix WHERE a = 13;
+ id | a | b
+----+----+-----
+ 1 | 13 | 101
+(1 row)
+
+SELECT * FROM verify_heapam('hi_stubmix');
+ blkno | offnum | attnum | msg
+-------+--------+--------+-----
+(0 rows)
+
+DROP TABLE hi_stubmix;
+-- ---------------------------------------------------------------------------
+-- 25. Exclusion-constraint tables are HOT-indexed-eligible.
+--
+-- An exclusion constraint is enforced by check_exclusion_or_unique_constraint,
+-- which rechecks each candidate against the live tuple's current index-form
+-- with the constraint's own operators, so a stale entry left by a HOT-indexed
+-- update is skipped while the live key always has its own entry. Updating a
+-- non-constrained indexed column is HOT-indexed (the GiST exclusion index is
+-- skipped), and the constraint stays correct.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_excl (
+ id int PRIMARY KEY,
+ tag int,
+ during int4range,
+ EXCLUDE USING gist (during WITH &&)
+) WITH (fillfactor = 10);
+CREATE INDEX hi_excl_tag ON hi_excl(tag);
+INSERT INTO hi_excl VALUES (1, 100, int4range(1, 10)), (2, 200, int4range(20, 30));
+-- Update a non-constrained indexed column: HOT-indexed (GiST exclusion index
+-- and PK skipped), and the exclusion constraint is still enforced.
+UPDATE hi_excl SET tag = tag + 1 WHERE id = 1;
+SELECT hot_idx > 0 AS tag_update_hot_indexed FROM get_hi_count('hi_excl');
+ tag_update_hot_indexed
+------------------------
+ t
+(1 row)
+
+INSERT INTO hi_excl VALUES (3, 300, int4range(5, 15)); -- overlaps id=1's (1,10)
+ERROR: conflicting key value violates exclusion constraint "hi_excl_during_excl"
+DETAIL: Key (during)=([5,15)) conflicts with existing key (during)=([1,10)).
+-- Move id=1's range away (this updates the GiST index, leaving a stale entry
+-- for the old (1,10) range). A range overlapping only the OLD range now
+-- inserts cleanly (the stale entry is skipped); one overlapping the NEW range
+-- still conflicts.
+UPDATE hi_excl SET during = int4range(100, 110) WHERE id = 1;
+INSERT INTO hi_excl VALUES (4, 400, int4range(5, 15)); -- only overlapped old range: OK
+INSERT INTO hi_excl VALUES (5, 500, int4range(105, 115));-- overlaps new (100,110): conflict
+ERROR: conflicting key value violates exclusion constraint "hi_excl_during_excl"
+DETAIL: Key (during)=([105,115)) conflicts with existing key (during)=([100,110)).
+DROP TABLE hi_excl;
+-- ---------------------------------------------------------------------------
+-- 26. TOAST interaction. An indexed column stored out-of-line must behave
+-- correctly across HOT-indexed updates: an entry kept across an update of a
+-- different column still resolves to the (unchanged) toasted value, and after
+-- the toasted column itself is changed the stale entry is dropped by the
+-- crossed-attribute bitmap (no value comparison or detoasting is needed).
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_toast (id int PRIMARY KEY, big text, tag int) WITH (fillfactor = 50);
+ALTER TABLE hi_toast ALTER COLUMN big SET STORAGE EXTERNAL; -- no compression
+CREATE INDEX hi_toast_big ON hi_toast (big);
+CREATE INDEX hi_toast_tag ON hi_toast (tag);
+INSERT INTO hi_toast VALUES (1, repeat('A', 2000), 10);
+-- The big value is stored out-of-line.
+SELECT pg_column_size(big) > 1500 AS big_is_external FROM hi_toast WHERE id = 1;
+ big_is_external
+-----------------
+ t
+(1 row)
+
+-- HOT-indexed update of tag leaves big (and its index entry) unchanged.
+UPDATE hi_toast SET tag = 11 WHERE id = 1;
+SELECT hot_idx > 0 AS tag_update_hot_indexed FROM get_hi_count('hi_toast');
+ tag_update_hot_indexed
+------------------------
+ t
+(1 row)
+
+SET enable_seqscan = off;
+SET enable_bitmapscan = off;
+SELECT id, tag, length(big) FROM hi_toast WHERE big = repeat('A', 2000);
+ id | tag | length
+----+-----+--------
+ 1 | 11 | 2000
+(1 row)
+
+-- HOT-indexed update of the toasted indexed column itself: the old entry is
+-- now stale because the crossed-attribute bitmap shows big changed.
+UPDATE hi_toast SET big = repeat('B', 2000) WHERE id = 1;
+SELECT id FROM hi_toast WHERE big = repeat('A', 2000); -- stale: 0 rows
+ id
+----
+(0 rows)
+
+SELECT id, length(big) FROM hi_toast WHERE big = repeat('B', 2000); -- current
+ id | length
+----+--------
+ 1 | 2000
+(1 row)
+
+RESET enable_bitmapscan;
+RESET enable_seqscan;
+SELECT * FROM verify_heapam('hi_toast');
+ blkno | offnum | attnum | msg
+-------+--------+--------+-----
+(0 rows)
+
+DROP TABLE hi_toast;
+-- ---------------------------------------------------------------------------
+-- 27. ABA on an indexed column. A HOT-indexed update that sets an indexed
+-- column to a value an earlier chain member already held leaves two leaves
+-- with that same key, both chain-resolving to the live tuple. A value-based
+-- recheck cannot tell them apart and would return the row twice; the
+-- crossed-attribute bitmap drops the stale ancestor leaf (its walk crosses the
+-- key-changing hops) and keeps only the fresh entry, so a forced index scan
+-- returns the row exactly once. REINDEX must not change that.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_aba (id int PRIMARY KEY, k int, v int) WITH (fillfactor = 50);
+CREATE INDEX hi_aba_k ON hi_aba (k);
+CREATE INDEX hi_aba_v ON hi_aba (v);
+INSERT INTO hi_aba VALUES (1, 1, 100);
+UPDATE hi_aba SET k = 3 WHERE id = 1; -- HOT-indexed: k changed, v kept
+UPDATE hi_aba SET k = 1 WHERE id = 1; -- HOT-indexed: k cycled back (ABA)
+SET enable_seqscan = off;
+SET enable_bitmapscan = off;
+SELECT count(*) AS k1_once FROM hi_aba WHERE k = 1; -- exactly 1
+ k1_once
+---------
+ 1
+(1 row)
+
+SELECT count(*) AS k3_gone FROM hi_aba WHERE k = 3; -- 0 (stale dropped)
+ k3_gone
+---------
+ 0
+(1 row)
+
+REINDEX INDEX hi_aba_k;
+SELECT count(*) AS k1_after_reindex FROM hi_aba WHERE k = 1; -- still 1
+ k1_after_reindex
+------------------
+ 1
+(1 row)
+
+RESET enable_bitmapscan;
+RESET enable_seqscan;
+SELECT * FROM verify_heapam('hi_aba');
+ blkno | offnum | attnum | msg
+-------+--------+--------+-----
+(0 rows)
+
+DROP TABLE hi_aba;
+-- ---------------------------------------------------------------------------
+-- 28. Partial index, predicate column changed but the row STAYS in the index
+-- (predicate still true, key unchanged). The update is HOT-indexed; selective
+-- maintenance re-inserts a fresh entry (the predicate column changed and still
+-- holds), so the row is still returned -- the bitmap drops the older entry and
+-- the fresh one re-supplies it. Guards against a "lost row" from over-eager
+-- dropping.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_partstay (id int PRIMARY KEY, k int, n int) WITH (fillfactor = 50);
+CREATE INDEX hi_partstay_k ON hi_partstay (k) WHERE n > 0;
+CREATE INDEX hi_partstay_id2 ON hi_partstay (id);
+INSERT INTO hi_partstay VALUES (1, 5, 3);
+UPDATE hi_partstay SET n = 7 WHERE id = 1; -- n 3->7, still > 0, k unchanged
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT hot_idx > 0 AS stay_is_hot_indexed FROM get_hi_count('hi_partstay');
+ stay_is_hot_indexed
+---------------------
+ t
+(1 row)
+
+SET enable_seqscan = off;
+SET enable_bitmapscan = off;
+SELECT count(*) AS stay_rows FROM hi_partstay WHERE k = 5 AND n > 0; -- want 1
+ stay_rows
+-----------
+ 1
+(1 row)
+
+RESET enable_bitmapscan;
+RESET enable_seqscan;
+DROP TABLE hi_partstay;
+-- ---------------------------------------------------------------------------
+-- 29. Partitioned table. A within-partition UPDATE of one indexed column is
+-- HOT-indexed on the leaf partition's heap exactly as for a non-partitioned
+-- table; a cross-partition update is a delete+insert and never HOT.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_part (id int, a int, b int) PARTITION BY RANGE (id);
+CREATE TABLE hi_part1 PARTITION OF hi_part FOR VALUES FROM (0) TO (100)
+ WITH (fillfactor = 50);
+CREATE INDEX hi_part_a ON hi_part (a);
+CREATE INDEX hi_part_b ON hi_part (b);
+INSERT INTO hi_part VALUES (1, 10, 20);
+UPDATE hi_part SET a = 11 WHERE id = 1; -- one indexed col, within partition
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT hot_idx > 0 AS part_is_hot_indexed FROM get_hi_count('hi_part1');
+ part_is_hot_indexed
+---------------------
+ t
+(1 row)
+
+SET enable_seqscan = off;
+SET enable_bitmapscan = off;
+SELECT count(*) AS a11 FROM hi_part WHERE a = 11; -- want 1
+ a11
+-----
+ 1
+(1 row)
+
+SELECT count(*) AS a10 FROM hi_part WHERE a = 10; -- want 0 (stale dropped)
+ a10
+-----
+ 0
+(1 row)
+
+RESET enable_bitmapscan;
+RESET enable_seqscan;
+SELECT * FROM verify_heapam('hi_part1');
+ blkno | offnum | attnum | msg
+-------+--------+--------+-----
+(0 rows)
+
+DROP TABLE hi_part;
+-- ---------------------------------------------------------------------------
+-- 30. Non-btree access method (hash). Read-side staleness is access-method
+-- agnostic (the crossed-attribute bitmap), so any index AM's column is
+-- HOT-indexed. Hash is the sharpest case: its scans recheck the heap value,
+-- which alone cannot disambiguate a value cycled away and back (ABA) -- the
+-- bitmap drops the stale ancestor so the row is returned exactly once.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_hash (id int PRIMARY KEY, v int, w int) WITH (fillfactor = 50);
+CREATE INDEX hi_hash_v ON hi_hash USING hash (v);
+CREATE INDEX hi_hash_w ON hi_hash (w);
+INSERT INTO hi_hash VALUES (1, 10, 100);
+UPDATE hi_hash SET v = 99 WHERE id = 1;
+UPDATE hi_hash SET v = 10 WHERE id = 1; -- ABA: 10 -> 99 -> 10, w unchanged
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT hot_idx > 0 AS hash_is_hot_indexed FROM get_hi_count('hi_hash');
+ hash_is_hot_indexed
+---------------------
+ t
+(1 row)
+
+SET enable_seqscan = off;
+SET enable_bitmapscan = off;
+SELECT count(*) AS hash_v10 FROM hi_hash WHERE v = 10; -- want 1 (no duplicate)
+ hash_v10
+----------
+ 1
+(1 row)
+
+SELECT count(*) AS hash_v99 FROM hi_hash WHERE v = 99; -- want 0 (stale dropped)
+ hash_v99
+----------
+ 0
+(1 row)
+
+RESET enable_bitmapscan;
+RESET enable_seqscan;
+DROP TABLE hi_hash;
+-- ---------------------------------------------------------------------------
+-- 31. DDL after a HOT-indexed chain exists. The per-hop modified-attrs
+-- bitmap on the page is keyed by physical attribute number and sized by the
+-- relation's natts AT WRITE TIME. Indexes added/dropped after the chain
+-- forms, and ADD/DROP COLUMN, must not corrupt the read-side staleness test.
+-- The sharp case is ADD COLUMN crossing an 8-attribute boundary, which grows
+-- ceil(natts/8): readers must locate each hop's bitmap from that hop's own
+-- write-time natts (HeapTupleHeaderGetNatts / the stub's stashed natts), not
+-- the relation's current natts.
+-- ---------------------------------------------------------------------------
+-- Exactly 8 attributes (c1..c7 + payload) so adding the 9th flips the bitmap
+-- from 1 byte to 2. c7 is the column we churn; c2 is an unchanged indexed
+-- column whose leaf must stay current.
+CREATE TABLE hi_ddl (
+ c1 int PRIMARY KEY, c2 int, c3 int, c4 int,
+ c5 int, c6 int, c7 int, payload text
+) WITH (fillfactor = 50, autovacuum_enabled = false);
+CREATE INDEX hi_ddl_c2 ON hi_ddl(c2);
+CREATE INDEX hi_ddl_c7 ON hi_ddl(c7);
+INSERT INTO hi_ddl VALUES (1, 10, 20, 30, 40, 50, 70, 'p');
+-- Form a HOT-indexed chain on c7 BEFORE any further DDL.
+UPDATE hi_ddl SET c7 = 71 WHERE c1 = 1;
+UPDATE hi_ddl SET c7 = 72 WHERE c1 = 1;
+-- (a) CREATE INDEX after the chain exists: the new index is built against the
+-- live tuple under its own TID, so its entry is never stale.
+CREATE INDEX hi_ddl_c3 ON hi_ddl(c3);
+-- (b) ADD COLUMN crossing the 8-attribute boundary (natts 8 -> 9). Existing
+-- hops keep their 1-byte bitmaps; the relation now wants 2. Reads through the
+-- old chain must still be correct.
+ALTER TABLE hi_ddl ADD COLUMN c9 int;
+CREATE INDEX hi_ddl_c9 ON hi_ddl(c9);
+SET enable_seqscan = off;
+SET enable_bitmapscan = off;
+SET enable_indexonlyscan = off;
+-- Live c7 is 72. The c7 index must return the live row for 72 and drop the
+-- stale leaves for 70 and 71 (offsets misread would corrupt this).
+SELECT count(*) AS c7_eq_72 FROM hi_ddl WHERE c7 = 72 AND payload IS NOT NULL;
+ c7_eq_72
+----------
+ 1
+(1 row)
+
+SELECT count(*) AS c7_eq_70_stale FROM hi_ddl WHERE c7 = 70 AND payload IS NOT NULL;
+ c7_eq_70_stale
+----------------
+ 0
+(1 row)
+
+SELECT count(*) AS c7_eq_71_stale FROM hi_ddl WHERE c7 = 71 AND payload IS NOT NULL;
+ c7_eq_71_stale
+----------------
+ 0
+(1 row)
+
+-- c2 never changed across the chain: its leaf must NOT be judged stale even
+-- though a crossed hop changed c7. A misread bitmap could spuriously flag it.
+SELECT count(*) AS c2_eq_10_current FROM hi_ddl WHERE c2 = 10 AND payload IS NOT NULL;
+ c2_eq_10_current
+------------------
+ 1
+(1 row)
+
+-- (c) Continue churning c7 AFTER the ADD COLUMN: the new hop's bitmap is sized
+-- for natts 9 (2 bytes); the old hops are 1 byte. A chain with mixed-size
+-- bitmaps must still resolve correctly.
+UPDATE hi_ddl SET c7 = 73 WHERE c1 = 1;
+SELECT count(*) AS c7_eq_73 FROM hi_ddl WHERE c7 = 73 AND payload IS NOT NULL;
+ c7_eq_73
+----------
+ 1
+(1 row)
+
+SELECT count(*) AS c7_eq_72_now_stale FROM hi_ddl WHERE c7 = 72 AND payload IS NOT NULL;
+ c7_eq_72_now_stale
+--------------------
+ 0
+(1 row)
+
+-- (d) Collapse the chain to stubs via VACUUM, then read again: the stub must
+-- preserve its write-time natts so its bitmap stays locatable post-ADD COLUMN.
+UPDATE hi_ddl SET c7 = 74 WHERE c1 = 1;
+VACUUM (INDEX_CLEANUP off) hi_ddl;
+SELECT count(*) AS c7_eq_74_after_vacuum FROM hi_ddl WHERE c7 = 74 AND payload IS NOT NULL;
+ c7_eq_74_after_vacuum
+-----------------------
+ 1
+(1 row)
+
+SELECT count(*) AS c2_eq_10_after_vacuum FROM hi_ddl WHERE c2 = 10 AND payload IS NOT NULL;
+ c2_eq_10_after_vacuum
+-----------------------
+ 1
+(1 row)
+
+-- (e) DROP COLUMN keeps the attnum slot (no renumber), so bitmaps stay aligned.
+ALTER TABLE hi_ddl DROP COLUMN c4;
+SELECT count(*) AS c7_after_drop FROM hi_ddl WHERE c7 = 74 AND payload IS NOT NULL;
+ c7_after_drop
+---------------
+ 1
+(1 row)
+
+SELECT count(*) AS c2_after_drop FROM hi_ddl WHERE c2 = 10 AND payload IS NOT NULL;
+ c2_after_drop
+---------------
+ 1
+(1 row)
+
+-- (f) DROP INDEX on the churned column: remaining indexes still resolve.
+DROP INDEX hi_ddl_c7;
+SELECT count(*) AS c2_after_dropidx FROM hi_ddl WHERE c2 = 10 AND payload IS NOT NULL;
+ c2_after_dropidx
+------------------
+ 1
+(1 row)
+
+RESET enable_seqscan;
+RESET enable_bitmapscan;
+RESET enable_indexonlyscan;
+-- The seqscan truth confirms the live row; the count assertions above (read
+-- through the post-DDL indexes) match it, which is what would break if a
+-- mis-sized bitmap corrupted the staleness verdict.
+SELECT c1, c2, c7 FROM hi_ddl WHERE c1 = 1;
+ c1 | c2 | c7
+----+----+----
+ 1 | 10 | 74
+(1 row)
+
+DROP TABLE hi_ddl;
+-- ---------------------------------------------------------------------------
+-- Cleanup
+-- ---------------------------------------------------------------------------
+DROP FUNCTION get_hi_count(text);
+DROP FUNCTION get_hot_count(text);
+-- pageinspect and amcheck were both created above with IF NOT EXISTS and may
+-- have pre-existed this test; leave them, matching amcheck's treatment,
+-- rather than risk dropping an extension this test did not create.
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 39905c2de14..d8285423220 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1810,6 +1810,8 @@ pg_stat_all_indexes| SELECT c.oid AS relid,
pg_stat_get_lastscan(i.oid) AS last_idx_scan,
pg_stat_get_tuples_returned(i.oid) AS idx_tup_read,
pg_stat_get_tuples_fetched(i.oid) AS idx_tup_fetch,
+ pg_stat_get_tuples_hot_indexed_updated_skipped(i.oid) AS n_tup_hot_indexed_upd_skipped,
+ pg_stat_get_tuples_hot_indexed_updated_matched(i.oid) AS n_tup_hot_indexed_upd_matched,
pg_stat_get_stat_reset_time(i.oid) AS stats_reset
FROM (((pg_class c
JOIN pg_index x ON ((c.oid = x.indrelid)))
@@ -1829,6 +1831,7 @@ pg_stat_all_tables| SELECT c.oid AS relid,
pg_stat_get_tuples_updated(c.oid) AS n_tup_upd,
pg_stat_get_tuples_deleted(c.oid) AS n_tup_del,
pg_stat_get_tuples_hot_updated(c.oid) AS n_tup_hot_upd,
+ pg_stat_get_tuples_hot_indexed_updated(c.oid) AS n_tup_hot_indexed_upd,
pg_stat_get_tuples_newpage_updated(c.oid) AS n_tup_newpage_upd,
pg_stat_get_live_tuples(c.oid) AS n_live_tup,
pg_stat_get_dead_tuples(c.oid) AS n_dead_tup,
@@ -2332,6 +2335,8 @@ pg_stat_sys_indexes| SELECT relid,
last_idx_scan,
idx_tup_read,
idx_tup_fetch,
+ n_tup_hot_indexed_upd_skipped,
+ n_tup_hot_indexed_upd_matched,
stats_reset
FROM pg_stat_all_indexes
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
@@ -2348,6 +2353,7 @@ pg_stat_sys_tables| SELECT relid,
n_tup_upd,
n_tup_del,
n_tup_hot_upd,
+ n_tup_hot_indexed_upd,
n_tup_newpage_upd,
n_live_tup,
n_dead_tup,
@@ -2387,6 +2393,8 @@ pg_stat_user_indexes| SELECT relid,
last_idx_scan,
idx_tup_read,
idx_tup_fetch,
+ n_tup_hot_indexed_upd_skipped,
+ n_tup_hot_indexed_upd_matched,
stats_reset
FROM pg_stat_all_indexes
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
@@ -2403,6 +2411,7 @@ pg_stat_user_tables| SELECT relid,
n_tup_upd,
n_tup_del,
n_tup_hot_upd,
+ n_tup_hot_indexed_upd,
n_tup_newpage_upd,
n_live_tup,
n_dead_tup,
@@ -2458,6 +2467,7 @@ pg_stat_xact_all_tables| SELECT c.oid AS relid,
pg_stat_get_xact_tuples_updated(c.oid) AS n_tup_upd,
pg_stat_get_xact_tuples_deleted(c.oid) AS n_tup_del,
pg_stat_get_xact_tuples_hot_updated(c.oid) AS n_tup_hot_upd,
+ pg_stat_get_xact_tuples_hot_indexed_updated(c.oid) AS n_tup_hot_indexed_upd,
pg_stat_get_xact_tuples_newpage_updated(c.oid) AS n_tup_newpage_upd
FROM ((pg_class c
LEFT JOIN pg_index i ON ((c.oid = i.indrelid)))
@@ -2475,6 +2485,7 @@ pg_stat_xact_sys_tables| SELECT relid,
n_tup_upd,
n_tup_del,
n_tup_hot_upd,
+ n_tup_hot_indexed_upd,
n_tup_newpage_upd
FROM pg_stat_xact_all_tables
WHERE ((schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR (schemaname ~ '^pg_toast'::text));
@@ -2498,6 +2509,7 @@ pg_stat_xact_user_tables| SELECT relid,
n_tup_upd,
n_tup_del,
n_tup_hot_upd,
+ n_tup_hot_indexed_upd,
n_tup_newpage_upd
FROM pg_stat_xact_all_tables
WHERE ((schemaname <> ALL (ARRAY['pg_catalog'::name, 'information_schema'::name])) AND (schemaname !~ '^pg_toast'::text));
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index bd95cc24977..4fde5b6b0c6 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -147,6 +147,7 @@ test: fast_default
# HOT updates tests
# ----------
test: hot_updates
+test: hot_indexed_updates
# run tablespace test at the end because it drops the tablespace created during
# setup that other tests may use.
diff --git a/src/test/regress/sql/hot_indexed_updates.sql b/src/test/regress/sql/hot_indexed_updates.sql
new file mode 100644
index 00000000000..feac7191c52
--- /dev/null
+++ b/src/test/regress/sql/hot_indexed_updates.sql
@@ -0,0 +1,1167 @@
+--
+-- HOT_INDEXED_UPDATES
+-- Test HOT-indexed update (hot-indexed), aka HOT-indexed, behaviour
+--
+-- Every UPDATE in this file modifies at least one non-summarizing
+-- indexed attribute. On a pre-hot-indexed server all of these would be
+-- non-HOT; on the hot-indexed branch each eligible update stays on-page and
+-- inserts into only the indexes whose attributes actually changed.
+--
+-- We verify four things:
+-- (A) pg_stat counters: HOT and hot-indexed counts increment as expected
+-- (B) index lookups return the new value and not the stale value
+-- for EQUALITY queries (the read-side staleness test drops a
+-- leaf whose covered attribute changed on the way to the live tuple)
+-- (C) pg_relation_hot_indexed_stats reports the HOT-indexed versions we expect
+-- (D) **RANGE/INEQUALITY** queries return the correct number of
+-- tuples -- this is the class of bugs where a stale btree
+-- entry's key is still reachable via a looser scan key; the
+-- crossed-attribute bitmap drops the stale arrival because the index's
+-- attribute changed between that leaf's target and the live tuple
+--
+
+CREATE EXTENSION IF NOT EXISTS pageinspect;
+
+CREATE OR REPLACE FUNCTION get_hot_count(rel_name text)
+RETURNS TABLE (updates BIGINT, hot BIGINT) AS $$
+DECLARE rel_oid oid;
+BEGIN
+ rel_oid := rel_name::regclass::oid;
+ updates := COALESCE(pg_stat_get_tuples_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_updated(rel_oid), 0);
+ hot := COALESCE(pg_stat_get_tuples_hot_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_hot_updated(rel_oid), 0);
+ RETURN NEXT;
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE OR REPLACE FUNCTION get_hi_count(rel_name text)
+RETURNS TABLE (updates BIGINT, hot BIGINT, hot_idx BIGINT) AS $$
+DECLARE rel_oid oid;
+BEGIN
+ rel_oid := rel_name::regclass::oid;
+ updates := COALESCE(pg_stat_get_tuples_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_updated(rel_oid), 0);
+ hot := COALESCE(pg_stat_get_tuples_hot_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_hot_updated(rel_oid), 0);
+ hot_idx := COALESCE(pg_stat_get_tuples_hot_indexed_updated(rel_oid), 0) +
+ COALESCE(pg_stat_get_xact_tuples_hot_indexed_updated(rel_oid), 0);
+ RETURN NEXT;
+END;
+$$ LANGUAGE plpgsql;
+
+
+-- ---------------------------------------------------------------------------
+-- 1. Basic hot-indexed: modifying an indexed column stays HOT and counts as hot-indexed
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_basic (
+ id int PRIMARY KEY,
+ indexed_col int,
+ non_indexed_col text
+) WITH (fillfactor = 50);
+CREATE INDEX hi_basic_idx ON hi_basic(indexed_col);
+
+INSERT INTO hi_basic VALUES (1, 100, 'initial');
+
+-- Pre-hot-indexed this would be non-HOT. Under hot-indexed it's HOT-indexed; both the
+-- HOT counter and the hot-indexed counter advance.
+UPDATE hi_basic SET indexed_col = 150 WHERE id = 1;
+SELECT pg_stat_force_next_flush();
+SELECT * FROM get_hi_count('hi_basic');
+
+-- The new value is reachable via the index.
+SET enable_seqscan = off;
+EXPLAIN (COSTS OFF) SELECT id, indexed_col FROM hi_basic WHERE indexed_col = 150;
+SELECT id, indexed_col FROM hi_basic WHERE indexed_col = 150;
+
+-- The old value is not reachable through this index: the stale btree
+-- entry (indexed_col=100) walks to the current tuple via the hot-indexed hop,
+-- nodeIndexscan re-evaluates `indexed_col = 100` against the current
+-- tuple (indexed_col=150), and the row is correctly dropped. This is
+-- the equality-lookup case the crossed-attribute bitmap handles.
+EXPLAIN (COSTS OFF) SELECT id FROM hi_basic WHERE indexed_col = 100;
+SELECT id FROM hi_basic WHERE indexed_col = 100;
+RESET enable_seqscan;
+
+-- pg_relation_hot_indexed_stats sees one HOT-indexed version, zero HOT redirects (the
+-- chain has not yet been pruned so no LP_REDIRECT exists).
+SELECT n_hot_indexed, n_chains, avg_chain_len, max_chain_len
+FROM pg_relation_hot_indexed_stats('hi_basic');
+
+DROP TABLE hi_basic;
+
+-- ---------------------------------------------------------------------------
+-- 2. RANGE/INEQUALITY correctness after hot-indexed on an indexed column
+--
+-- This is the test class that catches the hot-indexed false-dup bug: a stale
+-- btree entry whose key value still satisfies the range predicate,
+-- reachable via the hot-indexed chain hop.
+--
+-- To exercise the bug we must force an IndexScan plan (the
+-- IndexOnlyScan path permissively drops every hot-indexed-reachable index-only
+-- hit; the BitmapHeapScan path dedups by TID). We include a payload
+-- column not present in the PK so the planner must heap-fetch.
+--
+-- The read-side crossed-attribute bitmap makes the IndexScan return the correct
+-- count of 1: the stale entry ('1','5') chain-walks to the live tuple across
+-- the b-changing hop, and because the PK covers b the overlap is non-empty, so
+-- the stale leaf is dropped. The fresh entry ('1','15') points directly at the
+-- live tuple (no hop after it) and is kept. The ORDER BY likewise returns the
+-- single live row.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_range (
+ a int,
+ b int,
+ payload text,
+ PRIMARY KEY (a, b)
+) WITH (fillfactor = 50);
+
+INSERT INTO hi_range VALUES (1, 5, 'hi');
+
+-- hot-indexed update on the second PK column: stale btree entry ('1','5')
+-- remains, new entry ('1','15') inserted. The stale entry points at
+-- the chain root; the fresh entry points directly at the new
+-- heap-only tuple.
+UPDATE hi_range SET b = 15 WHERE a = 1 AND b = 5;
+
+SET enable_seqscan = off;
+SET enable_bitmapscan = off;
+
+-- IndexScan: payload IS NOT NULL forces heap fetch, no IndexOnlyScan.
+-- The stale ('1','5') leaf is dropped by the crossed-attribute bitmap, so this
+-- returns 1.
+EXPLAIN (COSTS OFF)
+SELECT count(*) FROM hi_range WHERE a = 1 AND b < 100 AND payload IS NOT NULL;
+SELECT count(*) FROM hi_range WHERE a = 1 AND b < 100 AND payload IS NOT NULL;
+SELECT a, b FROM hi_range WHERE a = 1 AND payload IS NOT NULL ORDER BY b;
+
+-- IndexOnlyScan: the page holds a preserved HOT-indexed member so it is never all-visible; IOS
+-- performs the heap fetch and the crossed-attribute bitmap drops the stale ('1','5')
+-- leaf, so count = 1.
+EXPLAIN (COSTS OFF) SELECT count(*) FROM hi_range WHERE a = 1 AND b < 100;
+SELECT count(*) FROM hi_range WHERE a = 1 AND b < 100;
+
+-- BitmapHeapScan: TID dedup collapses the stale and fresh hits.
+SET enable_indexscan = off;
+SET enable_indexonlyscan = off;
+RESET enable_bitmapscan;
+EXPLAIN (COSTS OFF) SELECT count(*) FROM hi_range WHERE a = 1 AND b < 100;
+SELECT count(*) FROM hi_range WHERE a = 1 AND b < 100;
+RESET enable_indexscan;
+RESET enable_indexonlyscan;
+
+-- SeqScan: reads the heap directly, sees exactly one live tuple.
+RESET enable_seqscan;
+SET enable_indexscan = off;
+SET enable_indexonlyscan = off;
+SET enable_bitmapscan = off;
+EXPLAIN (COSTS OFF) SELECT count(*) FROM hi_range WHERE a = 1 AND b < 100;
+SELECT count(*) FROM hi_range WHERE a = 1 AND b < 100;
+RESET enable_indexscan;
+RESET enable_indexonlyscan;
+RESET enable_bitmapscan;
+
+-- Same shape on a secondary (non-PK) btree: another hot-indexed update on b.
+CREATE INDEX hi_range_b_idx ON hi_range(b);
+UPDATE hi_range SET b = 25 WHERE a = 1 AND b = 15;
+
+SET enable_seqscan = off;
+SET enable_bitmapscan = off;
+-- IndexScan path on the secondary index; same fix applies.
+SELECT count(*) FROM hi_range WHERE b BETWEEN 0 AND 100 AND payload IS NOT NULL;
+RESET enable_seqscan;
+RESET enable_bitmapscan;
+
+DROP TABLE hi_range;
+
+-- ---------------------------------------------------------------------------
+-- 3. All-or-none on a multi-indexed table: hot-indexed only touches indexes
+-- whose attributes changed
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_multi (
+ id int PRIMARY KEY,
+ col_a int,
+ col_b int,
+ col_c int,
+ non_indexed text
+) WITH (fillfactor = 50);
+CREATE INDEX hi_multi_a_idx ON hi_multi(col_a);
+CREATE INDEX hi_multi_b_idx ON hi_multi(col_b);
+CREATE INDEX hi_multi_c_idx ON hi_multi(col_c);
+
+INSERT INTO hi_multi VALUES (1, 10, 20, 30, 'initial');
+
+-- col_a only: under hot-indexed this is HOT-indexed, and only hi_multi_a_idx
+-- gets a new entry. hi_multi_b_idx / hi_multi_c_idx keep pointing
+-- at the chain root.
+UPDATE hi_multi SET col_a = 15 WHERE id = 1;
+SELECT pg_stat_force_next_flush();
+SELECT * FROM get_hi_count('hi_multi');
+
+-- Lookups on all three indexes return the row.
+SET enable_seqscan = off;
+SELECT id FROM hi_multi WHERE col_a = 15;
+SELECT id FROM hi_multi WHERE col_b = 20;
+SELECT id FROM hi_multi WHERE col_c = 30;
+
+-- Old col_a value is unreachable by equality (stale entry dropped by the
+-- read-side crossed-attribute bitmap).
+SELECT id FROM hi_multi WHERE col_a = 10;
+RESET enable_seqscan;
+
+DROP TABLE hi_multi;
+
+-- ---------------------------------------------------------------------------
+-- 4. Multi-column btree: hot-indexed on part of a composite key
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_composite (
+ id int PRIMARY KEY,
+ col_a int,
+ col_b int,
+ data text
+) WITH (fillfactor = 50);
+CREATE INDEX hi_composite_ab_idx ON hi_composite(col_a, col_b);
+
+INSERT INTO hi_composite VALUES (1, 10, 20, 'data');
+
+-- col_a is part of the composite key: hot-indexed.
+UPDATE hi_composite SET col_a = 15;
+SELECT pg_stat_force_next_flush();
+SELECT * FROM get_hi_count('hi_composite');
+
+-- Reset and then update col_b (also part of the key).
+UPDATE hi_composite SET col_a = 10;
+UPDATE hi_composite SET col_b = 25;
+SELECT pg_stat_force_next_flush();
+SELECT * FROM get_hi_count('hi_composite');
+
+DROP TABLE hi_composite;
+
+-- ---------------------------------------------------------------------------
+-- 5. Partial index: status transition out-of-predicate
+--
+-- 'status' is a partial-index predicate column. A change to a predicate
+-- column can flip a row in or out of the index, which the read-side key
+-- recheck cannot detect, so HeapUpdateHotAllowable conservatively disqualifies
+-- HOT-indexed for any predicate-column change (even this out-of-predicate ->
+-- out-of-predicate case). The update is therefore non-HOT, and the partial
+-- index correctly stays empty for these non-'active' rows.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_partial (
+ id int PRIMARY KEY,
+ status text,
+ data text
+) WITH (fillfactor = 50);
+CREATE INDEX hi_partial_active_idx ON hi_partial(status) WHERE status = 'active';
+
+INSERT INTO hi_partial VALUES (1, 'active', 'data1');
+INSERT INTO hi_partial VALUES (2, 'inactive', 'data2');
+INSERT INTO hi_partial VALUES (3, 'deleted', 'data3');
+
+-- out -> out transition on the predicate column: HOT-indexed keeps it on-page,
+-- and the partial index gets no entry (the row satisfies the predicate neither
+-- before nor after the update).
+UPDATE hi_partial SET status = 'deleted' WHERE id = 2;
+SELECT pg_stat_force_next_flush();
+SELECT * FROM get_hi_count('hi_partial');
+
+-- The partial index still correctly answers "active" queries.
+SELECT id, status FROM hi_partial WHERE status = 'active';
+
+DROP TABLE hi_partial;
+
+-- ---------------------------------------------------------------------------
+-- 6. Partition: hot-indexed inside one partition
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_part (
+ id int,
+ partition_key int,
+ indexed_col int,
+ data text,
+ PRIMARY KEY (id, partition_key)
+) PARTITION BY RANGE (partition_key);
+CREATE TABLE hi_part_1 PARTITION OF hi_part
+ FOR VALUES FROM (1) TO (100) WITH (fillfactor = 50);
+CREATE INDEX hi_part_idx ON hi_part(indexed_col);
+
+INSERT INTO hi_part VALUES (1, 50, 100, 'data');
+
+UPDATE hi_part SET indexed_col = 150 WHERE id = 1;
+SELECT pg_stat_force_next_flush();
+SELECT * FROM get_hi_count('hi_part_1');
+
+SET enable_seqscan = off;
+SELECT id FROM hi_part WHERE indexed_col = 150;
+SELECT id FROM hi_part WHERE indexed_col = 100;
+RESET enable_seqscan;
+
+DROP TABLE hi_part CASCADE;
+
+-- ---------------------------------------------------------------------------
+-- 7. Trigger modifies indexed column: hot-indexed, not non-HOT
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_trigger (
+ id int PRIMARY KEY,
+ triggered_col int,
+ data text
+) WITH (fillfactor = 50);
+CREATE INDEX hi_trigger_idx ON hi_trigger(triggered_col);
+
+CREATE OR REPLACE FUNCTION hi_trigger_bump()
+RETURNS TRIGGER AS $$
+BEGIN
+ NEW.triggered_col = NEW.triggered_col + 1;
+ RETURN NEW;
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE TRIGGER before_update_bump
+ BEFORE UPDATE ON hi_trigger
+ FOR EACH ROW
+ EXECUTE FUNCTION hi_trigger_bump();
+
+INSERT INTO hi_trigger VALUES (1, 100, 'initial');
+
+-- UPDATE's SET clause doesn't touch the indexed column, but the
+-- trigger modifies it via heap_modify_tuple. hot-indexed must detect this
+-- and keep the tuple on-page (HEAP_INDEXED_UPDATED) plus a new btree entry.
+UPDATE hi_trigger SET data = 'updated' WHERE id = 1;
+SELECT pg_stat_force_next_flush();
+SELECT * FROM get_hi_count('hi_trigger');
+SELECT triggered_col FROM hi_trigger WHERE id = 1;
+
+-- New value reachable.
+SET enable_seqscan = off;
+SELECT id FROM hi_trigger WHERE triggered_col = 101;
+SELECT id FROM hi_trigger WHERE triggered_col = 100;
+RESET enable_seqscan;
+
+DROP TABLE hi_trigger CASCADE;
+DROP FUNCTION hi_trigger_bump();
+
+-- ---------------------------------------------------------------------------
+-- 8. JSONB expression index: HOT-indexed is not yet supported on expression
+-- indexes, so the update falls back to a non-HOT update (hot_idx = 0).
+-- Reads stay correct.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_jsonb (
+ id int PRIMARY KEY,
+ data jsonb
+) WITH (fillfactor = 50);
+CREATE INDEX hi_jsonb_name_idx ON hi_jsonb ((data->>'name'));
+
+INSERT INTO hi_jsonb VALUES (1, '{"name":"Alice","age":30}');
+
+-- Changing the indexed expression's value (name): expression indexes are not
+-- yet supported, so this is a non-HOT update.
+UPDATE hi_jsonb SET data = jsonb_set(data, '{name}', '"Alice2"') WHERE id = 1;
+SELECT pg_stat_force_next_flush();
+SELECT * FROM get_hi_count('hi_jsonb');
+
+SET enable_seqscan = off;
+SELECT id FROM hi_jsonb WHERE data->>'name' = 'Alice2';
+SELECT id FROM hi_jsonb WHERE data->>'name' = 'Alice';
+RESET enable_seqscan;
+
+DROP TABLE hi_jsonb;
+
+-- ---------------------------------------------------------------------------
+-- 9. GIN index with changed extracted keys: hot-indexed
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_gin (
+ id int PRIMARY KEY,
+ tags text[]
+) WITH (fillfactor = 50);
+CREATE INDEX hi_gin_tags_idx ON hi_gin USING gin (tags);
+
+INSERT INTO hi_gin VALUES (1, ARRAY['tag1', 'tag2']);
+
+-- Adding a tag yields a different extracted-key set: hot-indexed.
+UPDATE hi_gin SET tags = ARRAY['tag1', 'tag2', 'tag5'] WHERE id = 1;
+SELECT pg_stat_force_next_flush();
+SELECT * FROM get_hi_count('hi_gin');
+
+SET enable_seqscan = off;
+SELECT id FROM hi_gin WHERE tags @> ARRAY['tag5'];
+RESET enable_seqscan;
+
+DROP TABLE hi_gin;
+
+-- ---------------------------------------------------------------------------
+-- 10. Per-index HOT-indexed counters: skipped vs matched
+--
+-- A table with two independent secondary indexes. An UPDATE touches a
+-- column covered by only one of them; the HOT-indexed path must insert
+-- into that one index and skip the other. pg_stat_all_indexes reports
+-- matched>0 on the updated index and skipped>0 on the untouched index.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hotidx_perindex (
+ id int PRIMARY KEY,
+ a int,
+ b int
+) WITH (fillfactor = 50);
+CREATE INDEX hotidx_perindex_a ON hotidx_perindex(a);
+CREATE INDEX hotidx_perindex_b ON hotidx_perindex(b);
+
+INSERT INTO hotidx_perindex VALUES (1, 100, 200);
+
+-- Modify only column a. HOT-indexed inserts into hotidx_perindex_a and
+-- skips hotidx_perindex_b (primary key indrelid is the table itself and
+-- also unchanged, so it counts as skipped too).
+UPDATE hotidx_perindex SET a = 101 WHERE id = 1;
+
+-- Force flush of pending stats to the shared entry.
+SELECT pg_stat_force_next_flush();
+
+SELECT indexrelname,
+ n_tup_hot_indexed_upd_matched AS matched,
+ n_tup_hot_indexed_upd_skipped AS skipped
+ FROM pg_stat_all_indexes
+ WHERE relname = 'hotidx_perindex'
+ ORDER BY indexrelname;
+
+-- A second UPDATE touching only b inverts the assignment.
+UPDATE hotidx_perindex SET b = 201 WHERE id = 1;
+SELECT pg_stat_force_next_flush();
+
+SELECT indexrelname,
+ n_tup_hot_indexed_upd_matched AS matched,
+ n_tup_hot_indexed_upd_skipped AS skipped
+ FROM pg_stat_all_indexes
+ WHERE relname = 'hotidx_perindex'
+ ORDER BY indexrelname;
+
+-- Invariant: matched + skipped == owning table's n_tup_hot_indexed_upd.
+SELECT indexrelname,
+ n_tup_hot_indexed_upd_matched + n_tup_hot_indexed_upd_skipped AS total,
+ (SELECT n_tup_hot_indexed_upd FROM pg_stat_all_tables
+ WHERE relname = 'hotidx_perindex') AS table_hot_idx_upd
+ FROM pg_stat_all_indexes
+ WHERE relname = 'hotidx_perindex'
+ ORDER BY indexrelname;
+
+-- Boolean assertion of the same invariant. This is the canonical form
+-- reviewers asked for: every index entry is either matched (the index
+-- got a fresh insert this UPDATE) or skipped (HOT-indexed correctly
+-- avoided an insert because the index's attrs did not change). If the
+-- two counters drift apart from the table-level n_tup_hot_indexed_upd we
+-- have either lost a per-index increment or double-counted one.
+SELECT bool_and((n_tup_hot_indexed_upd_matched + n_tup_hot_indexed_upd_skipped) =
+ (SELECT n_tup_hot_indexed_upd FROM pg_stat_all_tables
+ WHERE relname = 'hotidx_perindex'))
+ AS perindex_invariant_holds
+ FROM pg_stat_all_indexes
+ WHERE relname = 'hotidx_perindex';
+
+DROP TABLE hotidx_perindex;
+
+-- ---------------------------------------------------------------------------
+-- 11. Long hot-loop UPDATE stays compact and HOT-indexed
+--
+-- A long run of HOT-indexed UPDATEs to a single row stays compact: prune
+-- collapses each dead version to a redirect to the live tuple and reuses its
+-- slot, so the heap stays bounded and the chain does not grow unbounded.
+-- Every UPDATE that changes the indexed column (and leaves another index,
+-- here the PK, unchanged) takes the HOT-indexed path.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_chaincap (
+ id int PRIMARY KEY,
+ a int
+) WITH (fillfactor = 10);
+CREATE INDEX hi_chaincap_a_idx ON hi_chaincap(a);
+
+INSERT INTO hi_chaincap VALUES (1, 0);
+
+DO $$
+DECLARE
+ i int;
+BEGIN
+ FOR i IN 1 .. 200 LOOP
+ UPDATE hi_chaincap SET a = i WHERE id = 1;
+ END LOOP;
+END $$;
+
+-- After 200 UPDATEs the row's value is 200.
+SELECT a FROM hi_chaincap WHERE id = 1;
+
+-- Every UPDATE took the HOT-indexed path (the PK index is unchanged, so it is
+-- skipped), so n_tup_hot_indexed_upd advanced.
+SELECT pg_stat_force_next_flush();
+SELECT hot_idx > 0 AS hot_indexed_fired
+ FROM get_hi_count('hi_chaincap');
+
+-- The heap stayed compact: prune+collapse reclaimed the dead versions, so the
+-- single live row stays within a couple of pages. pg_relation_size reflects
+-- the table's actual current size regardless of vacuum/analyze stats, unlike
+-- pg_class.relpages (which is only updated by VACUUM/ANALYZE and would be
+-- trivially <= 1 on this never-vacuumed table even if pruning had failed and
+-- the heap had ballooned).
+SELECT pg_relation_size('hi_chaincap') <= 8192 * 2 AS heap_stayed_compact;
+
+DROP TABLE hi_chaincap;
+
+-- ---------------------------------------------------------------------------
+-- 12. Reclamation of a collapsed HOT-indexed chain by prune
+--
+-- A dead HOT-indexed chain member is preserved at prune time (its stale leaf
+-- may still exist) and the chain collapses to an LP_REDIRECT forwarder; the
+-- index cleanup pass then sweeps the stale leaf, and a later VACUUM reclaims
+-- the now-unreferenced member and re-points the redirect once it falls below
+-- the global xmin horizon. We assert only that the chain collapses
+-- (n_chains = 0): that is the deterministic proof the mechanism worked. The
+-- final member's reclaim (n_hot_indexed reaching 0) additionally requires the
+-- deleted tuple to fall below the global xmin horizon, which a snapshot held
+-- elsewhere in the running regression cluster can delay indefinitely -- so we
+-- do not assert an exact n_hot_indexed here (autovacuum_enabled is off only to
+-- keep our own VACUUMs from being skipped on a stolen lock).
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_reclaim (
+ id int PRIMARY KEY,
+ a int
+) WITH (fillfactor = 50, autovacuum_enabled = false);
+CREATE INDEX hi_reclaim_a_idx ON hi_reclaim(a);
+
+INSERT INTO hi_reclaim VALUES (1, 100);
+-- Generate a collapsed chain via a HOT-indexed update.
+UPDATE hi_reclaim SET a = 200 WHERE id = 1;
+SELECT n_hot_indexed >= 1 AS hot_indexed_present_before_reclaim
+ FROM pg_relation_hot_indexed_stats('hi_reclaim');
+
+-- Delete the live tuple. The first VACUUM collapses the dead chain and sweeps
+-- the stale leaf; a later VACUUM reclaims the now-unreferenced member once it
+-- falls below the global xmin horizon. Assert only the deterministic
+-- collapse (n_chains = 0); the member-reclaim count is horizon-dependent and
+-- not something a regress test can pin down.
+DELETE FROM hi_reclaim WHERE id = 1;
+VACUUM hi_reclaim;
+VACUUM hi_reclaim;
+
+SELECT n_chains = 0 AS chain_collapsed_after_reclaim
+ FROM pg_relation_hot_indexed_stats('hi_reclaim');
+
+DROP TABLE hi_reclaim;
+
+-- ---------------------------------------------------------------------------
+-- 13. Page with a preserved HOT-indexed member is never marked all-visible
+--
+-- pruneheap deliberately leaves PD_ALL_VISIBLE clear on any page that still
+-- carries a preserved HOT-indexed member: an index-only scan must heap-fetch
+-- through the chain so the read-side crossed-attribute bitmap can filter stale btree
+-- entries.
+--
+-- We force the freeze path with VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) and
+-- then read pd_flags via pageinspect.page_header. The page must still carry
+-- a HOT-indexed member (n_hot_indexed > 0) AND must not have PD_ALL_VISIBLE
+-- (0x0004).
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_vm (
+ id int PRIMARY KEY,
+ a int
+) WITH (fillfactor = 50, autovacuum_enabled = false);
+CREATE INDEX hi_vm_a_idx ON hi_vm(a);
+
+INSERT INTO hi_vm VALUES (1, 1);
+-- Two HOT-indexed updates leave a multi-hop chain, so a preserved HOT-indexed
+-- member remains on the page after prune, which is what this test needs.
+UPDATE hi_vm SET a = 2 WHERE id = 1;
+UPDATE hi_vm SET a = 3 WHERE id = 1;
+
+-- Force the all-visible bit decision: VACUUM with DISABLE_PAGE_SKIPPING
+-- considers every page; FREEZE pushes hint bits hard. After this, any
+-- page bearing a preserved HOT-indexed member must still report all_visible = 0.
+VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) hi_vm;
+
+SELECT n_hot_indexed >= 1 AS hot_indexed_present
+ FROM pg_relation_hot_indexed_stats('hi_vm');
+
+-- PD_ALL_VISIBLE = 0x0004. Must be 0 on a page with a preserved member.
+SELECT (flags & 4) = 0 AS not_marked_all_visible
+ FROM page_header(get_raw_page('hi_vm', 0));
+
+DROP TABLE hi_vm;
+
+-- ---------------------------------------------------------------------------
+-- 14. Cycle-key dedup: column rename a -> b -> a stays correct
+--
+-- A rename does not rewrite heap or index entries; it only updates the
+-- catalog. The relcache invalidation must trigger a fresh attribute
+-- bitmap and the HOT-indexed predicate must compare attribute *numbers*,
+-- not attribute *names*. After two renames that net to identity, every
+-- subsequent UPDATE must continue to drive the HOT-indexed path.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_cycle (
+ id int PRIMARY KEY,
+ a int
+) WITH (fillfactor = 50);
+CREATE INDEX hi_cycle_a_idx ON hi_cycle(a);
+
+INSERT INTO hi_cycle VALUES (1, 100);
+
+-- Cycle the column name and confirm both intermediate forms drive HOT-indexed.
+ALTER TABLE hi_cycle RENAME COLUMN a TO b;
+UPDATE hi_cycle SET b = 200 WHERE id = 1;
+SELECT pg_stat_force_next_flush();
+SELECT hot_idx > 0 AS hot_indexed_after_first_rename
+ FROM get_hi_count('hi_cycle');
+
+ALTER TABLE hi_cycle RENAME COLUMN b TO a;
+UPDATE hi_cycle SET a = 300 WHERE id = 1;
+-- Lookup via the index returns the current value, not any of the
+-- pre-rename values.
+SET enable_seqscan = off;
+SELECT id, a FROM hi_cycle WHERE a = 300;
+SELECT id FROM hi_cycle WHERE a = 100;
+SELECT id FROM hi_cycle WHERE a = 200;
+RESET enable_seqscan;
+
+DROP TABLE hi_cycle;
+
+-- ---------------------------------------------------------------------------
+-- 15. Summarizing-only column UPDATE produces CLASSIC, not INDEXED
+--
+-- HeapUpdateHotAllowable returns HEAP_HEAP_ONLY_UPDATE when every
+-- modified indexed attribute is covered only by summarizing indexes.
+-- A BRIN-only column is the canonical case: the BRIN index gets a
+-- new summary entry via aminsert, but no per-update btree entry is
+-- needed and HOT-indexed does not fire. The signal is
+-- n_tup_hot_upd > 0 with n_tup_hot_indexed_upd unchanged.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_brin (
+ id int PRIMARY KEY,
+ bcol int
+) WITH (fillfactor = 50);
+CREATE INDEX hi_brin_idx ON hi_brin USING brin(bcol);
+
+INSERT INTO hi_brin VALUES (1, 100);
+
+-- Capture the HOT-indexed counter before, drive a BRIN-only update,
+-- and assert that classic HOT advanced while HOT-indexed did not.
+SELECT pg_stat_force_next_flush();
+SELECT hot_idx AS hot_idx_before FROM get_hi_count('hi_brin') \gset
+UPDATE hi_brin SET bcol = 200 WHERE id = 1;
+SELECT pg_stat_force_next_flush();
+SELECT (hot - 0) > 0 AS classic_hot_fired,
+ hot_idx = :hot_idx_before AS hot_indexed_did_not_fire
+ FROM get_hi_count('hi_brin');
+
+-- The BRIN index sees the new value via aminsert.
+SELECT bcol FROM hi_brin WHERE id = 1;
+
+DROP TABLE hi_brin;
+
+-- ---------------------------------------------------------------------------
+-- 16. UNIQUE index on a type where image equality != operator equality
+--
+-- numeric 1.0 and 1.00 are equal under the btree opclass but have
+-- different on-disk images. A HOT-indexed update 1.0 -> 1.00 inserts a
+-- fresh leaf carrying the live image and leaves a stale leaf for 1.0
+-- (the hop's modified-attrs bitmap marks k changed, since modified-column
+-- detection is image-based). A later INSERT of a value equal under the
+-- opclass must still be detected as a duplicate: the unique check reaches
+-- the live tuple through the fresh leaf, which points directly at it (no hop
+-- after it, so the overlap is empty and the leaf is a genuine conflict); the
+-- stale 1.0 leaf is skipped because the k-changing hop overlaps the unique
+-- index's attribute.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_unum (k numeric UNIQUE, j int) WITH (fillfactor = 50);
+CREATE INDEX hi_unum_j ON hi_unum(j); -- 2nd indexed attr, kept fixed
+INSERT INTO hi_unum VALUES (1.0, 100);
+UPDATE hi_unum SET k = 1.00 WHERE j = 100; -- HOT-indexed: 1.0 -> 1.00
+SELECT n_hot_indexed > 0 AS made_hot_indexed
+ FROM pg_relation_hot_indexed_stats('hi_unum');
+-- A numerically-equal insert must conflict (the fresh leaf catches it):
+INSERT INTO hi_unum VALUES (1.0, 1); -- expect duplicate key error
+-- A genuinely different value is accepted:
+INSERT INTO hi_unum VALUES (2.0, 2);
+SELECT k, j FROM hi_unum ORDER BY j;
+DROP TABLE hi_unum;
+
+-- ---------------------------------------------------------------------------
+-- 17. CREATE INDEX and REINDEX over live HOT-indexed chains
+--
+-- A freshly built or rebuilt index must reflect current values, never a
+-- stale chain member: the build scans live tuples only and points each
+-- HOT-indexed live tuple's entry at its own TID, so the new entries have no
+-- hop after them and the crossed-attribute bitmap keeps them.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_reindex (id int PRIMARY KEY, a int, b int) WITH (fillfactor = 50);
+CREATE INDEX hi_reindex_a ON hi_reindex(a);
+INSERT INTO hi_reindex SELECT g, g, g FROM generate_series(1, 6) g;
+UPDATE hi_reindex SET a = a + 100; -- HOT-indexed on a
+UPDATE hi_reindex SET a = a + 100; -- again -> longer chains
+SELECT n_hot_indexed > 0 AS made_hot_indexed
+ FROM pg_relation_hot_indexed_stats('hi_reindex');
+-- Build a NEW index and REINDEX the existing one over the live chains.
+CREATE INDEX hi_reindex_b ON hi_reindex(b);
+REINDEX INDEX hi_reindex_a;
+SET enable_seqscan = off;
+SELECT id, a FROM hi_reindex WHERE a = 204; -- current value -> id 4
+SELECT count(*) FROM hi_reindex WHERE a = 4; -- obsolete value -> 0
+SELECT id FROM hi_reindex WHERE b = 2; -- via freshly built index -> 2
+RESET enable_seqscan;
+DROP TABLE hi_reindex;
+
+-- ---------------------------------------------------------------------------
+-- 18. DROP every index over live HOT-indexed chains, then VACUUM
+--
+-- After all indexes are dropped, heap pages may still carry preserved
+-- HOT-indexed members left by earlier updates. VACUUM of such a no-index
+-- relation must complete without error, and reads must stay correct via the
+-- redirect forwarders.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_dropidx (id int PRIMARY KEY, a int) WITH (fillfactor = 50, autovacuum_enabled = false);
+CREATE INDEX hi_dropidx_a ON hi_dropidx(a);
+INSERT INTO hi_dropidx SELECT g, g FROM generate_series(1, 6) g;
+UPDATE hi_dropidx SET a = a + 100; -- HOT-indexed on a
+UPDATE hi_dropidx SET a = a + 100; -- again -> longer chains
+SELECT n_hot_indexed > 0 AS made_hot_indexed
+ FROM pg_relation_hot_indexed_stats('hi_dropidx');
+-- Drop every index, leaving preserved HOT-indexed members with no index to sweep.
+DROP INDEX hi_dropidx_a;
+ALTER TABLE hi_dropidx DROP CONSTRAINT hi_dropidx_pkey;
+-- Must not crash on the no-index path; two passes exercise the second-pass
+-- reclaim guard as well.
+VACUUM hi_dropidx;
+VACUUM hi_dropidx;
+-- Reads remain correct after the indexes are gone.
+SELECT id, a FROM hi_dropidx ORDER BY id;
+DROP TABLE hi_dropidx;
+
+-- ---------------------------------------------------------------------------
+-- 19. Re-collapse of a data-redirect chain across partial VACUUMs
+--
+-- A chain that collapses to a HOT-indexed data redirect, is vacuumed with
+-- INDEX_CLEANUP off (so the stale leaves and the redirect survive), then
+-- receives further HOT-indexed updates that re-collapse the chain and
+-- re-point the redirect at a new live tuple, must not leave the redirect
+-- dangling. A subsequent full VACUUM must complete without error, leave the
+-- heap consistent (verify_heapam reports nothing), and reads must stay
+-- correct. (Regression: an earlier revision crashed reclaiming a mid-chain
+-- member while a data redirect still pointed past it.)
+-- ---------------------------------------------------------------------------
+CREATE EXTENSION IF NOT EXISTS amcheck;
+CREATE TABLE hi_recollapse (id int PRIMARY KEY, a int) WITH (fillfactor = 50, autovacuum_enabled = false);
+CREATE INDEX hi_recollapse_a ON hi_recollapse(a);
+INSERT INTO hi_recollapse VALUES (1, 1);
+-- First chain: two HOT-indexed updates, then prune to a data redirect while
+-- leaving the stale btree leaves in place (INDEX_CLEANUP off).
+UPDATE hi_recollapse SET a = 2 WHERE id = 1;
+UPDATE hi_recollapse SET a = 3 WHERE id = 1;
+VACUUM (INDEX_CLEANUP off) hi_recollapse;
+-- Re-collapse: more HOT-indexed updates extend the chain past the redirect
+-- target; the next prune re-points the data redirect at the new first live
+-- tuple and extends its union.
+UPDATE hi_recollapse SET a = 4 WHERE id = 1;
+UPDATE hi_recollapse SET a = 5 WHERE id = 1;
+VACUUM (INDEX_CLEANUP off) hi_recollapse;
+-- Full vacuum now reclaims the dead chain; the re-pointed redirect must not
+-- dangle. Two passes also exercise the redirect re-point second pass.
+VACUUM hi_recollapse;
+VACUUM hi_recollapse;
+-- Heap must be structurally consistent (no rows == no corruption).
+SELECT * FROM verify_heapam('hi_recollapse');
+SET enable_seqscan = off;
+SELECT id, a FROM hi_recollapse WHERE a = 5; -- current value -> id 1
+SELECT count(*) FROM hi_recollapse WHERE a = 3; -- obsolete value -> 0
+RESET enable_seqscan;
+SELECT id, a FROM hi_recollapse ORDER BY id;
+DROP TABLE hi_recollapse;
+
+-- ---------------------------------------------------------------------------
+-- 20. Index deletion over an entry that points at a data-redirect root
+--
+-- A data redirect is an LP_REDIRECT that carries a bitmap, so it reports
+-- lp_len > 0 (ItemIdHasStorage true) even though it is not a normal tuple.
+-- index_delete_check_htid must treat it as a redirect, not read its blob as a
+-- HeapTupleHeader. Reproduce: collapse a chain root to a data redirect while
+-- keeping the stale leaf that points at it (INDEX_CLEANUP off), then insert
+-- many duplicates of the stale key so btree bottom-up deletion runs
+-- heap_index_delete_tuples over that stale entry.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_iddel (id int, a int) WITH (fillfactor = 50, autovacuum_enabled = false);
+CREATE INDEX hi_iddel_a ON hi_iddel(a);
+INSERT INTO hi_iddel VALUES (1, 1);
+UPDATE hi_iddel SET a = a + 1 WHERE id = 1; -- HOT-indexed
+UPDATE hi_iddel SET a = a + 1 WHERE id = 1; -- multi-hop chain
+VACUUM (INDEX_CLEANUP off) hi_iddel; -- root -> data redirect, keep stale a=1 leaf
+-- Many duplicates of the stale key fill the leaf and trigger bottom-up
+-- deletion, which feeds the stale a=1 entry (htid -> the data-redirect root)
+-- to heap_index_delete_tuples. Must not crash or misread the blob.
+INSERT INTO hi_iddel SELECT g, 1 FROM generate_series(2, 3000) g;
+VACUUM hi_iddel;
+SELECT * FROM verify_heapam('hi_iddel');
+SET enable_seqscan = off;
+SELECT id, a FROM hi_iddel WHERE id = 1; -- current value -> a = 3
+RESET enable_seqscan;
+DROP TABLE hi_iddel;
+
+-- ---------------------------------------------------------------------------
+-- 21. A change to a column covered by a non-btree index AM is HOT-indexed
+--
+-- A HOT-indexed update leaves a stale pre-update leaf that the read side
+-- filters via the crossed-attribute bitmap, which is access-method agnostic.
+-- A column covered by a non-btree index (here a GiST index on a point column)
+-- is therefore HOT-indexed like any other, and the GiST index still returns
+-- correct results across the chain. A change to a btree-only column on the
+-- same table is likewise HOT-indexed.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_nonbtree (id int PRIMARY KEY, tag int, p point)
+ WITH (fillfactor = 10);
+CREATE INDEX hi_nonbtree_tag ON hi_nonbtree (tag); -- btree index
+CREATE INDEX hi_nonbtree_p ON hi_nonbtree USING gist (p); -- GiST, non-btree
+INSERT INTO hi_nonbtree SELECT g, g, point(g, g)
+ FROM generate_series(1, 200) g;
+
+-- Change the GiST-covered column first: HOT-indexed (hot_idx = 200).
+UPDATE hi_nonbtree SET p = point(p[0] + 1000, p[1] + 1000);
+SELECT hot_idx AS gist_col_hot_indexed FROM get_hi_count('hi_nonbtree');
+
+-- The GiST index must return correct results: the old positions are gone and
+-- every row is found at its new position (no stale leaf surfaces an old key).
+SET enable_seqscan = off;
+SELECT count(*) AS at_old_positions
+ FROM hi_nonbtree WHERE p <@ box(point(0, 0), point(300, 300));
+SELECT count(*) AS at_new_positions
+ FROM hi_nonbtree WHERE p <@ box(point(1000, 1000), point(1300, 1300));
+RESET enable_seqscan;
+
+-- Changing the btree-only column (p unchanged) stays HOT-indexed.
+UPDATE hi_nonbtree SET tag = tag + 1000;
+SELECT hot_idx > 0 AS btree_col_is_hot_indexed FROM get_hi_count('hi_nonbtree');
+DROP TABLE hi_nonbtree;
+
+-- ---------------------------------------------------------------------------
+-- 22. ABA on a unique key across two distinct live rows: a key cycled away
+-- and back must still collide with another row that holds it. The stale
+-- leaves left by the cycle must not let a genuine duplicate slip past the
+-- uniqueness check -- the read-side recheck compares the live key, not just
+-- a changed-attribute bitmap.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_aba (k int, v int) WITH (fillfactor = 50);
+CREATE UNIQUE INDEX hi_aba_k ON hi_aba (k);
+CREATE INDEX hi_aba_v ON hi_aba (v);
+INSERT INTO hi_aba VALUES (1, 10), (2, 20);
+
+-- Cycle row1's unique key 1 -> 3 -> 1 (v unchanged, so each step is
+-- HOT-indexed and leaves stale entries in hi_aba_k).
+UPDATE hi_aba SET k = 3 WHERE v = 10;
+UPDATE hi_aba SET k = 1 WHERE v = 10;
+SELECT hot_idx > 0 AS cycled_hot_indexed FROM get_hi_count('hi_aba');
+
+-- row1 is live at k = 1 again. Moving row2 onto k = 1 must raise a unique
+-- violation despite the stale '1' leaves from the cycle.
+UPDATE hi_aba SET k = 1 WHERE v = 20;
+DROP TABLE hi_aba;
+
+-- ---------------------------------------------------------------------------
+-- 23. Partial index whose predicate references a non-key column. Flipping the
+-- row out of the predicate while leaving the indexed key unchanged is
+-- HOT-indexed: the predicate column is part of the index's attribute set, so
+-- the crossed-attribute bitmap drops the now-stale partial-index entry on read
+-- (no value recheck is involved).
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_partpred (id int PRIMARY KEY, k int, active boolean)
+ WITH (fillfactor = 50);
+CREATE INDEX hi_partpred_k ON hi_partpred (k) WHERE active;
+INSERT INTO hi_partpred VALUES (1, 100, true);
+
+-- Flip the predicate column 'active' true -> false; the index key k is
+-- unchanged. The row no longer satisfies the predicate, so its partial-index
+-- entry must be removed, not left pointing into the chain.
+UPDATE hi_partpred SET active = false WHERE id = 1;
+SELECT pg_stat_force_next_flush();
+SELECT hot, hot_idx FROM get_hi_count('hi_partpred');
+
+-- The partial index must not surface the row now that active = false.
+-- A query whose qual exactly matches the partial predicate uses the index
+-- without re-filtering 'active' on the heap, so a stale entry would surface.
+SET enable_seqscan = off;
+SET enable_bitmapscan = off;
+EXPLAIN (COSTS OFF) SELECT id FROM hi_partpred WHERE active;
+SELECT id FROM hi_partpred WHERE k = 100 AND active;
+SELECT id FROM hi_partpred WHERE active;
+RESET enable_bitmapscan;
+RESET enable_seqscan;
+DROP TABLE hi_partpred;
+
+-- ---------------------------------------------------------------------------
+-- 24. Reclaim + stub mix. Repeated updates of column a followed by an update
+-- of column b build a chain whose prune reclaims the members whose change was
+-- superseded (a changed again) and keeps stubs for those that were not, so a
+-- root redirect ends up pointing at a stub and a later walk crosses mid-chain
+-- stubs. Reads through each index and amcheck must stay correct across the
+-- collapse, and a second round must walk the existing stubs without severing
+-- the chain.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_stubmix (id int PRIMARY KEY, a int, b int) WITH (fillfactor = 50, autovacuum_enabled = false);
+CREATE INDEX hi_stubmix_a ON hi_stubmix (a);
+CREATE INDEX hi_stubmix_b ON hi_stubmix (b);
+INSERT INTO hi_stubmix VALUES (1, 10, 100);
+UPDATE hi_stubmix SET a = 11 WHERE id = 1; -- changes a
+UPDATE hi_stubmix SET a = 12 WHERE id = 1; -- changes a again (supersedes)
+UPDATE hi_stubmix SET b = 101 WHERE id = 1; -- changes b
+VACUUM hi_stubmix;
+SET enable_seqscan = off;
+SET enable_bitmapscan = off;
+SELECT id, a, b FROM hi_stubmix WHERE a = 12; -- current a
+SELECT id, a, b FROM hi_stubmix WHERE b = 101; -- current b
+SELECT id FROM hi_stubmix WHERE a = 10; -- stale a: 0 rows
+RESET enable_bitmapscan;
+RESET enable_seqscan;
+SELECT * FROM verify_heapam('hi_stubmix'); -- no corruption across stubs
+-- A second round must walk the existing stubs (no priorXmax sever).
+UPDATE hi_stubmix SET a = 13 WHERE id = 1;
+VACUUM hi_stubmix;
+SELECT id, a, b FROM hi_stubmix WHERE a = 13;
+SELECT * FROM verify_heapam('hi_stubmix');
+DROP TABLE hi_stubmix;
+
+-- ---------------------------------------------------------------------------
+-- 25. Exclusion-constraint tables are HOT-indexed-eligible.
+--
+-- An exclusion constraint is enforced by check_exclusion_or_unique_constraint,
+-- which rechecks each candidate against the live tuple's current index-form
+-- with the constraint's own operators, so a stale entry left by a HOT-indexed
+-- update is skipped while the live key always has its own entry. Updating a
+-- non-constrained indexed column is HOT-indexed (the GiST exclusion index is
+-- skipped), and the constraint stays correct.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_excl (
+ id int PRIMARY KEY,
+ tag int,
+ during int4range,
+ EXCLUDE USING gist (during WITH &&)
+) WITH (fillfactor = 10);
+CREATE INDEX hi_excl_tag ON hi_excl(tag);
+INSERT INTO hi_excl VALUES (1, 100, int4range(1, 10)), (2, 200, int4range(20, 30));
+
+-- Update a non-constrained indexed column: HOT-indexed (GiST exclusion index
+-- and PK skipped), and the exclusion constraint is still enforced.
+UPDATE hi_excl SET tag = tag + 1 WHERE id = 1;
+SELECT hot_idx > 0 AS tag_update_hot_indexed FROM get_hi_count('hi_excl');
+INSERT INTO hi_excl VALUES (3, 300, int4range(5, 15)); -- overlaps id=1's (1,10)
+
+-- Move id=1's range away (this updates the GiST index, leaving a stale entry
+-- for the old (1,10) range). A range overlapping only the OLD range now
+-- inserts cleanly (the stale entry is skipped); one overlapping the NEW range
+-- still conflicts.
+UPDATE hi_excl SET during = int4range(100, 110) WHERE id = 1;
+INSERT INTO hi_excl VALUES (4, 400, int4range(5, 15)); -- only overlapped old range: OK
+INSERT INTO hi_excl VALUES (5, 500, int4range(105, 115));-- overlaps new (100,110): conflict
+DROP TABLE hi_excl;
+
+-- ---------------------------------------------------------------------------
+-- 26. TOAST interaction. An indexed column stored out-of-line must behave
+-- correctly across HOT-indexed updates: an entry kept across an update of a
+-- different column still resolves to the (unchanged) toasted value, and after
+-- the toasted column itself is changed the stale entry is dropped by the
+-- crossed-attribute bitmap (no value comparison or detoasting is needed).
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_toast (id int PRIMARY KEY, big text, tag int) WITH (fillfactor = 50);
+ALTER TABLE hi_toast ALTER COLUMN big SET STORAGE EXTERNAL; -- no compression
+CREATE INDEX hi_toast_big ON hi_toast (big);
+CREATE INDEX hi_toast_tag ON hi_toast (tag);
+INSERT INTO hi_toast VALUES (1, repeat('A', 2000), 10);
+-- The big value is stored out-of-line.
+SELECT pg_column_size(big) > 1500 AS big_is_external FROM hi_toast WHERE id = 1;
+-- HOT-indexed update of tag leaves big (and its index entry) unchanged.
+UPDATE hi_toast SET tag = 11 WHERE id = 1;
+SELECT hot_idx > 0 AS tag_update_hot_indexed FROM get_hi_count('hi_toast');
+SET enable_seqscan = off;
+SET enable_bitmapscan = off;
+SELECT id, tag, length(big) FROM hi_toast WHERE big = repeat('A', 2000);
+-- HOT-indexed update of the toasted indexed column itself: the old entry is
+-- now stale because the crossed-attribute bitmap shows big changed.
+UPDATE hi_toast SET big = repeat('B', 2000) WHERE id = 1;
+SELECT id FROM hi_toast WHERE big = repeat('A', 2000); -- stale: 0 rows
+SELECT id, length(big) FROM hi_toast WHERE big = repeat('B', 2000); -- current
+RESET enable_bitmapscan;
+RESET enable_seqscan;
+SELECT * FROM verify_heapam('hi_toast');
+DROP TABLE hi_toast;
+
+-- ---------------------------------------------------------------------------
+-- 27. ABA on an indexed column. A HOT-indexed update that sets an indexed
+-- column to a value an earlier chain member already held leaves two leaves
+-- with that same key, both chain-resolving to the live tuple. A value-based
+-- recheck cannot tell them apart and would return the row twice; the
+-- crossed-attribute bitmap drops the stale ancestor leaf (its walk crosses the
+-- key-changing hops) and keeps only the fresh entry, so a forced index scan
+-- returns the row exactly once. REINDEX must not change that.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_aba (id int PRIMARY KEY, k int, v int) WITH (fillfactor = 50);
+CREATE INDEX hi_aba_k ON hi_aba (k);
+CREATE INDEX hi_aba_v ON hi_aba (v);
+INSERT INTO hi_aba VALUES (1, 1, 100);
+UPDATE hi_aba SET k = 3 WHERE id = 1; -- HOT-indexed: k changed, v kept
+UPDATE hi_aba SET k = 1 WHERE id = 1; -- HOT-indexed: k cycled back (ABA)
+SET enable_seqscan = off;
+SET enable_bitmapscan = off;
+SELECT count(*) AS k1_once FROM hi_aba WHERE k = 1; -- exactly 1
+SELECT count(*) AS k3_gone FROM hi_aba WHERE k = 3; -- 0 (stale dropped)
+REINDEX INDEX hi_aba_k;
+SELECT count(*) AS k1_after_reindex FROM hi_aba WHERE k = 1; -- still 1
+RESET enable_bitmapscan;
+RESET enable_seqscan;
+SELECT * FROM verify_heapam('hi_aba');
+DROP TABLE hi_aba;
+
+-- ---------------------------------------------------------------------------
+-- 28. Partial index, predicate column changed but the row STAYS in the index
+-- (predicate still true, key unchanged). The update is HOT-indexed; selective
+-- maintenance re-inserts a fresh entry (the predicate column changed and still
+-- holds), so the row is still returned -- the bitmap drops the older entry and
+-- the fresh one re-supplies it. Guards against a "lost row" from over-eager
+-- dropping.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_partstay (id int PRIMARY KEY, k int, n int) WITH (fillfactor = 50);
+CREATE INDEX hi_partstay_k ON hi_partstay (k) WHERE n > 0;
+CREATE INDEX hi_partstay_id2 ON hi_partstay (id);
+INSERT INTO hi_partstay VALUES (1, 5, 3);
+UPDATE hi_partstay SET n = 7 WHERE id = 1; -- n 3->7, still > 0, k unchanged
+SELECT pg_stat_force_next_flush();
+SELECT hot_idx > 0 AS stay_is_hot_indexed FROM get_hi_count('hi_partstay');
+SET enable_seqscan = off;
+SET enable_bitmapscan = off;
+SELECT count(*) AS stay_rows FROM hi_partstay WHERE k = 5 AND n > 0; -- want 1
+RESET enable_bitmapscan;
+RESET enable_seqscan;
+DROP TABLE hi_partstay;
+
+-- ---------------------------------------------------------------------------
+-- 29. Partitioned table. A within-partition UPDATE of one indexed column is
+-- HOT-indexed on the leaf partition's heap exactly as for a non-partitioned
+-- table; a cross-partition update is a delete+insert and never HOT.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_part (id int, a int, b int) PARTITION BY RANGE (id);
+CREATE TABLE hi_part1 PARTITION OF hi_part FOR VALUES FROM (0) TO (100)
+ WITH (fillfactor = 50);
+CREATE INDEX hi_part_a ON hi_part (a);
+CREATE INDEX hi_part_b ON hi_part (b);
+INSERT INTO hi_part VALUES (1, 10, 20);
+UPDATE hi_part SET a = 11 WHERE id = 1; -- one indexed col, within partition
+SELECT pg_stat_force_next_flush();
+SELECT hot_idx > 0 AS part_is_hot_indexed FROM get_hi_count('hi_part1');
+SET enable_seqscan = off;
+SET enable_bitmapscan = off;
+SELECT count(*) AS a11 FROM hi_part WHERE a = 11; -- want 1
+SELECT count(*) AS a10 FROM hi_part WHERE a = 10; -- want 0 (stale dropped)
+RESET enable_bitmapscan;
+RESET enable_seqscan;
+SELECT * FROM verify_heapam('hi_part1');
+DROP TABLE hi_part;
+
+-- ---------------------------------------------------------------------------
+-- 30. Non-btree access method (hash). Read-side staleness is access-method
+-- agnostic (the crossed-attribute bitmap), so any index AM's column is
+-- HOT-indexed. Hash is the sharpest case: its scans recheck the heap value,
+-- which alone cannot disambiguate a value cycled away and back (ABA) -- the
+-- bitmap drops the stale ancestor so the row is returned exactly once.
+-- ---------------------------------------------------------------------------
+CREATE TABLE hi_hash (id int PRIMARY KEY, v int, w int) WITH (fillfactor = 50);
+CREATE INDEX hi_hash_v ON hi_hash USING hash (v);
+CREATE INDEX hi_hash_w ON hi_hash (w);
+INSERT INTO hi_hash VALUES (1, 10, 100);
+UPDATE hi_hash SET v = 99 WHERE id = 1;
+UPDATE hi_hash SET v = 10 WHERE id = 1; -- ABA: 10 -> 99 -> 10, w unchanged
+SELECT pg_stat_force_next_flush();
+SELECT hot_idx > 0 AS hash_is_hot_indexed FROM get_hi_count('hi_hash');
+SET enable_seqscan = off;
+SET enable_bitmapscan = off;
+SELECT count(*) AS hash_v10 FROM hi_hash WHERE v = 10; -- want 1 (no duplicate)
+SELECT count(*) AS hash_v99 FROM hi_hash WHERE v = 99; -- want 0 (stale dropped)
+RESET enable_bitmapscan;
+RESET enable_seqscan;
+DROP TABLE hi_hash;
+
+-- ---------------------------------------------------------------------------
+-- 31. DDL after a HOT-indexed chain exists. The per-hop modified-attrs
+-- bitmap on the page is keyed by physical attribute number and sized by the
+-- relation's natts AT WRITE TIME. Indexes added/dropped after the chain
+-- forms, and ADD/DROP COLUMN, must not corrupt the read-side staleness test.
+-- The sharp case is ADD COLUMN crossing an 8-attribute boundary, which grows
+-- ceil(natts/8): readers must locate each hop's bitmap from that hop's own
+-- write-time natts (HeapTupleHeaderGetNatts / the stub's stashed natts), not
+-- the relation's current natts.
+-- ---------------------------------------------------------------------------
+-- Exactly 8 attributes (c1..c7 + payload) so adding the 9th flips the bitmap
+-- from 1 byte to 2. c7 is the column we churn; c2 is an unchanged indexed
+-- column whose leaf must stay current.
+CREATE TABLE hi_ddl (
+ c1 int PRIMARY KEY, c2 int, c3 int, c4 int,
+ c5 int, c6 int, c7 int, payload text
+) WITH (fillfactor = 50, autovacuum_enabled = false);
+CREATE INDEX hi_ddl_c2 ON hi_ddl(c2);
+CREATE INDEX hi_ddl_c7 ON hi_ddl(c7);
+INSERT INTO hi_ddl VALUES (1, 10, 20, 30, 40, 50, 70, 'p');
+
+-- Form a HOT-indexed chain on c7 BEFORE any further DDL.
+UPDATE hi_ddl SET c7 = 71 WHERE c1 = 1;
+UPDATE hi_ddl SET c7 = 72 WHERE c1 = 1;
+
+-- (a) CREATE INDEX after the chain exists: the new index is built against the
+-- live tuple under its own TID, so its entry is never stale.
+CREATE INDEX hi_ddl_c3 ON hi_ddl(c3);
+
+-- (b) ADD COLUMN crossing the 8-attribute boundary (natts 8 -> 9). Existing
+-- hops keep their 1-byte bitmaps; the relation now wants 2. Reads through the
+-- old chain must still be correct.
+ALTER TABLE hi_ddl ADD COLUMN c9 int;
+CREATE INDEX hi_ddl_c9 ON hi_ddl(c9);
+
+SET enable_seqscan = off;
+SET enable_bitmapscan = off;
+SET enable_indexonlyscan = off;
+
+-- Live c7 is 72. The c7 index must return the live row for 72 and drop the
+-- stale leaves for 70 and 71 (offsets misread would corrupt this).
+SELECT count(*) AS c7_eq_72 FROM hi_ddl WHERE c7 = 72 AND payload IS NOT NULL;
+SELECT count(*) AS c7_eq_70_stale FROM hi_ddl WHERE c7 = 70 AND payload IS NOT NULL;
+SELECT count(*) AS c7_eq_71_stale FROM hi_ddl WHERE c7 = 71 AND payload IS NOT NULL;
+
+-- c2 never changed across the chain: its leaf must NOT be judged stale even
+-- though a crossed hop changed c7. A misread bitmap could spuriously flag it.
+SELECT count(*) AS c2_eq_10_current FROM hi_ddl WHERE c2 = 10 AND payload IS NOT NULL;
+
+-- (c) Continue churning c7 AFTER the ADD COLUMN: the new hop's bitmap is sized
+-- for natts 9 (2 bytes); the old hops are 1 byte. A chain with mixed-size
+-- bitmaps must still resolve correctly.
+UPDATE hi_ddl SET c7 = 73 WHERE c1 = 1;
+SELECT count(*) AS c7_eq_73 FROM hi_ddl WHERE c7 = 73 AND payload IS NOT NULL;
+SELECT count(*) AS c7_eq_72_now_stale FROM hi_ddl WHERE c7 = 72 AND payload IS NOT NULL;
+
+-- (d) Collapse the chain to stubs via VACUUM, then read again: the stub must
+-- preserve its write-time natts so its bitmap stays locatable post-ADD COLUMN.
+UPDATE hi_ddl SET c7 = 74 WHERE c1 = 1;
+VACUUM (INDEX_CLEANUP off) hi_ddl;
+SELECT count(*) AS c7_eq_74_after_vacuum FROM hi_ddl WHERE c7 = 74 AND payload IS NOT NULL;
+SELECT count(*) AS c2_eq_10_after_vacuum FROM hi_ddl WHERE c2 = 10 AND payload IS NOT NULL;
+
+-- (e) DROP COLUMN keeps the attnum slot (no renumber), so bitmaps stay aligned.
+ALTER TABLE hi_ddl DROP COLUMN c4;
+SELECT count(*) AS c7_after_drop FROM hi_ddl WHERE c7 = 74 AND payload IS NOT NULL;
+SELECT count(*) AS c2_after_drop FROM hi_ddl WHERE c2 = 10 AND payload IS NOT NULL;
+
+-- (f) DROP INDEX on the churned column: remaining indexes still resolve.
+DROP INDEX hi_ddl_c7;
+SELECT count(*) AS c2_after_dropidx FROM hi_ddl WHERE c2 = 10 AND payload IS NOT NULL;
+
+RESET enable_seqscan;
+RESET enable_bitmapscan;
+RESET enable_indexonlyscan;
+
+-- The seqscan truth confirms the live row; the count assertions above (read
+-- through the post-DDL indexes) match it, which is what would break if a
+-- mis-sized bitmap corrupted the staleness verdict.
+SELECT c1, c2, c7 FROM hi_ddl WHERE c1 = 1;
+
+DROP TABLE hi_ddl;
+
+-- ---------------------------------------------------------------------------
+-- Cleanup
+-- ---------------------------------------------------------------------------
+DROP FUNCTION get_hi_count(text);
+DROP FUNCTION get_hot_count(text);
+-- pageinspect and amcheck were both created above with IF NOT EXISTS and may
+-- have pre-existed this test; leave them, matching amcheck's treatment,
+-- rather than risk dropping an extension this test did not create.
--
2.50.1
[text/x-patch] v57-0008-Gate-HOT-indexed-updates-on-the-logical-replicat.patch (116.8K, ../[email protected]/9-v57-0008-Gate-HOT-indexed-updates-on-the-logical-replicat.patch)
download | inline diff:
From a216d81a7f2ac669dffab6eca210be7e1360d6a0 Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Wed, 17 Jun 2026 21:43:53 -0400
Subject: [PATCH v57 8/9] Gate HOT-indexed updates on the logical-replication
apply path
A HOT-indexed update of a replica-identity attribute on a subscriber leaves a
stale index leaf that the apply worker's replica-identity lookups must tolerate
-- which they do, but only when the subscriber's indexed attributes do not
extend past the columns those lookups key on. Add the per-subscription
hot_indexed_on_apply option (subhotindexedonapply: off / subset_only (default)
/ always) and have HeapUpdateHotAllowable consult it when running in an apply
worker, comparing the relation's indexed-attribute set against its primary-key
attributes: "off" disqualifies HOT-indexed whenever any indexed attribute lies
outside the primary key, "subset_only" requires the indexed attributes to be a
subset of the primary key, and "always" applies no apply-path gating.
Wire the option through CREATE/ALTER SUBSCRIPTION, pg_subscription, pg_dump,
and psql's \dRs+, and document it (create_subscription, alter_subscription,
catalogs). Cover apply under each mode (039), apply under REPLICA IDENTITY
FULL and a non-PK USING INDEX whose key is cycled (040), and decoding of
HOT-indexed update chains (test_decoding).
Authored-by: Greg Burd <[email protected]>
---
contrib/test_decoding/Makefile | 2 +-
.../test_decoding/expected/hot_indexed.out | 59 +++
contrib/test_decoding/meson.build | 1 +
contrib/test_decoding/sql/hot_indexed.sql | 29 ++
doc/src/sgml/catalogs.sgml | 16 +
doc/src/sgml/ref/alter_subscription.sgml | 7 +-
doc/src/sgml/ref/create_subscription.sgml | 55 +++
src/backend/access/heap/README.HOT-INDEXED | 34 ++
src/backend/access/heap/heapam.c | 30 ++
src/backend/catalog/pg_subscription.c | 1 +
src/backend/catalog/system_views.sql | 1 +
src/backend/commands/subscriptioncmds.c | 43 +-
src/backend/replication/logical/worker.c | 33 ++
src/bin/pg_dump/pg_dump.c | 17 +
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/psql/describe.c | 10 +-
src/include/catalog/catversion.h | 3 +
src/include/catalog/pg_subscription.h | 27 ++
src/include/replication/logicalworker.h | 8 +
src/test/regress/expected/subscription.out | 176 ++++----
src/test/subscription/meson.build | 2 +
.../subscription/t/039_hot_indexed_apply.pl | 416 ++++++++++++++++++
.../t/040_hot_indexed_replica_identity.pl | 110 +++++
23 files changed, 987 insertions(+), 94 deletions(-)
create mode 100644 contrib/test_decoding/expected/hot_indexed.out
create mode 100644 contrib/test_decoding/sql/hot_indexed.sql
create mode 100644 src/test/subscription/t/039_hot_indexed_apply.pl
create mode 100644 src/test/subscription/t/040_hot_indexed_replica_identity.pl
diff --git a/contrib/test_decoding/Makefile b/contrib/test_decoding/Makefile
index 0111124399a..800216b2ae4 100644
--- a/contrib/test_decoding/Makefile
+++ b/contrib/test_decoding/Makefile
@@ -5,7 +5,7 @@ PGFILEDESC = "test_decoding - example of a logical decoding output plugin"
REGRESS = ddl xact rewrite toast permissions decoding_in_xact \
decoding_into_rel binary prepared replorigin time messages \
- repack spill slot truncate stream stats twophase twophase_stream
+ repack spill slot truncate stream stats twophase twophase_stream hot_indexed
ISOLATION = mxact delayed_startup ondisk_startup concurrent_ddl_dml \
oldest_xmin snapshot_transfer subxact_without_top concurrent_stream \
twophase_snapshot slot_creation_error catalog_change_snapshot \
diff --git a/contrib/test_decoding/expected/hot_indexed.out b/contrib/test_decoding/expected/hot_indexed.out
new file mode 100644
index 00000000000..1e2186fda56
--- /dev/null
+++ b/contrib/test_decoding/expected/hot_indexed.out
@@ -0,0 +1,59 @@
+-- Logical decoding of HOT-indexed UPDATEs. A HOT-indexed update is an
+-- ordinary heap update at the WAL level (the new version is logged in full),
+-- so it must decode exactly like any other update. Exercise a chain of
+-- HOT-indexed updates under REPLICA IDENTITY FULL so the decoded old tuple and
+-- new tuple can both be checked.
+SET synchronous_commit = on;
+SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
+ ?column?
+----------
+ init
+(1 row)
+
+CREATE TABLE hi_decode (id int PRIMARY KEY, a int, b int) WITH (fillfactor = 50);
+CREATE INDEX hi_decode_a ON hi_decode (a);
+CREATE INDEX hi_decode_b ON hi_decode (b);
+ALTER TABLE hi_decode REPLICA IDENTITY FULL;
+INSERT INTO hi_decode VALUES (1, 10, 100);
+-- each update changes one indexed column, so each stays HOT-indexed
+UPDATE hi_decode SET a = 11 WHERE id = 1;
+UPDATE hi_decode SET b = 101 WHERE id = 1;
+UPDATE hi_decode SET a = 12 WHERE id = 1;
+-- cycle a away and back (ABA) and then delete
+UPDATE hi_decode SET a = 99 WHERE id = 1;
+UPDATE hi_decode SET a = 12 WHERE id = 1;
+DELETE FROM hi_decode WHERE id = 1;
+SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL,
+ 'include-xids', '0', 'skip-empty-xacts', '1');
+ data
+-------------------------------------------------------------------------------------------------------------------------------------------
+ BEGIN
+ table public.hi_decode: INSERT: id[integer]:1 a[integer]:10 b[integer]:100
+ COMMIT
+ BEGIN
+ table public.hi_decode: UPDATE: old-key: id[integer]:1 a[integer]:10 b[integer]:100 new-tuple: id[integer]:1 a[integer]:11 b[integer]:100
+ COMMIT
+ BEGIN
+ table public.hi_decode: UPDATE: old-key: id[integer]:1 a[integer]:11 b[integer]:100 new-tuple: id[integer]:1 a[integer]:11 b[integer]:101
+ COMMIT
+ BEGIN
+ table public.hi_decode: UPDATE: old-key: id[integer]:1 a[integer]:11 b[integer]:101 new-tuple: id[integer]:1 a[integer]:12 b[integer]:101
+ COMMIT
+ BEGIN
+ table public.hi_decode: UPDATE: old-key: id[integer]:1 a[integer]:12 b[integer]:101 new-tuple: id[integer]:1 a[integer]:99 b[integer]:101
+ COMMIT
+ BEGIN
+ table public.hi_decode: UPDATE: old-key: id[integer]:1 a[integer]:99 b[integer]:101 new-tuple: id[integer]:1 a[integer]:12 b[integer]:101
+ COMMIT
+ BEGIN
+ table public.hi_decode: DELETE: id[integer]:1 a[integer]:12 b[integer]:101
+ COMMIT
+(21 rows)
+
+SELECT pg_drop_replication_slot('regression_slot');
+ pg_drop_replication_slot
+--------------------------
+
+(1 row)
+
+DROP TABLE hi_decode;
diff --git a/contrib/test_decoding/meson.build b/contrib/test_decoding/meson.build
index ac655853d26..91765ca0e72 100644
--- a/contrib/test_decoding/meson.build
+++ b/contrib/test_decoding/meson.build
@@ -42,6 +42,7 @@ tests += {
'stats',
'twophase',
'twophase_stream',
+ 'hot_indexed',
],
'regress_args': [
'--temp-config', files('logical.conf'),
diff --git a/contrib/test_decoding/sql/hot_indexed.sql b/contrib/test_decoding/sql/hot_indexed.sql
new file mode 100644
index 00000000000..05d7d091b62
--- /dev/null
+++ b/contrib/test_decoding/sql/hot_indexed.sql
@@ -0,0 +1,29 @@
+-- Logical decoding of HOT-indexed UPDATEs. A HOT-indexed update is an
+-- ordinary heap update at the WAL level (the new version is logged in full),
+-- so it must decode exactly like any other update. Exercise a chain of
+-- HOT-indexed updates under REPLICA IDENTITY FULL so the decoded old tuple and
+-- new tuple can both be checked.
+SET synchronous_commit = on;
+
+SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
+
+CREATE TABLE hi_decode (id int PRIMARY KEY, a int, b int) WITH (fillfactor = 50);
+CREATE INDEX hi_decode_a ON hi_decode (a);
+CREATE INDEX hi_decode_b ON hi_decode (b);
+ALTER TABLE hi_decode REPLICA IDENTITY FULL;
+
+INSERT INTO hi_decode VALUES (1, 10, 100);
+-- each update changes one indexed column, so each stays HOT-indexed
+UPDATE hi_decode SET a = 11 WHERE id = 1;
+UPDATE hi_decode SET b = 101 WHERE id = 1;
+UPDATE hi_decode SET a = 12 WHERE id = 1;
+-- cycle a away and back (ABA) and then delete
+UPDATE hi_decode SET a = 99 WHERE id = 1;
+UPDATE hi_decode SET a = 12 WHERE id = 1;
+DELETE FROM hi_decode WHERE id = 1;
+
+SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL,
+ 'include-xids', '0', 'skip-empty-xacts', '1');
+
+SELECT pg_drop_replication_slot('regression_slot');
+DROP TABLE hi_decode;
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 4b474c13917..4c9aba72ba7 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -8727,6 +8727,22 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>subhotindexedonapply</structfield> <type>char</type>
+ </para>
+ <para>
+ Gating mode for the HOT-indexed apply path. Corresponds to the
+ <link linkend="sql-createsubscription-params-with-hot-indexed-on-apply"><literal>hot_indexed_on_apply</literal></link>
+ subscription option:
+ <itemizedlist>
+ <listitem><para><literal>o</literal> = off</para></listitem>
+ <listitem><para><literal>s</literal> = subset_only (default)</para></listitem>
+ <listitem><para><literal>a</literal> = always</para></listitem>
+ </itemizedlist>
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>subserver</structfield> <type>oid</type>
diff --git a/doc/src/sgml/ref/alter_subscription.sgml b/doc/src/sgml/ref/alter_subscription.sgml
index fad1f90956a..7ba19180947 100644
--- a/doc/src/sgml/ref/alter_subscription.sgml
+++ b/doc/src/sgml/ref/alter_subscription.sgml
@@ -294,8 +294,11 @@ ALTER SUBSCRIPTION <replaceable class="parameter">name</replaceable> RENAME TO <
<link linkend="sql-createsubscription-params-with-two-phase"><literal>two_phase</literal></link>,
<link linkend="sql-createsubscription-params-with-retain-dead-tuples"><literal>retain_dead_tuples</literal></link>,
<link linkend="sql-createsubscription-params-with-max-retention-duration"><literal>max_retention_duration</literal></link>,
- <link linkend="sql-createsubscription-params-with-wal-receiver-timeout"><literal>wal_receiver_timeout</literal></link>, and
- <link linkend="sql-createsubscription-params-with-conflict-log-destination"><literal>conflict_log_destination</literal></link>.
+ <link linkend="sql-createsubscription-params-with-wal-receiver-timeout"><literal>wal_receiver_timeout</literal></link>,
+ <link linkend="sql-createsubscription-params-with-conflict-log-destination"><literal>conflict_log_destination</literal></link>, and
+ <link linkend="sql-createsubscription-params-with-hot-indexed-on-apply"><literal>hot_indexed_on_apply</literal></link>.
+ For <literal>hot_indexed_on_apply</literal>, the new value takes effect
+ at the apply worker's next catalog reload.
Only a superuser can set <literal>password_required = false</literal>.
</para>
diff --git a/doc/src/sgml/ref/create_subscription.sgml b/doc/src/sgml/ref/create_subscription.sgml
index 81fbf3487a4..0d59f857bb3 100644
--- a/doc/src/sgml/ref/create_subscription.sgml
+++ b/doc/src/sgml/ref/create_subscription.sgml
@@ -651,6 +651,61 @@ CREATE SUBSCRIPTION <replaceable class="parameter">subscription_name</replaceabl
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="sql-createsubscription-params-with-hot-indexed-on-apply">
+ <term><literal>hot_indexed_on_apply</literal> (<type>text</type>)</term>
+ <listitem>
+ <para>
+ Controls whether the subscription's apply worker may take the
+ HOT-indexed update path when an <command>UPDATE</command> replicated
+ from the publisher touches an indexed attribute. Because the
+ subscriber's index set may differ from the publisher's, an
+ unconstrained HOT-indexed decision on the apply path can produce a
+ heap chain whose index state disagrees with the upstream row. The
+ option restricts when the apply worker is allowed to take that path.
+ </para>
+ <para>
+ Accepted values are:
+ <variablelist>
+ <varlistentry>
+ <term><literal>off</literal></term>
+ <listitem>
+ <para>
+ Force non-HOT on apply whenever the subscriber has any indexed
+ attribute beyond the primary key. This matches the conservative
+ pre-existing behaviour.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>subset_only</literal></term>
+ <listitem>
+ <para>
+ Allow the HOT-indexed apply path when the subscriber's
+ indexed-attr set is a subset of its primary-key attrs (which
+ includes the no-secondary-index case). This is the default and
+ captures the common replication-ready schema shape while staying
+ safe when the subscriber adds indexes the publisher does not
+ have.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>always</literal></term>
+ <listitem>
+ <para>
+ Unconditional HOT-indexed eligibility on apply. The operator
+ takes responsibility for keeping the subscriber's indexed-attr
+ set compatible with the publisher's; divergent schemas can
+ produce spurious duplicate-key conflicts for subsequent
+ inserts on the subscriber.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist></para>
</listitem>
</varlistentry>
diff --git a/src/backend/access/heap/README.HOT-INDEXED b/src/backend/access/heap/README.HOT-INDEXED
index 2e5e5f4a081..5561498d95f 100644
--- a/src/backend/access/heap/README.HOT-INDEXED
+++ b/src/backend/access/heap/README.HOT-INDEXED
@@ -257,6 +257,40 @@ per-index recheck outcomes; and pg_relation_hot_indexed_stats() reports
per-relation HOT-indexed chain counts.
+Logical replication
+-------------------
+
+A HOT-indexed update of a replica-identity attribute on a subscriber leaves a
+stale index leaf; the apply worker's replica-identity lookups tolerate that
+only when the indexed attributes are covered by the replica identity. The
+per-subscription hot_indexed_on_apply option (pg_subscription.subhotindexedonapply,
+off / subset_only / always; subset_only is the default) controls this:
+HeapUpdateHotAllowable consults GetHotIndexedApplyMode on the apply path and
+falls back to non-HOT when the indexed attributes are not exactly the primary
+key (off) or not a subset of it (subset_only).
+
+
+Recovery
+--------
+
+HOT-indexed updates and the prune/collapse use the existing heap UPDATE and
+prune/freeze WAL records, so crash recovery replays them with no new record
+types. src/test/recovery/t/054_hot_indexed_recovery.pl builds a chain,
+crashes without a checkpoint (forcing WAL redo), and verifies the chain walk,
+verify_heapam, and vacuum reclamation after restart, with
+wal_consistency_checking = 'all' comparing each replayed page to its FPI.
+
+
+Adversarial tests
+-----------------
+
+src/test/isolation/specs/hot_indexed_adversarial.spec exercises the cases the
+invariant must satisfy under concurrency: key cycling (X->Y->X), aborted
+HOT-indexed updates, concurrent unique inserts against a freed/taken key,
+snapshot-safe reclaim of stale leaves, and reader consistency across a
+concurrent prune/collapse.
+
+
Appendices
----------
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 3bfac8c0b18..3dfc619cc48 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -4504,6 +4504,36 @@ HeapUpdateHotAllowable(Relation relation, const Bitmapset *modified_idx_attrs)
all_idx_attrs = RelationGetIndexAttrBitmapNoCopy(relation,
INDEX_ATTR_BITMAP_INDEXED);
+ /*
+ * The logical-replication apply path gates HOT-indexed updates on the
+ * per-subscription hot_indexed_on_apply option. A HOT-indexed update of
+ * a replica-identity attribute leaves a stale index leaf; the apply
+ * worker's replica-identity lookups cope with that (see
+ * RelationFindReplTupleByIndex), but only when the indexed attributes are
+ * a subset of the replica identity. "off" disqualifies whenever the
+ * subscriber has any indexed attribute beyond its PK; "subset_only" (the
+ * default) requires the indexed attributes to be a subset of the PK;
+ * "always" applies no apply-path gating.
+ */
+ if (IsLogicalWorker())
+ {
+ char mode = GetHotIndexedApplyMode();
+ const Bitmapset *pk_attrs = RelationGetIndexAttrBitmapNoCopy(relation,
+ INDEX_ATTR_BITMAP_PRIMARY_KEY);
+
+ if (mode == LOGICALREP_HOT_INDEXED_OFF)
+ {
+ if (!bms_equal(all_idx_attrs, pk_attrs))
+ return HEAP_UPDATE_ALL_INDEXES;
+ }
+ else if (mode == LOGICALREP_HOT_INDEXED_SUBSET_ONLY)
+ {
+ if (!bms_is_subset(all_idx_attrs, pk_attrs))
+ return HEAP_UPDATE_ALL_INDEXES;
+ }
+ /* LOGICALREP_HOT_INDEXED_ALWAYS: no apply-path gating. */
+ }
+
/*
* System catalogs keep classic HOT (an UPDATE touching no non-summarizing
* indexed attribute already returned HEAP_HEAP_ONLY_UPDATE above), but do
diff --git a/src/backend/catalog/pg_subscription.c b/src/backend/catalog/pg_subscription.c
index 2068e03c571..b065a7be249 100644
--- a/src/backend/catalog/pg_subscription.c
+++ b/src/backend/catalog/pg_subscription.c
@@ -133,6 +133,7 @@ GetSubscription(Oid subid, bool missing_ok, bool conninfo_needed,
sub->maxretention = subform->submaxretention;
sub->retentionactive = subform->subretentionactive;
sub->conflictlogrelid = subform->subconflictlogrelid;
+ sub->hotindexedonapply = subform->subhotindexedonapply;
if (conninfo_needed)
{
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 02c8a049a32..9f545fd2647 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1542,6 +1542,7 @@ GRANT SELECT (oid, subdbid, subskiplsn, subname, subowner, subenabled,
subbinary, substream, subtwophasestate, subdisableonerr,
subpasswordrequired, subrunasowner, subfailover,
subretaindeadtuples, submaxretention, subretentionactive,
+ subhotindexedonapply,
subserver, subconflictlogrelid, subconflictlogdest, subslotname,
subsynccommit, subwalrcvtimeout, subpublications, suborigin)
ON pg_subscription TO public;
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 4292e7fb8f4..bf9dba3f4e4 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -82,6 +82,7 @@
#define SUBOPT_LSN 0x00020000
#define SUBOPT_ORIGIN 0x00040000
#define SUBOPT_CONFLICT_LOG_DEST 0x00080000
+#define SUBOPT_HOT_INDEXED_ON_APPLY 0x00100000
/* check if the 'val' has 'bits' set */
#define IsSet(val, bits) (((val) & (bits)) == (bits))
@@ -113,6 +114,7 @@ typedef struct SubOpts
ConflictLogDest conflictlogdest;
XLogRecPtr lsn;
char *wal_receiver_timeout;
+ char hotindexedonapply;
} SubOpts;
/*
@@ -207,6 +209,8 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
opts->origin = pstrdup(LOGICALREP_ORIGIN_ANY);
if (IsSet(supported_opts, SUBOPT_CONFLICT_LOG_DEST))
opts->conflictlogdest = CONFLICT_LOG_DEST_LOG;
+ if (IsSet(supported_opts, SUBOPT_HOT_INDEXED_ON_APPLY))
+ opts->hotindexedonapply = LOGICALREP_HOT_INDEXED_SUBSET_ONLY;
/* Parse options */
foreach(lc, stmt_options)
@@ -459,6 +463,30 @@ parse_subscription_options(ParseState *pstate, List *stmt_options,
opts->conflictlogdest = GetConflictLogDest(val);
opts->specified_opts |= SUBOPT_CONFLICT_LOG_DEST;
}
+ else if (IsSet(supported_opts, SUBOPT_HOT_INDEXED_ON_APPLY) &&
+ strcmp(defel->defname, "hot_indexed_on_apply") == 0)
+ {
+ char *val;
+
+ if (IsSet(opts->specified_opts, SUBOPT_HOT_INDEXED_ON_APPLY))
+ errorConflictingDefElem(defel, pstate);
+
+ opts->specified_opts |= SUBOPT_HOT_INDEXED_ON_APPLY;
+ val = defGetString(defel);
+
+ if (pg_strcasecmp(val, "off") == 0)
+ opts->hotindexedonapply = LOGICALREP_HOT_INDEXED_OFF;
+ else if (pg_strcasecmp(val, "subset_only") == 0)
+ opts->hotindexedonapply = LOGICALREP_HOT_INDEXED_SUBSET_ONLY;
+ else if (pg_strcasecmp(val, "always") == 0)
+ opts->hotindexedonapply = LOGICALREP_HOT_INDEXED_ALWAYS;
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unrecognized value for subscription parameter \"%s\": \"%s\"",
+ "hot_indexed_on_apply", val),
+ errhint("Valid values are \"off\", \"subset_only\", and \"always\".")));
+ }
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -699,7 +727,8 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
SUBOPT_RETAIN_DEAD_TUPLES |
SUBOPT_MAX_RETENTION_DURATION |
SUBOPT_WAL_RECEIVER_TIMEOUT | SUBOPT_ORIGIN |
- SUBOPT_CONFLICT_LOG_DEST);
+ SUBOPT_CONFLICT_LOG_DEST |
+ SUBOPT_HOT_INDEXED_ON_APPLY);
parse_subscription_options(pstate, stmt->options, supported_opts, &opts);
/*
@@ -853,6 +882,8 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
Int32GetDatum(opts.maxretention);
values[Anum_pg_subscription_subretentionactive - 1] =
BoolGetDatum(opts.retaindeadtuples);
+ values[Anum_pg_subscription_subhotindexedonapply - 1] =
+ CharGetDatum(opts.hotindexedonapply);
values[Anum_pg_subscription_subserver - 1] = ObjectIdGetDatum(serverid);
if (!OidIsValid(serverid))
values[Anum_pg_subscription_subconninfo - 1] =
@@ -1624,7 +1655,8 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
SUBOPT_MAX_RETENTION_DURATION |
SUBOPT_WAL_RECEIVER_TIMEOUT |
SUBOPT_ORIGIN |
- SUBOPT_CONFLICT_LOG_DEST);
+ SUBOPT_CONFLICT_LOG_DEST |
+ SUBOPT_HOT_INDEXED_ON_APPLY);
break;
case ALTER_SUBSCRIPTION_ENABLED:
@@ -2011,6 +2043,13 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
}
}
+ if (IsSet(opts.specified_opts, SUBOPT_HOT_INDEXED_ON_APPLY))
+ {
+ values[Anum_pg_subscription_subhotindexedonapply - 1] =
+ CharGetDatum(opts.hotindexedonapply);
+ replaces[Anum_pg_subscription_subhotindexedonapply - 1] = true;
+ }
+
update_tuple = true;
break;
}
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 7799266c614..4e2a2ac98ec 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -484,6 +484,14 @@ WalReceiverConn *LogRepWorkerWalRcvConn = NULL;
Subscription *MySubscription = NULL;
static bool MySubscriptionValid = false;
+/*
+ * Cache of the per-subscription hot_indexed_on_apply mode. The apply worker
+ * refreshes this after every successful load of MySubscription; readers
+ * outside worker.c go through GetHotIndexedApplyMode() so they don't need
+ * visibility into the Subscription struct or the apply worker's globals.
+ */
+static char hot_indexed_apply_mode = LOGICALREP_HOT_INDEXED_OFF;
+
static List *on_commit_wakeup_workers_subids = NIL;
bool in_remote_transaction = false;
@@ -5171,6 +5179,9 @@ maybe_reread_subscription(void)
MemoryContextDelete(MySubscription->cxt);
MySubscription = newsub;
+ /* Refresh the cached HOT-indexed apply mode from the new tuple. */
+ hot_indexed_apply_mode = MySubscription->hotindexedonapply;
+
/* Change synchronous commit according to the user's wishes */
SetConfigOption("synchronous_commit", MySubscription->synccommit,
PGC_BACKEND, PGC_S_OVERRIDE);
@@ -5844,6 +5855,12 @@ InitializeLogRepWorker(void)
MySubscriptionValid = true;
+ /*
+ * Cache the subscription's HOT-indexed apply mode so it is cheap to
+ * consult from the heap access method (via GetHotIndexedApplyMode()).
+ */
+ hot_indexed_apply_mode = MySubscription->hotindexedonapply;
+
if (!MySubscription->enabled)
{
ereport(LOG,
@@ -6083,6 +6100,22 @@ IsLogicalWorker(void)
return MyLogicalRepWorker != NULL;
}
+/*
+ * Return the cached HOT-indexed apply mode of the current logical replication
+ * worker's subscription.
+ *
+ * Callers outside worker.c (notably heapam.c's HeapUpdateHotAllowable) use
+ * this accessor to avoid pulling in worker_internal.h or the Subscription
+ * struct. Non-apply processes get LOGICALREP_HOT_INDEXED_OFF, which is the
+ * conservative value; callers are expected to guard with IsLogicalWorker()
+ * first for clarity, but the accessor is safe either way.
+ */
+char
+GetHotIndexedApplyMode(void)
+{
+ return hot_indexed_apply_mode;
+}
+
/*
* Is current process a logical replication parallel apply worker?
*/
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 4d660d14b4c..b754af17cd2 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -5138,6 +5138,7 @@ getSubscriptions(Archive *fout)
int i_subfailover;
int i_subretaindeadtuples;
int i_submaxretention;
+ int i_subhotindexedonapply;
int i,
ntups;
@@ -5235,6 +5236,14 @@ getSubscriptions(Archive *fout)
appendPQExpBufferStr(query,
" '-1' AS subwalrcvtimeout,\n");
+ if (fout->remoteVersion >= 190000)
+ appendPQExpBufferStr(query,
+ " s.subhotindexedonapply,\n");
+ else
+ appendPQExpBuffer(query,
+ " '%c' AS subhotindexedonapply,\n",
+ LOGICALREP_HOT_INDEXED_SUBSET_ONLY);
+
if (fout->remoteVersion >= 190000)
appendPQExpBufferStr(query, " fs.srvname AS subservername\n");
else
@@ -5279,6 +5288,7 @@ getSubscriptions(Archive *fout)
i_subfailover = PQfnumber(res, "subfailover");
i_subretaindeadtuples = PQfnumber(res, "subretaindeadtuples");
i_submaxretention = PQfnumber(res, "submaxretention");
+ i_subhotindexedonapply = PQfnumber(res, "subhotindexedonapply");
i_subservername = PQfnumber(res, "subservername");
i_subconninfo = PQfnumber(res, "subconninfo");
i_subslotname = PQfnumber(res, "subslotname");
@@ -5322,6 +5332,8 @@ getSubscriptions(Archive *fout)
(strcmp(PQgetvalue(res, i, i_subretaindeadtuples), "t") == 0);
subinfo[i].submaxretention =
atoi(PQgetvalue(res, i, i_submaxretention));
+ subinfo[i].subhotindexedonapply =
+ *(PQgetvalue(res, i, i_subhotindexedonapply));
if (PQgetisnull(res, i, i_subconninfo))
subinfo[i].subconninfo = NULL;
else
@@ -5599,6 +5611,11 @@ dumpSubscription(Archive *fout, const SubscriptionInfo *subinfo)
if (subinfo->submaxretention)
appendPQExpBuffer(query, ", max_retention_duration = %d", subinfo->submaxretention);
+ if (subinfo->subhotindexedonapply == LOGICALREP_HOT_INDEXED_OFF)
+ appendPQExpBufferStr(query, ", hot_indexed_on_apply = off");
+ else if (subinfo->subhotindexedonapply == LOGICALREP_HOT_INDEXED_ALWAYS)
+ appendPQExpBufferStr(query, ", hot_indexed_on_apply = always");
+
if (strcmp(subinfo->subsynccommit, "off") != 0)
appendPQExpBuffer(query, ", synchronous_commit = %s", fmtId(subinfo->subsynccommit));
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 5a6726d8b12..e3e7401df08 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -722,6 +722,7 @@ typedef struct _SubscriptionInfo
bool subfailover;
bool subretaindeadtuples;
int submaxretention;
+ char subhotindexedonapply;
char *subservername;
char *subconninfo;
char *subslotname;
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index fef86b4cca3..ab3da64cbcb 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6870,7 +6870,7 @@ describeSubscriptions(const char *pattern, bool verbose)
printQueryOpt myopt = pset.popt;
static const bool translate_columns[] = {false, false, false, false,
false, false, false, false, false, false, false, false, false, false,
- false, false, false, false, false, false, false};
+ false, false, false, false, false, false, false, false};
initPQExpBuffer(&buf);
@@ -6943,6 +6943,14 @@ describeSubscriptions(const char *pattern, bool verbose)
", submaxretention AS \"%s\"\n",
gettext_noop("Max retention duration"));
+ appendPQExpBuffer(&buf,
+ ", (CASE subhotindexedonapply\n"
+ " WHEN " CppAsString2(LOGICALREP_HOT_INDEXED_OFF) " THEN 'off'\n"
+ " WHEN " CppAsString2(LOGICALREP_HOT_INDEXED_SUBSET_ONLY) " THEN 'subset_only'\n"
+ " WHEN " CppAsString2(LOGICALREP_HOT_INDEXED_ALWAYS) " THEN 'always'\n"
+ " END) AS \"%s\"\n",
+ gettext_noop("HOT-indexed on apply"));
+
appendPQExpBuffer(&buf,
", subretentionactive AS \"%s\"\n",
gettext_noop("Retention active"));
diff --git a/src/include/catalog/catversion.h b/src/include/catalog/catversion.h
index be36ca97903..c2ea0adcbd6 100644
--- a/src/include/catalog/catversion.h
+++ b/src/include/catalog/catversion.h
@@ -57,6 +57,9 @@
*/
/* yyyymmddN */
+/* XXX bump catversion -- this commit adds a catalog column
+ * (subhotindexedonapply) and touches pg_subscription; the committer
+ * sets the real value at commit time to avoid needless rebase churn. */
#define CATALOG_VERSION_NO 202607072
#endif
diff --git a/src/include/catalog/pg_subscription.h b/src/include/catalog/pg_subscription.h
index 65ce8e145fb..9be556b884f 100644
--- a/src/include/catalog/pg_subscription.h
+++ b/src/include/catalog/pg_subscription.h
@@ -92,6 +92,10 @@ CATALOG(pg_subscription,6100,SubscriptionRelationId) BKI_SHARED_RELATION BKI_ROW
* exceeded max_retention_duration, when
* defined */
+ char subhotindexedonapply; /* Per-subscription gating of the HOT-
+ * indexed apply path. See
+ * LOGICALREP_HOT_INDEXED_* constants. */
+
Oid subserver BKI_LOOKUP_OPT(pg_foreign_server); /* If connection uses
* server */
@@ -173,6 +177,9 @@ typedef struct Subscription
* exceeded max_retention_duration, when
* defined */
Oid conflictlogrelid; /* conflict log table Oid */
+ char hotindexedonapply; /* Per-subscription gating of the
+ * HOT-indexed apply path. See
+ * LOGICALREP_HOT_INDEXED_* constants. */
char *conninfo; /* Connection string to the publisher */
char *slotname; /* Name of the replication slot */
char *synccommit; /* Synchronous commit setting for worker */
@@ -220,6 +227,26 @@ typedef struct Subscription
*/
#define LOGICALREP_STREAM_PARALLEL 'p'
+/*
+ * Per-subscription gating of the HOT-indexed apply path. Recorded as a
+ * single-character code in pg_subscription.subhotindexedonapply.
+ *
+ * 'o' -- OFF: force non-HOT on apply whenever the subscriber carries any
+ * indexed attribute beyond the primary key. Matches the conservative
+ * behaviour before this option was introduced.
+ * 's' -- SUBSET_ONLY (default for freshly created subscriptions): allow the
+ * HOT-indexed apply path when the subscriber's full indexed-attr set is
+ * a subset of its primary-key attrs (which covers the no-secondary-
+ * index case as well). Safe on matching schemas; falls back to non-HOT
+ * when the subscriber adds indexes beyond the primary key.
+ * 'a' -- ALWAYS: unconditional HOT-indexed eligibility on apply. The
+ * operator accepts responsibility for keeping subscriber and publisher
+ * indexed-attr sets compatible.
+ */
+#define LOGICALREP_HOT_INDEXED_OFF 'o'
+#define LOGICALREP_HOT_INDEXED_SUBSET_ONLY 's'
+#define LOGICALREP_HOT_INDEXED_ALWAYS 'a'
+
#endif /* EXPOSE_TO_CLIENT_CODE */
extern Subscription *GetSubscription(Oid subid, bool missing_ok,
diff --git a/src/include/replication/logicalworker.h b/src/include/replication/logicalworker.h
index 7d748a28da8..c9df7d32f2d 100644
--- a/src/include/replication/logicalworker.h
+++ b/src/include/replication/logicalworker.h
@@ -24,6 +24,14 @@ extern void SequenceSyncWorkerMain(Datum main_arg);
extern bool IsLogicalWorker(void);
extern bool IsLogicalParallelApplyWorker(void);
+/*
+ * Accessor for the cached hot_indexed_on_apply mode of the current apply
+ * worker's subscription. Returns a LOGICALREP_HOT_INDEXED_* code (see
+ * catalog/pg_subscription.h). Non-apply processes always see
+ * LOGICALREP_HOT_INDEXED_OFF.
+ */
+extern char GetHotIndexedApplyMode(void);
+
extern void HandleParallelApplyMessageInterrupt(void);
extern void ProcessParallelApplyMessages(void);
diff --git a/src/test/regress/expected/subscription.out b/src/test/regress/expected/subscription.out
index d201ad764f0..cc1d9bbb146 100644
--- a/src/test/regress/expected/subscription.out
+++ b/src/test/regress/expected/subscription.out
@@ -139,18 +139,18 @@ CREATE SUBSCRIPTION regress_testsub4 CONNECTION 'dbname=regress_doesnotexist' PU
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and alter the subscription to refresh publications.
\dRs+ regress_testsub4
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
-------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
- regress_testsub4 | regress_subscription_user | f | {testpub} | f | parallel | d | f | none | t | f | f | | f | 0 | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | HOT-indexed on apply | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
+------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+----------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
+ regress_testsub4 | regress_subscription_user | f | {testpub} | f | parallel | d | f | none | t | f | f | | f | 0 | subset_only | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
(1 row)
ALTER SUBSCRIPTION regress_testsub4 SET (origin = any);
\dRs+ regress_testsub4
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
-------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
- regress_testsub4 | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | | f | 0 | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | HOT-indexed on apply | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
+------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+----------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
+ regress_testsub4 | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | | f | 0 | subset_only | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
(1 row)
DROP SUBSCRIPTION regress_testsub3;
@@ -237,10 +237,10 @@ ALTER SUBSCRIPTION regress_testsub CONNECTION 'foobar';
ERROR: invalid connection string syntax: missing "=" after "foobar" in connection info string
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------------
- regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | | f | 0 | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 | test subscription
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | HOT-indexed on apply | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+----------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | | f | 0 | subset_only | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 | test subscription
(1 row)
ALTER SUBSCRIPTION regress_testsub SET PUBLICATION testpub2, testpub3 WITH (refresh = false);
@@ -249,10 +249,10 @@ ALTER SUBSCRIPTION regress_testsub SET (slot_name = 'newname');
ALTER SUBSCRIPTION regress_testsub SET (password_required = false);
ALTER SUBSCRIPTION regress_testsub SET (run_as_owner = true);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+------------------+--------------------+------------------------------+------------------+------------+-------------------
- regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | parallel | d | f | any | f | t | f | | f | 0 | f | off | dbname=regress_doesnotexist2 | -1 | 0/00000000 | test subscription
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | HOT-indexed on apply | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
+-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+----------------------+------------------+--------------------+------------------------------+------------------+------------+-------------------
+ regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | parallel | d | f | any | f | t | f | | f | 0 | subset_only | f | off | dbname=regress_doesnotexist2 | -1 | 0/00000000 | test subscription
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (password_required = true);
@@ -268,10 +268,10 @@ ERROR: unrecognized subscription parameter: "create_slot"
-- ok
ALTER SUBSCRIPTION regress_testsub SKIP (lsn = '0/12345');
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+------------------+--------------------+------------------------------+------------------+------------+-------------------
- regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | parallel | d | f | any | t | f | f | | f | 0 | f | off | dbname=regress_doesnotexist2 | -1 | 0/00012345 | test subscription
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | HOT-indexed on apply | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
+-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+----------------------+------------------+--------------------+------------------------------+------------------+------------+-------------------
+ regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | parallel | d | f | any | t | f | f | | f | 0 | subset_only | f | off | dbname=regress_doesnotexist2 | -1 | 0/00012345 | test subscription
(1 row)
-- ok - with lsn = NONE
@@ -280,10 +280,10 @@ ALTER SUBSCRIPTION regress_testsub SKIP (lsn = NONE);
ALTER SUBSCRIPTION regress_testsub SKIP (lsn = '0/0');
ERROR: invalid WAL location (LSN): 0/0
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+------------------+--------------------+------------------------------+------------------+------------+-------------------
- regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | parallel | d | f | any | t | f | f | | f | 0 | f | off | dbname=regress_doesnotexist2 | -1 | 0/00000000 | test subscription
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | HOT-indexed on apply | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
+-----------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+----------------------+------------------+--------------------+------------------------------+------------------+------------+-------------------
+ regress_testsub | regress_subscription_user | f | {testpub2,testpub3} | f | parallel | d | f | any | t | f | f | | f | 0 | subset_only | f | off | dbname=regress_doesnotexist2 | -1 | 0/00000000 | test subscription
(1 row)
BEGIN;
@@ -319,10 +319,10 @@ ALTER SUBSCRIPTION regress_testsub_foo SET (wal_receiver_timeout = '80s');
ALTER SUBSCRIPTION regress_testsub_foo SET (wal_receiver_timeout = 'foobar');
ERROR: invalid value for parameter "wal_receiver_timeout": "foobar"
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
----------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+------------------+--------------------+------------------------------+------------------+------------+-------------------
- regress_testsub_foo | regress_subscription_user | f | {testpub2,testpub3} | f | parallel | d | f | any | t | f | f | | f | 0 | f | local | dbname=regress_doesnotexist2 | 80s | 0/00000000 | test subscription
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | HOT-indexed on apply | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
+---------------------+---------------------------+---------+---------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+----------------------+------------------+--------------------+------------------------------+------------------+------------+-------------------
+ regress_testsub_foo | regress_subscription_user | f | {testpub2,testpub3} | f | parallel | d | f | any | t | f | f | | f | 0 | subset_only | f | local | dbname=regress_doesnotexist2 | 80s | 0/00000000 | test subscription
(1 row)
-- rename back to keep the rest simple
@@ -351,19 +351,19 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and alter the subscription to refresh publications.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
- regress_testsub | regress_subscription_user | f | {testpub} | t | parallel | d | f | any | t | f | f | | f | 0 | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | HOT-indexed on apply | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+----------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
+ regress_testsub | regress_subscription_user | f | {testpub} | t | parallel | d | f | any | t | f | f | | f | 0 | subset_only | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (binary = false);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
- regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | | f | 0 | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | HOT-indexed on apply | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+----------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | | f | 0 | subset_only | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
(1 row)
DROP SUBSCRIPTION regress_testsub;
@@ -375,27 +375,27 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and alter the subscription to refresh publications.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | d | f | any | t | f | f | | f | 0 | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | HOT-indexed on apply | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+----------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | d | f | any | t | f | f | | f | 0 | subset_only | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (streaming = parallel);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
- regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | | f | 0 | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | HOT-indexed on apply | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+----------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | | f | 0 | subset_only | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (streaming = false);
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | | f | 0 | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | HOT-indexed on apply | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+----------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | | f | 0 | subset_only | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
(1 row)
-- fail - publication already exists
@@ -410,10 +410,10 @@ ALTER SUBSCRIPTION regress_testsub ADD PUBLICATION testpub1, testpub2 WITH (refr
ALTER SUBSCRIPTION regress_testsub ADD PUBLICATION testpub1, testpub2 WITH (refresh = false);
ERROR: publication "testpub1" is already in subscription "regress_testsub"
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
------------------+---------------------------+---------+-----------------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
- regress_testsub | regress_subscription_user | f | {testpub,testpub1,testpub2} | f | off | d | f | any | t | f | f | | f | 0 | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | HOT-indexed on apply | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
+-----------------+---------------------------+---------+-----------------------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+----------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
+ regress_testsub | regress_subscription_user | f | {testpub,testpub1,testpub2} | f | off | d | f | any | t | f | f | | f | 0 | subset_only | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
(1 row)
-- fail - publication used more than once
@@ -428,10 +428,10 @@ ERROR: publication "testpub3" is not in subscription "regress_testsub"
-- ok - delete publications
ALTER SUBSCRIPTION regress_testsub DROP PUBLICATION testpub1, testpub2 WITH (refresh = false);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
- regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | | f | 0 | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | HOT-indexed on apply | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+----------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | off | d | f | any | t | f | f | | f | 0 | subset_only | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
(1 row)
DROP SUBSCRIPTION regress_testsub;
@@ -467,19 +467,19 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and alter the subscription to refresh publications.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
- regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | p | f | any | t | f | f | | f | 0 | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | HOT-indexed on apply | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+----------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | p | f | any | t | f | f | | f | 0 | subset_only | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
(1 row)
-- we can alter streaming when two_phase enabled
ALTER SUBSCRIPTION regress_testsub SET (streaming = true);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | t | f | f | | f | 0 | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | HOT-indexed on apply | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+----------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | t | f | f | | f | 0 | subset_only | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
@@ -489,10 +489,10 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and alter the subscription to refresh publications.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
- regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | t | f | f | | f | 0 | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | HOT-indexed on apply | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+----------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | on | p | f | any | t | f | f | | f | 0 | subset_only | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
@@ -505,18 +505,18 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and alter the subscription to refresh publications.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
- regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | | f | 0 | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | HOT-indexed on apply | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+----------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | | f | 0 | subset_only | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (disable_on_error = true);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
- regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | t | any | t | f | f | | f | 0 | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | HOT-indexed on apply | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+----------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | t | any | t | f | f | | f | 0 | subset_only | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
@@ -529,10 +529,10 @@ CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUB
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and alter the subscription to refresh publications.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
- regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | | f | 0 | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | HOT-indexed on apply | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+----------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | | f | 0 | subset_only | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
@@ -549,10 +549,10 @@ NOTICE: max_retention_duration is ineffective when retain_dead_tuples is disabl
WARNING: subscription was created, but is not connected
HINT: To initiate replication, you must manually create the replication slot, enable the subscription, and alter the subscription to refresh publications.
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
- regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | | f | 1000 | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | HOT-indexed on apply | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+----------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | | f | 1000 | subset_only | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
(1 row)
-- fail - max_retention_duration must be non-negative
@@ -561,10 +561,10 @@ ERROR: max_retention_duration cannot be negative
-- ok
ALTER SUBSCRIPTION regress_testsub SET (max_retention_duration = 0);
\dRs+
- List of subscriptions
- Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
------------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
- regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | | f | 0 | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
+ List of subscriptions
+ Name | Owner | Enabled | Publication | Binary | Streaming | Two-phase commit | Disable on error | Origin | Password required | Run as owner? | Failover | Server | Retain dead tuples | Max retention duration | HOT-indexed on apply | Retention active | Synchronous commit | Conninfo | Receiver timeout | Skip LSN | Description
+-----------------+---------------------------+---------+-------------+--------+-----------+------------------+------------------+--------+-------------------+---------------+----------+--------+--------------------+------------------------+----------------------+------------------+--------------------+-----------------------------+------------------+------------+-------------
+ regress_testsub | regress_subscription_user | f | {testpub} | f | parallel | d | f | any | t | f | f | | f | 0 | subset_only | f | off | dbname=regress_doesnotexist | -1 | 0/00000000 |
(1 row)
ALTER SUBSCRIPTION regress_testsub SET (slot_name = NONE);
diff --git a/src/test/subscription/meson.build b/src/test/subscription/meson.build
index e71e95c6297..f7ee5b8449a 100644
--- a/src/test/subscription/meson.build
+++ b/src/test/subscription/meson.build
@@ -48,6 +48,8 @@ tests += {
't/036_sequences.pl',
't/037_except.pl',
't/038_walsnd_shutdown_timeout.pl',
+ 't/039_hot_indexed_apply.pl',
+ 't/040_hot_indexed_replica_identity.pl',
't/100_bugs.pl',
],
},
diff --git a/src/test/subscription/t/039_hot_indexed_apply.pl b/src/test/subscription/t/039_hot_indexed_apply.pl
new file mode 100644
index 00000000000..fe108e6f081
--- /dev/null
+++ b/src/test/subscription/t/039_hot_indexed_apply.pl
@@ -0,0 +1,416 @@
+
+# Copyright (c) 2026, PostgreSQL Global Development Group
+
+# Per-subscription hot_indexed_on_apply option: parser, catalog round-trip,
+# ALTER behaviour, and apply-path gating under each of the three modes.
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+use Time::HiRes qw(usleep);
+
+my $publisher = PostgreSQL::Test::Cluster->new('publisher');
+$publisher->init(allows_streaming => 'logical');
+$publisher->start;
+
+my $subscriber = PostgreSQL::Test::Cluster->new('subscriber');
+$subscriber->init;
+$subscriber->start;
+
+my $pub_conninfo = $publisher->connstr . ' dbname=postgres';
+
+# --- Schema ----------------------------------------------------------------
+# tab_extra has an extra btree index beyond the primary key on the
+# subscriber side; that is the schema shape that subset_only must demote
+# to non-HOT on apply but always must let through.
+$publisher->safe_psql('postgres',
+ q{CREATE TABLE tab_extra (id int PRIMARY KEY, payload int, tag text)});
+
+# tab_pk has only the primary key; indexed-attr set is a subset of the PK
+# attrs, so subset_only and always should both allow HOT-indexed on apply.
+$publisher->safe_psql('postgres',
+ q{CREATE TABLE tab_pk (id int PRIMARY KEY, payload int)});
+
+$publisher->safe_psql('postgres',
+ q{CREATE PUBLICATION pub FOR TABLE tab_extra, tab_pk});
+
+# Subscriber mirrors both tables. tab_extra has the extra secondary index
+# only on the subscriber, which is the schema-divergence case the option
+# gates.
+$subscriber->safe_psql('postgres',
+ q{CREATE TABLE tab_extra (id int PRIMARY KEY, payload int, tag text)});
+$subscriber->safe_psql('postgres',
+ q{CREATE INDEX tab_extra_payload_idx ON tab_extra(payload)});
+$subscriber->safe_psql('postgres',
+ q{CREATE TABLE tab_pk (id int PRIMARY KEY, payload int)});
+
+# --- Parser / catalog checks ----------------------------------------------
+# Default on fresh subscription is 's' (subset_only).
+$subscriber->safe_psql('postgres', qq{
+ CREATE SUBSCRIPTION sub_default
+ CONNECTION '$pub_conninfo'
+ PUBLICATION pub
+ WITH (connect = false, slot_name = NONE, enabled = false,
+ create_slot = false);
+});
+is( $subscriber->safe_psql('postgres',
+ q{SELECT subhotindexedonapply FROM pg_subscription
+ WHERE subname = 'sub_default'}),
+ 's',
+ 'fresh subscription defaults to subset_only');
+
+# Explicit 'always' is stored as 'a'.
+$subscriber->safe_psql('postgres', qq{
+ CREATE SUBSCRIPTION sub_always_p
+ CONNECTION '$pub_conninfo'
+ PUBLICATION pub
+ WITH (connect = false, slot_name = NONE, enabled = false,
+ create_slot = false, hot_indexed_on_apply = 'always');
+});
+is( $subscriber->safe_psql('postgres',
+ q{SELECT subhotindexedonapply FROM pg_subscription
+ WHERE subname = 'sub_always_p'}),
+ 'a',
+ 'CREATE with hot_indexed_on_apply = always stores a');
+
+# ALTER SUBSCRIPTION SET updates the column.
+$subscriber->safe_psql('postgres',
+ q{ALTER SUBSCRIPTION sub_default SET (hot_indexed_on_apply = 'off')});
+is( $subscriber->safe_psql('postgres',
+ q{SELECT subhotindexedonapply FROM pg_subscription
+ WHERE subname = 'sub_default'}),
+ 'o',
+ 'ALTER SUBSCRIPTION SET hot_indexed_on_apply = off stores o');
+
+# Unknown values are rejected.
+my ($ret, $stdout, $stderr) = $subscriber->psql('postgres', qq{
+ CREATE SUBSCRIPTION sub_bogus
+ CONNECTION '$pub_conninfo'
+ PUBLICATION pub
+ WITH (connect = false, slot_name = NONE, enabled = false,
+ create_slot = false, hot_indexed_on_apply = 'bogus');
+});
+isnt($ret, 0, 'bogus hot_indexed_on_apply value is rejected');
+like($stderr,
+ qr/unrecognized value for subscription parameter "hot_indexed_on_apply"/,
+ 'bogus hot_indexed_on_apply value reports the expected error');
+
+# Drop the placeholder subscriptions so we can rebuild with real slots.
+$subscriber->safe_psql('postgres', 'DROP SUBSCRIPTION sub_default');
+$subscriber->safe_psql('postgres', 'DROP SUBSCRIPTION sub_always_p');
+
+# --- Apply-path behaviour -------------------------------------------------
+# Pre-populate both sides identically so we can use copy_data=false and
+# avoid duplicate-key conflicts when we recreate subscriptions across the
+# three test cases. We update non-overlapping id ranges per case so the
+# pg_stat counters segment cleanly.
+$publisher->safe_psql('postgres',
+ q{INSERT INTO tab_extra
+ SELECT g, 0, 't' FROM generate_series(1, 200) g});
+$publisher->safe_psql('postgres',
+ q{INSERT INTO tab_pk
+ SELECT g, 0 FROM generate_series(1, 200) g});
+$subscriber->safe_psql('postgres',
+ q{INSERT INTO tab_extra
+ SELECT g, 0, 't' FROM generate_series(1, 200) g});
+$subscriber->safe_psql('postgres',
+ q{INSERT INTO tab_pk
+ SELECT g, 0 FROM generate_series(1, 200) g});
+
+# Helper: read counters and poll up to 10 s for n_tup_upd to reach a
+# minimum target value (the apply worker flushes pgstat asynchronously).
+sub poll_counters
+{
+ my ($node, $table, $upd_target) = @_;
+
+ my $deadline = time() + 10;
+ my $row = '';
+ while (1)
+ {
+ $row = $node->safe_psql('postgres',
+ qq{SELECT coalesce(n_tup_upd, 0),
+ coalesce(n_tup_hot_upd, 0),
+ coalesce(n_tup_hot_indexed_upd, 0)
+ FROM pg_stat_user_tables WHERE relname = '$table'});
+ my ($upd) = split /\|/, $row;
+ last if ($upd + 0) >= $upd_target || time() >= $deadline;
+ usleep(100_000);
+ }
+ my ($upd, $hot, $hot_idx) = split /\|/, $row;
+ return ($upd + 0, $hot + 0, $hot_idx + 0);
+}
+
+# Helper: fire UPDATEs that touch the indexed payload column on a given
+# id range and return the deltas in (n_tup_upd, n_tup_hot_upd,
+# n_tup_hot_indexed_upd) on the subscriber.
+sub apply_updates_and_read
+{
+ my ($table, $sub_name, $id_lo, $id_hi) = @_;
+
+ my ($upd0, $hot0, $hotidx0) =
+ poll_counters($subscriber, $table, 0);
+
+ for my $i ($id_lo .. $id_hi)
+ {
+ $publisher->safe_psql('postgres',
+ "UPDATE $table SET payload = payload + 1 WHERE id = $i");
+ }
+ $publisher->wait_for_catchup($sub_name);
+
+ my $n = $id_hi - $id_lo + 1;
+ my ($upd1, $hot1, $hotidx1) =
+ poll_counters($subscriber, $table, $upd0 + $n);
+ note("$table $sub_name $id_lo..$id_hi: dn_upd="
+ . ($upd1 - $upd0) . " dhot=" . ($hot1 - $hot0)
+ . " dhotidx=" . ($hotidx1 - $hotidx0));
+ return ($upd1 - $upd0, $hot1 - $hot0, $hotidx1 - $hotidx0);
+}
+
+# Case 1: off, subscriber-only secondary index. HOT-indexed must be
+# suppressed on tab_extra. Plain HOT updates also stay zero because every
+# UPDATE touches `payload` which is indexed on the subscriber.
+$subscriber->safe_psql('postgres', qq{
+ CREATE SUBSCRIPTION sub_off
+ CONNECTION '$pub_conninfo'
+ PUBLICATION pub
+ WITH (slot_name = 'sub_off_slot', create_slot = true,
+ hot_indexed_on_apply = 'off', copy_data = false);
+});
+$publisher->wait_for_catchup('sub_off');
+
+my (undef, undef, $off_extra_hotidx) =
+ apply_updates_and_read('tab_extra', 'sub_off', 1, 20);
+is($off_extra_hotidx, 0,
+ 'hot_indexed_on_apply = off: no HOT-indexed updates on tab_extra');
+
+$subscriber->safe_psql('postgres', 'DROP SUBSCRIPTION sub_off');
+
+# Case 2: subset_only. On tab_pk (no secondary index, indexed-attr set is
+# a subset of PK attrs), classic HOT must fire because `payload` is not
+# indexed there. On tab_extra (subscriber's `payload` index is NOT covered
+# by the PK), the apply worker must demote to non-HOT just like 'off'.
+$subscriber->safe_psql('postgres', qq{
+ CREATE SUBSCRIPTION sub_subset
+ CONNECTION '$pub_conninfo'
+ PUBLICATION pub
+ WITH (slot_name = 'sub_subset_slot', create_slot = true,
+ hot_indexed_on_apply = 'subset_only', copy_data = false);
+});
+$publisher->wait_for_catchup('sub_subset');
+
+my (undef, $ss_pk_hot, $ss_pk_hotidx) =
+ apply_updates_and_read('tab_pk', 'sub_subset', 1, 20);
+cmp_ok($ss_pk_hot, '>', 0,
+ 'hot_indexed_on_apply = subset_only: classic HOT fires on tab_pk');
+
+my (undef, undef, $ss_extra_hotidx) =
+ apply_updates_and_read('tab_extra', 'sub_subset', 21, 40);
+is($ss_extra_hotidx, 0,
+ 'hot_indexed_on_apply = subset_only: no HOT-indexed on tab_extra');
+
+$subscriber->safe_psql('postgres', 'DROP SUBSCRIPTION sub_subset');
+
+# Case 3: always. Unconditional HOT-indexed eligibility. On tab_extra
+# updates touching the indexed payload column should now run on the
+# HOT-indexed path: n_tup_hot_indexed_upd must increase.
+$subscriber->safe_psql('postgres', qq{
+ CREATE SUBSCRIPTION sub_always
+ CONNECTION '$pub_conninfo'
+ PUBLICATION pub
+ WITH (slot_name = 'sub_always_slot', create_slot = true,
+ hot_indexed_on_apply = 'always', copy_data = false);
+});
+$publisher->wait_for_catchup('sub_always');
+
+my (undef, undef, $al_extra_hotidx) =
+ apply_updates_and_read('tab_extra', 'sub_always', 41, 80);
+cmp_ok($al_extra_hotidx, '>', 0,
+ 'hot_indexed_on_apply = always: HOT-indexed fires on tab_extra');
+
+# ALTER back to off and verify the apply worker picks up the new mode.
+$subscriber->safe_psql('postgres',
+ q{ALTER SUBSCRIPTION sub_always SET (hot_indexed_on_apply = 'off')});
+is( $subscriber->safe_psql('postgres',
+ q{SELECT subhotindexedonapply FROM pg_subscription
+ WHERE subname = 'sub_always'}),
+ 'o',
+ 'ALTER sub_always SET hot_indexed_on_apply = off persists');
+
+# Drive another batch of updates and confirm n_tup_hot_indexed_upd does NOT
+# advance after the worker rereads the catalog.
+my (undef, undef, $post_alter_hotidx) =
+ apply_updates_and_read('tab_extra', 'sub_always', 81, 100);
+is($post_alter_hotidx, 0,
+ 'ALTER to off freezes n_tup_hot_indexed_upd after worker reread');
+
+$subscriber->safe_psql('postgres', 'DROP SUBSCRIPTION sub_always');
+
+# --- Subscriber INSERT-after-replicated-UPDATE per mode -------------------
+#
+# Verify that a subscriber INSERT using the OLD value of a replicated
+# UPDATE's indexed column succeeds without a spurious unique-violation
+# under each apply mode. Use a dedicated table (tab_uk) so the unique
+# constraint can be defined up-front and the test does not collide with
+# pre-populated rows from the apply-path scenarios above.
+#
+# Publisher updates row $upd_id changing payload from 0 to 999. The
+# subscriber then inserts a fresh row with payload=0 (the pre-update
+# value). Under all three modes _bt_check_unique's recheck of the
+# conflicting tuple's live key must recognize the stale leaf entry pointing
+# at the chain root, so the INSERT succeeds.
+
+$publisher->safe_psql('postgres',
+ q{CREATE TABLE tab_uk (
+ id int PRIMARY KEY,
+ payload int,
+ tag text,
+ UNIQUE (payload, tag))});
+$subscriber->safe_psql('postgres',
+ q{CREATE TABLE tab_uk (
+ id int PRIMARY KEY,
+ payload int,
+ tag text,
+ UNIQUE (payload, tag))});
+$publisher->safe_psql('postgres',
+ q{ALTER PUBLICATION pub ADD TABLE tab_uk});
+
+for my $mode ('off', 'subset_only', 'always')
+{
+ my $base_id = ($mode eq 'off') ? 1
+ : ($mode eq 'subset_only') ? 100 : 200;
+ my $upd_id = $base_id + 1;
+ my $ins_id = $base_id + 2;
+
+ # Seed a row that we will UPDATE on the publisher (payload starts at 0),
+ # and drain the apply for it before changing payload.
+ $publisher->safe_psql('postgres',
+ "INSERT INTO tab_uk VALUES ($upd_id, 0, 'mode_$mode')");
+
+ $subscriber->safe_psql('postgres', qq{
+ CREATE SUBSCRIPTION sub_uk_$mode
+ CONNECTION '$pub_conninfo'
+ PUBLICATION pub
+ WITH (slot_name = 'sub_uk_${mode}_slot', create_slot = true,
+ hot_indexed_on_apply = '$mode', copy_data = true);
+ });
+ $publisher->wait_for_catchup("sub_uk_$mode");
+
+ # Publisher UPDATE: payload 0 -> 999.
+ $publisher->safe_psql('postgres',
+ "UPDATE tab_uk SET payload = 999 WHERE id = $upd_id");
+ $publisher->wait_for_catchup("sub_uk_$mode");
+
+ # Subscriber INSERT with the OLD payload value but a unique tag. The
+ # existing chain leaf with key (0, 'mode_$mode') is now stale: the
+ # live tuple at the chain root has payload=999. _bt_check_unique
+ # rechecks the conflicting tuple's live key and recognizes the stale
+ # leaf, allowing this INSERT to succeed.
+ my ($r, $out, $err) = $subscriber->psql('postgres',
+ "INSERT INTO tab_uk VALUES ($ins_id, 0, 'fresh_$mode')");
+ is($r, 0,
+ "hot_indexed_on_apply = $mode: "
+ . "subscriber INSERT with old payload value succeeds");
+ like($err, qr/^$/,
+ "hot_indexed_on_apply = $mode: "
+ . "INSERT did not raise an error");
+
+ $subscriber->safe_psql('postgres',
+ "DROP SUBSCRIPTION sub_uk_$mode");
+}
+
+# --- always-mode safety with indexed attrs beyond the replica identity -----
+#
+# Amit's corner: under hot_indexed_on_apply = 'always' the apply worker may
+# run a HOT-indexed update even when the table has an indexed attribute that
+# is NOT covered by the replica identity. The read-side staleness mechanism
+# must still let the apply worker's later replica-identity lookups find the
+# row, and DELETE/UPDATE replication must converge, including when the
+# replica-identity column itself is cycled away and back (ABA).
+#
+# tab_ri: replica identity is a UNIQUE index on rid (not the PK), and there
+# is an extra secondary index on payload that is NOT part of the replica
+# identity. So a payload change makes the payload leaf stale, and an rid
+# ABA cycle makes the rid (replica-identity) leaf stale -- both while
+# 'always' keeps the updates on the HOT chain.
+$publisher->safe_psql('postgres', q{
+ CREATE TABLE tab_ri (id int PRIMARY KEY, rid int NOT NULL, payload int);
+ CREATE UNIQUE INDEX tab_ri_rid_uk ON tab_ri(rid);
+ ALTER TABLE tab_ri REPLICA IDENTITY USING INDEX tab_ri_rid_uk;
+ CREATE PUBLICATION pub_ri FOR TABLE tab_ri;
+});
+$subscriber->safe_psql('postgres', q{
+ CREATE TABLE tab_ri (id int PRIMARY KEY, rid int NOT NULL, payload int);
+ CREATE UNIQUE INDEX tab_ri_rid_uk ON tab_ri(rid);
+ ALTER TABLE tab_ri REPLICA IDENTITY USING INDEX tab_ri_rid_uk;
+ CREATE INDEX tab_ri_payload_idx ON tab_ri(payload);
+});
+
+$publisher->safe_psql('postgres',
+ q{INSERT INTO tab_ri VALUES (1, 10, 0), (2, 20, 0)});
+
+$subscriber->safe_psql('postgres', qq{
+ CREATE SUBSCRIPTION sub_ri
+ CONNECTION '$pub_conninfo'
+ PUBLICATION pub_ri
+ WITH (slot_name = 'sub_ri_slot', create_slot = true,
+ hot_indexed_on_apply = 'always', copy_data = true);
+});
+# Wait for the initial table COPY to finish, not just streaming catch-up, so
+# the seeded rows are present before we start updating them.
+$subscriber->wait_for_subscription_sync($publisher, 'sub_ri');
+
+# Cycle the replica-identity column away and back (ABA), and also churn the
+# extra payload index, all replicated under 'always'. Each step is a
+# HOT-indexed update on the subscriber that leaves a stale leaf.
+$publisher->safe_psql('postgres', q{
+ UPDATE tab_ri SET rid = 11, payload = payload + 1 WHERE id = 1;
+ UPDATE tab_ri SET rid = 10, payload = payload + 1 WHERE id = 1; -- rid ABA
+ UPDATE tab_ri SET payload = payload + 1 WHERE id = 2;
+});
+$publisher->wait_for_catchup('sub_ri');
+
+# Confirm the apply worker actually took the HOT-indexed path on tab_ri (the
+# whole point of 'always' with indexed attrs beyond the replica identity).
+# Without this the convergence/verify_heapam asserts below could pass
+# vacuously if eligibility silently regressed to plain non-HOT.
+my (undef, undef, $ri_hotidx) = poll_counters($subscriber, 'tab_ri', 3);
+cmp_ok($ri_hotidx, '>', 0,
+ 'always-mode: HOT-indexed path fired on tab_ri (rid/payload churn)');
+
+# A subsequent replicated UPDATE keyed by the replica identity (rid) must
+# find the row despite the stale rid/payload leaves the ABA left behind.
+$publisher->safe_psql('postgres',
+ q{UPDATE tab_ri SET payload = 100 WHERE id = 1});
+# And a replicated DELETE resolved through the replica-identity index must
+# also find and remove the right row.
+$publisher->safe_psql('postgres', q{DELETE FROM tab_ri WHERE id = 2});
+$publisher->wait_for_catchup('sub_ri');
+
+is( $subscriber->safe_psql('postgres',
+ q{SELECT rid || ':' || payload FROM tab_ri WHERE id = 1}),
+ '10:100',
+ 'always-mode: replicated UPDATE found the row via RI after rid ABA');
+is( $subscriber->safe_psql('postgres',
+ q{SELECT count(*) FROM tab_ri WHERE id = 2}),
+ '0',
+ 'always-mode: replicated DELETE found the row via RI with stale leaves');
+# Full convergence cross-check.
+is( $subscriber->safe_psql('postgres',
+ q{SELECT string_agg(id || ',' || rid || ',' || payload, ';' ORDER BY id)
+ FROM tab_ri}),
+ $publisher->safe_psql('postgres',
+ q{SELECT string_agg(id || ',' || rid || ',' || payload, ';' ORDER BY id)
+ FROM tab_ri}),
+ 'always-mode: tab_ri converges between publisher and subscriber');
+
+# verify_heapam finds no corruption in the HOT-indexed chains left behind.
+$subscriber->safe_psql('postgres', 'CREATE EXTENSION IF NOT EXISTS amcheck');
+is( $subscriber->safe_psql('postgres',
+ q{SELECT count(*) FROM verify_heapam('tab_ri')}),
+ '0',
+ 'always-mode: verify_heapam clean on tab_ri after stale-leaf churn');
+
+$subscriber->safe_psql('postgres', 'DROP SUBSCRIPTION sub_ri');
+
+done_testing();
diff --git a/src/test/subscription/t/040_hot_indexed_replica_identity.pl b/src/test/subscription/t/040_hot_indexed_replica_identity.pl
new file mode 100644
index 00000000000..f801787b4c0
--- /dev/null
+++ b/src/test/subscription/t/040_hot_indexed_replica_identity.pl
@@ -0,0 +1,110 @@
+# Copyright (c) 2026, PostgreSQL Global Development Group
+
+# Live logical replication of HOT-indexed updates under non-default replica
+# identities. The apply worker locates the row to update or delete via the
+# replica identity: a seqscan for REPLICA IDENTITY FULL, and the nominated
+# index for REPLICA IDENTITY USING INDEX. On a subscriber whose tables carry
+# extra indexes (so apply performs HOT-indexed updates and leaves stale index
+# leaves), that lookup must still find the current row -- including after the
+# identity column's value is cycled away and back (ABA).
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $publisher = PostgreSQL::Test::Cluster->new('publisher');
+$publisher->init(allows_streaming => 'logical');
+$publisher->start;
+
+my $subscriber = PostgreSQL::Test::Cluster->new('subscriber');
+$subscriber->init;
+$subscriber->start;
+
+my $pub_conninfo = $publisher->connstr . ' dbname=postgres';
+
+# tab_full: REPLICA IDENTITY FULL (apply uses a sequential scan).
+# tab_idx: REPLICA IDENTITY USING INDEX on a non-PK unique index whose
+# column is itself updated, so the apply-side index lookup must
+# tolerate stale leaves left by earlier HOT-indexed updates.
+$publisher->safe_psql('postgres', q{
+ CREATE TABLE tab_full (a int, b int, c int);
+ ALTER TABLE tab_full REPLICA IDENTITY FULL;
+ CREATE TABLE tab_idx (k int NOT NULL, v int, w int);
+ CREATE UNIQUE INDEX tab_idx_k ON tab_idx (k);
+ ALTER TABLE tab_idx REPLICA IDENTITY USING INDEX tab_idx_k;
+ CREATE PUBLICATION pub FOR TABLE tab_full, tab_idx;
+});
+
+# The subscriber adds extra secondary indexes so that an UPDATE changing one
+# indexed column stays HOT-indexed on apply.
+$subscriber->safe_psql('postgres', q{
+ CREATE TABLE tab_full (a int, b int, c int) WITH (fillfactor = 50);
+ CREATE INDEX tab_full_b ON tab_full (b);
+ CREATE INDEX tab_full_c ON tab_full (c);
+ ALTER TABLE tab_full REPLICA IDENTITY FULL;
+ CREATE TABLE tab_idx (k int NOT NULL, v int, w int) WITH (fillfactor = 50);
+ CREATE UNIQUE INDEX tab_idx_k ON tab_idx (k);
+ CREATE INDEX tab_idx_v ON tab_idx (v);
+ CREATE INDEX tab_idx_w ON tab_idx (w);
+ ALTER TABLE tab_idx REPLICA IDENTITY USING INDEX tab_idx_k;
+});
+
+# Allow HOT-indexed updates on the apply path.
+$subscriber->safe_psql('postgres', qq{
+ CREATE SUBSCRIPTION sub
+ CONNECTION '$pub_conninfo'
+ PUBLICATION pub
+ WITH (hot_indexed_on_apply = always);
+});
+$subscriber->wait_for_subscription_sync($publisher, 'sub');
+
+# Seed both tables.
+$publisher->safe_psql('postgres', q{
+ INSERT INTO tab_full VALUES (1, 10, 100), (2, 20, 200);
+ INSERT INTO tab_idx VALUES (1, 10, 1000), (2, 20, 2000);
+});
+$publisher->wait_for_catchup('sub');
+
+# A run of single-column updates: each stays HOT-indexed on the subscriber and
+# leaves stale leaves, then the identity/PK row is matched again by the next
+# change. Include an ABA cycle on the USING INDEX column k (1 -> 3 -> 1).
+$publisher->safe_psql('postgres', q{
+ UPDATE tab_full SET b = b + 1 WHERE a = 1;
+ UPDATE tab_full SET c = c + 1 WHERE a = 1;
+ UPDATE tab_full SET b = b + 1 WHERE a = 1;
+ UPDATE tab_idx SET v = v + 1 WHERE k = 1;
+ UPDATE tab_idx SET k = 3 WHERE k = 1;
+ UPDATE tab_idx SET w = w + 1 WHERE k = 3;
+ UPDATE tab_idx SET k = 1 WHERE k = 3;
+ DELETE FROM tab_full WHERE a = 2;
+ DELETE FROM tab_idx WHERE k = 2;
+});
+$publisher->wait_for_catchup('sub');
+
+# The subscriber must match the publisher exactly: the RI lookups found the
+# right rows across the HOT-indexed chains and the ABA cycle.
+my $pub_full = $publisher->safe_psql('postgres',
+ q{SELECT a, b, c FROM tab_full ORDER BY a});
+my $sub_full = $subscriber->safe_psql('postgres',
+ q{SELECT a, b, c FROM tab_full ORDER BY a});
+is($sub_full, $pub_full, 'REPLICA IDENTITY FULL: subscriber matches publisher');
+
+my $pub_idx = $publisher->safe_psql('postgres',
+ q{SELECT k, v, w FROM tab_idx ORDER BY k});
+my $sub_idx = $subscriber->safe_psql('postgres',
+ q{SELECT k, v, w FROM tab_idx ORDER BY k});
+is($sub_idx, $pub_idx,
+ 'REPLICA IDENTITY USING INDEX: subscriber matches publisher across ABA');
+
+# The subscriber's tables must be structurally consistent (stubs recognised).
+$subscriber->safe_psql('postgres', q{CREATE EXTENSION amcheck});
+is( $subscriber->safe_psql('postgres',
+ q{SELECT count(*) FROM verify_heapam('tab_idx')}),
+ '0',
+ 'subscriber tab_idx has no heap corruption after HOT-indexed apply');
+
+$subscriber->stop;
+$publisher->stop;
+
+done_testing();
--
2.50.1
[text/x-patch] v57-0009-DO-NOT-MERGE-Add-a-HOT-SIU-benchmark-harness.patch (33.4K, ../[email protected]/10-v57-0009-DO-NOT-MERGE-Add-a-HOT-SIU-benchmark-harness.patch)
download | inline diff:
From 2342b30cc1381f94e04252d682cb5bae4d7eb8eb Mon Sep 17 00:00:00 2001
From: Greg Burd <[email protected]>
Date: Mon, 15 Jun 2026 14:22:45 -0400
Subject: [PATCH v57 9/9] [DO NOT MERGE] Add a HOT/SIU benchmark harness
A/B and single-variant benchmark scripts for HOT-indexed updates: build two
postgres variants, run pgbench workloads exercising classic-HOT, non-HOT, and
HOT-indexed paths, and a self-contained bloat probe that reports the skip count
(index writes avoided on unchanged indexes) and changed-index bounding. Not
for merge; kept for evaluating the feature.
---
src/test/benchmarks/siu/README.md | 82 ++++
src/test/benchmarks/siu/scripts/bloat.sh | 84 ++++
src/test/benchmarks/siu/scripts/build.sh | 54 +++
.../siu/scripts/hot_indexed_mixed.sql | 11 +
.../siu/scripts/hot_indexed_update.sql | 6 +
.../benchmarks/siu/scripts/read_indexscan.sql | 11 +
src/test/benchmarks/siu/scripts/run.sh | 377 ++++++++++++++++++
src/test/benchmarks/siu/scripts/soak.sh | 128 ++++++
.../benchmarks/siu/scripts/wide_update.sql | 7 +
9 files changed, 760 insertions(+)
create mode 100644 src/test/benchmarks/siu/README.md
create mode 100755 src/test/benchmarks/siu/scripts/bloat.sh
create mode 100755 src/test/benchmarks/siu/scripts/build.sh
create mode 100644 src/test/benchmarks/siu/scripts/hot_indexed_mixed.sql
create mode 100644 src/test/benchmarks/siu/scripts/hot_indexed_update.sql
create mode 100644 src/test/benchmarks/siu/scripts/read_indexscan.sql
create mode 100755 src/test/benchmarks/siu/scripts/run.sh
create mode 100755 src/test/benchmarks/siu/scripts/soak.sh
create mode 100644 src/test/benchmarks/siu/scripts/wide_update.sql
diff --git a/src/test/benchmarks/siu/README.md b/src/test/benchmarks/siu/README.md
new file mode 100644
index 00000000000..9a62d268ad7
--- /dev/null
+++ b/src/test/benchmarks/siu/README.md
@@ -0,0 +1,82 @@
+# hot-indexed (HOT-indexed) A/B benchmark harness
+
+Two postgres variants, identical pgdata layouts, pgbench workloads
+exercising classic HOT, non-HOT, and HOT-indexed paths.
+
+## Contents
+
+- `scripts/build.sh` -- builds two postgres variants (`master` = tepid's
+ merge-base with origin/master; `tepid` = the branch under test). Requires
+ a writable benchmark root via `BENCH` (default `/scratch/tepid-bench`).
+- `scripts/run.sh` -- A/B driver. Runs `simple_update` (pgbench -N),
+ `hot_indexed_update`, `hot_indexed_mixed`, `read_indexscan`, and `wide_N`
+ for N in `$WIDE_STEPS`.
+ Collects TPS, latency, WAL bytes, HOT update count, pre/post heap and
+ index size, peak CPU% and RSS. Writes a CSV per run to `$BENCH/results/`.
+- `scripts/soak.sh` -- long-running single-workload driver that samples
+ TPS/HOT%/WAL/bloat every `$SAMPLE` seconds under `$DURATION` seconds
+ of constant pressure, per variant.
+- `scripts/bloat.sh` -- single-variant bloat probe. Runs update+vacuum cycles
+ on a table whose changed column and an unchanged column are both indexed, and
+ reports (via pgstattuple) that the changed index stays bounded with periodic
+ VACUUM but grows unbounded without it, plus the skip count showing
+ HOT-indexed updates avoiding the unchanged index. Spins its own throwaway
+ cluster, so it does not touch the A/B pgdata.
+- `scripts/hot_indexed_update.sql` -- `UPDATE siu_table SET b = rand WHERE a = rand`.
+- `scripts/hot_indexed_mixed.sql` -- 80 % SELECT by PK + 20 % indexed-col UPDATE.
+- `scripts/read_indexscan.sql` -- read-only btree index scans on a freshly
+ reset `siu_table` (no stale entries); confirms the HOT-indexed read path
+ adds no per-scan overhead, since the crossed-attribute bitmap decides
+ staleness without a key comparison and the scan requests no index tuple.
+- `scripts/wide_update.sql` -- driver script for the wide-table workload;
+ the `SET` clause is built at run time from `$WIDE_STEPS`.
+
+## Running
+
+```
+# Build both variants (run once per benchmark host)
+REPO=$HOME/ws/postgres/tepid BENCH=/scratch/tepid-bench \
+ ./scripts/build.sh
+
+# Standard A/B
+SCALE=20 CLIENTS=16 THREADS=8 DURATION=120 \
+ WIDE_COLS=16 WIDE_STEPS=0,1,2,4,8,16 \
+ ./scripts/run.sh
+
+# Soak
+SCALE=50 CLIENTS=16 THREADS=8 DURATION=900 SAMPLE=60 \
+ ./scripts/soak.sh
+
+# Bloat probe (single variant; defaults to the tepid build)
+BENCH=/scratch/tepid-bench ROWS=5000 CYCLES=8 UPDATES=20 \
+ ./scripts/bloat.sh
+```
+
+## Env vars
+
+```
+REPO path to postgres source (has .git)
+BENCH bench root (install prefixes, build trees, results)
+SCALE pgbench -s (also drives siu_table row count = SCALE*100k)
+CLIENTS pgbench -c
+THREADS pgbench -j
+DURATION seconds per workload
+WIDE_COLS number of indexed int columns in wide_table (default 16)
+WIDE_STEPS comma-separated list of columns-modified values to exercise
+ (default 0,1,4,8,16)
+PORT postgres port for the bench servers
+SHARED_BUFFERS postgresql.conf setting (default 512MB)
+MASTER_REV revision for the master variant (default: tepid's merge-base
+ with origin/master)
+TEPID_REV revision for the tepid variant (default: tepid)
+
+bloat.sh only:
+BINDIR postgres bin dir to test (default: tepid variant under $BENCH)
+ROWS seeded rows (default 5000)
+CYCLES update+vacuum cycles (default 8)
+UPDATES updates per row per cycle (default 20)
+```
+
+The scripts are portable between Linux and FreeBSD; the CPU/RSS sampler
+uses `ps -o pcpu=,rss= --ppid LEADER -p LEADER` (Linux) or `pgrep -P` +
+per-pid `ps` (FreeBSD) -- peak values are approximate.
diff --git a/src/test/benchmarks/siu/scripts/bloat.sh b/src/test/benchmarks/siu/scripts/bloat.sh
new file mode 100755
index 00000000000..34171c68f5b
--- /dev/null
+++ b/src/test/benchmarks/siu/scripts/bloat.sh
@@ -0,0 +1,84 @@
+#!/usr/bin/env bash
+# Single-variant bloat benchmark for HOT-indexed (SIU) updates.
+#
+# The A/B run.sh measures TPS/WAL/aggregate size; it does not isolate two
+# bloat properties of the HOT-indexed path, which this script demonstrates:
+#
+# 1. Stale index entries on the CHANGED index accumulate between vacuums,
+# but VACUUM reclaims them, so size stays bounded across update+vacuum
+# cycles. Skipping vacuum lets them grow unbounded (the inherent
+# inter-vacuum bloat; read-filtered by the crossed-attribute bitmap meanwhile).
+# 2. An index on an UNCHANGED column is skipped by HOT-indexed updates (the
+# selective-update benefit) -- visible as a skip count. Updates that fall
+# back to non-HOT (e.g. when the page has no room for the chain) still
+# insert into it, so its size reflects only the non-HOT remainder.
+#
+# Uses the tepid variant built by build.sh (override BINDIR for any build) and
+# a throwaway cluster under $BENCH, so it never touches the A/B pgdata.
+#
+# Env: BENCH (default /scratch/siu-bench), BINDIR (default tepid variant bin),
+# PORT (default 57481), ROWS (default 5000), CYCLES (default 8),
+# UPDATES (updates per row per cycle, default 20).
+set -euo pipefail
+
+BENCH=${BENCH:-/scratch/siu-bench}
+BINDIR=${BINDIR:-$BENCH/tepid/usr/local/pgsql/bin}
+PORT=${PORT:-57481}
+ROWS=${ROWS:-5000}
+CYCLES=${CYCLES:-8}
+UPDATES=${UPDATES:-20}
+DATADIR=$BENCH/_data_bloat
+
+base=$(dirname "$BINDIR")
+if [ -d "$base/lib64" ]; then
+ export LD_LIBRARY_PATH="$base/lib64${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
+else
+ export LD_LIBRARY_PATH="$base/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
+fi
+
+PSQL=("$BINDIR/psql" -h /tmp -p "$PORT" -U postgres -X -q)
+P() { "${PSQL[@]}" -At "$@"; }
+
+"$BINDIR/pg_ctl" -D "$DATADIR" stop -m fast >/dev/null 2>&1 || true
+rm -rf "$DATADIR"
+"$BINDIR/initdb" -D "$DATADIR" -U postgres --no-sync >/dev/null
+cat >> "$DATADIR/postgresql.conf" <<EOF
+port = $PORT
+autovacuum = off
+fsync = off
+EOF
+"$BINDIR/pg_ctl" -D "$DATADIR" -o "-p $PORT" -l "$BENCH/_bloat.log" -w start >/dev/null
+trap '"$BINDIR/pg_ctl" -D "$DATADIR" stop -m fast >/dev/null 2>&1 || true' EXIT
+"${PSQL[@]}" -c "CREATE EXTENSION IF NOT EXISTS pgstattuple;" >/dev/null
+
+# $1 = table name, $2 = vacuum_each (1/0). Returns final idx_a composition.
+run_arm() {
+ local tbl=$1 vac=$2
+ "${PSQL[@]}" <<SQL >/dev/null
+DROP TABLE IF EXISTS $tbl;
+CREATE TABLE $tbl (id int PRIMARY KEY, a int, b int, pad text) WITH (fillfactor = 50);
+CREATE INDEX ${tbl}_a ON $tbl(a); -- changed column
+CREATE INDEX ${tbl}_b ON $tbl(b); -- never changed
+INSERT INTO $tbl SELECT g, g, g, repeat('x', 40) FROM generate_series(1, $ROWS) g;
+VACUUM (FREEZE, ANALYZE) $tbl;
+SQL
+ local cyc
+ for ((cyc = 1; cyc <= CYCLES; cyc++)); do
+ "${PSQL[@]}" -c "DO \$\$ BEGIN FOR u IN 1..$UPDATES LOOP UPDATE $tbl SET a = a + 1; END LOOP; END \$\$;" >/dev/null
+ [ "$vac" = 1 ] && "${PSQL[@]}" -c "VACUUM $tbl;" >/dev/null
+ done
+ P -c "SELECT '$tbl(vacuum_each=$vac):'
+ || ' idx_a_kb=' || pg_relation_size('${tbl}_a')/1024
+ || ' idx_a_live=' || (SELECT tuple_count FROM pgstattuple('${tbl}_a'))
+ || ' idx_a_free%=' || round((SELECT free_percent FROM pgstattuple('${tbl}_a'))::numeric,1)
+ || ' idx_b_kb=' || pg_relation_size('${tbl}_b')/1024
+ || ' idx_b_skips=' || coalesce((SELECT n_tup_hot_indexed_upd_skipped
+ FROM pg_stat_all_indexes WHERE indexrelname='${tbl}_b'), 0);"
+}
+
+echo "=== HOT-indexed bloat: $CYCLES cycles x ($UPDATES updates/row x $ROWS rows) ==="
+run_arm t_vac 1
+run_arm t_novac 0
+echo "idx_a: bounded with vacuum_each=1, unbounded with =0 (stale entries accumulate"
+echo "until reclaimed). idx_b_skips counts entries the HOT-indexed path avoided on"
+echo "the unchanged index; idx_b_kb is only the non-HOT-fallback remainder."
diff --git a/src/test/benchmarks/siu/scripts/build.sh b/src/test/benchmarks/siu/scripts/build.sh
new file mode 100755
index 00000000000..b2f0ee525d4
--- /dev/null
+++ b/src/test/benchmarks/siu/scripts/build.sh
@@ -0,0 +1,54 @@
+#!/usr/bin/env bash
+# Build two postgres variants for tepid (HOT-indexed) A/B benchmarks.
+#
+# Env vars (all optional):
+# REPO -- path to postgres source repo (default: $HOME/ws/postgres/tepid, or /scratch/siu-bench/repo)
+# BENCH -- bench root (default: /scratch/siu-bench)
+# MASTER_REV -- revision for the "master" variant (default: tepid's merge-base with origin/master)
+# TEPID_REV -- revision for the "tepid" variant (default: tepid)
+# JOBS -- parallel compile jobs (default: nproc or 8)
+set -euo pipefail
+
+BENCH=${BENCH:-/scratch/siu-bench}
+JOBS=${JOBS:-$( (command -v nproc >/dev/null && nproc) || sysctl -n hw.ncpu 2>/dev/null || echo 8 )}
+if [ -z "${REPO:-}" ]; then
+ for candidate in "$HOME/ws/postgres/tepid" "$BENCH/repo" /scratch/pg; do
+ if [ -d "$candidate/.git" ]; then REPO=$candidate; break; fi
+ done
+fi
+: "${REPO:?REPO not set and no default found}"
+cd "$REPO"
+
+TEPID_REV=${TEPID_REV:-tepid}
+MASTER_REV=${MASTER_REV:-$(git merge-base "$TEPID_REV" origin/master 2>/dev/null || git merge-base "$TEPID_REV" master)}
+
+echo "REPO=$REPO MASTER=$MASTER_REV TEPID=$TEPID_REV JOBS=$JOBS BENCH=$BENCH"
+
+die() { printf 'build: %s\n' "$*" >&2; exit 1; }
+if git status --porcelain | grep -v '^??' | grep -q .; then
+ die "repo has unstaged/uncommitted changes; stash or commit first"
+fi
+
+build_variant() {
+ local name=$1
+ local rev=$2
+ local prefix=$BENCH/$name
+ echo "=== building $name ($rev) into $prefix"
+ [ -d "$prefix" ] && find "$prefix" -mindepth 1 -delete && rmdir "$prefix"
+ mkdir -p "$prefix"
+ git checkout --quiet --detach "$rev"
+ local bld=$BENCH/_build_$name
+ [ -d "$bld" ] && find "$bld" -mindepth 1 -delete && rmdir "$bld"
+ meson setup "$bld" --prefix="$prefix/usr/local/pgsql" \
+ -Dbuildtype=release -Dcassert=false \
+ -Dextra_version=-siubench-$name >/dev/null
+ meson compile -C "$bld" -j "$JOBS"
+ meson install -C "$bld" --destdir=/ >/dev/null
+ "$prefix/usr/local/pgsql/bin/postgres" --version
+}
+
+ORIG=$(git symbolic-ref --quiet --short HEAD || git rev-parse HEAD)
+trap 'git checkout --quiet "$ORIG"' EXIT
+
+build_variant master "$MASTER_REV"
+build_variant tepid "$TEPID_REV"
diff --git a/src/test/benchmarks/siu/scripts/hot_indexed_mixed.sql b/src/test/benchmarks/siu/scripts/hot_indexed_mixed.sql
new file mode 100644
index 00000000000..3ab3289df27
--- /dev/null
+++ b/src/test/benchmarks/siu/scripts/hot_indexed_mixed.sql
@@ -0,0 +1,11 @@
+-- Mixed workload: 80% selects, 20% indexed-column updates.
+-- Exercises both the hot-indexed writer and the crossed-attribute-bitmap reader.
+\set aid random(1, :scale * 100000)
+\set bid random(1, 1000000)
+\set which random(1, 100)
+BEGIN;
+SELECT * FROM siu_table WHERE a = :aid;
+\if :which > 80
+ UPDATE siu_table SET b = :bid WHERE a = :aid;
+\endif
+COMMIT;
diff --git a/src/test/benchmarks/siu/scripts/hot_indexed_update.sql b/src/test/benchmarks/siu/scripts/hot_indexed_update.sql
new file mode 100644
index 00000000000..f1bcf959c67
--- /dev/null
+++ b/src/test/benchmarks/siu/scripts/hot_indexed_update.sql
@@ -0,0 +1,6 @@
+-- hot-indexed-friendly workload: narrow table with a few non-PK indexes.
+-- Each UPDATE changes a non-summarizing indexed column on a random row.
+-- With hot-indexed this is HOT-indexed; without hot-indexed it is non-HOT.
+\set aid random(1, :scale * 100000)
+\set new_b random(1, 1000000)
+UPDATE siu_table SET b = :new_b WHERE a = :aid;
diff --git a/src/test/benchmarks/siu/scripts/read_indexscan.sql b/src/test/benchmarks/siu/scripts/read_indexscan.sql
new file mode 100644
index 00000000000..465689763ea
--- /dev/null
+++ b/src/test/benchmarks/siu/scripts/read_indexscan.sql
@@ -0,0 +1,11 @@
+-- read_indexscan: read-only btree index-scan workload confirming the
+-- HOT-indexed read path adds no per-scan overhead on tables with no stale
+-- entries.
+-- The crossed-attribute bitmap decides staleness without a key comparison and
+-- the scan does not request the index tuple, so on a freshly reset siu_table
+-- (no stale HOT-indexed entries) there should be no master-vs-tepid
+-- difference on this cell. The predicate is an equality on the
+-- indexed column b and the target list includes the non-indexed column e,
+-- forcing a plain (heap-fetching) index scan rather than an index-only scan.
+\set id random(1, :rows)
+SELECT a, b, c, d, e FROM siu_table WHERE b = :id;
diff --git a/src/test/benchmarks/siu/scripts/run.sh b/src/test/benchmarks/siu/scripts/run.sh
new file mode 100755
index 00000000000..fa330ad4f6f
--- /dev/null
+++ b/src/test/benchmarks/siu/scripts/run.sh
@@ -0,0 +1,377 @@
+#!/usr/bin/env bash
+# A/B pgbench harness for tepid: master (upstream) vs tepid (HOT-indexed).
+#
+# Env vars:
+# SCALE -- pgbench -s (also multiplier for siu_table row count = SCALE*100k)
+# CLIENTS -- pgbench -c
+# THREADS -- pgbench -j
+# DURATION -- pgbench -T (seconds per workload)
+# WIDE_COLS -- # of indexed columns in the wide_table (default 16)
+# WIDE_STEPS -- comma-separated list of "updated columns" counts for
+# the wide workload (default "0,1,4,8,WIDE_COLS")
+# PORT -- postgres port (default 57480)
+#
+# For each variant in {master, tepid}:
+# initdb fresh pgdata, start postgres, create test objects,
+# run workloads (pgbench -N simple_update, hot_indexed_update, hot_indexed_mixed,
+# read_indexscan, and wide_N for each value in WIDE_STEPS), collect TPS + HOT
+# counts + WAL delta + peak CPU/RSS sampled via pidstat.
+# Emits CSV + Markdown summary under /scratch/siu-bench/results/.
+set -euo pipefail
+
+BENCH=${BENCH:-/scratch/siu-bench}
+SCALE=${SCALE:-20}
+CLIENTS=${CLIENTS:-16}
+THREADS=${THREADS:-8}
+DURATION=${DURATION:-120}
+WIDE_COLS=${WIDE_COLS:-16}
+WIDE_STEPS=${WIDE_STEPS:-0,1,4,8,16}
+PORT=${PORT:-57480}
+
+TS=$(date -u +%Y%m%dT%H%M%SZ)
+OUT=$BENCH/results/$TS.csv
+LOGDIR=$BENCH/logs/$TS
+mkdir -p "$LOGDIR"
+echo "variant,workload,tps,latency_avg_ms,classic_hot_updates,hot_indexed_updates,non_hot_updates,total_updates,wal_bytes,bloat_pages_before,bloat_pages_after,index_size_before,index_size_after,cpu_pct_peak,rss_mib_peak,per_index_before,per_index_after" > "$OUT"
+echo "=== siu-bench A/B run $TS -> $OUT (scale=$SCALE clients=$CLIENTS threads=$THREADS duration=${DURATION}s)"
+
+bin_of() {
+ echo "$BENCH/$1/usr/local/pgsql/bin"
+}
+
+LD_of() {
+ local base=$BENCH/$1/usr/local/pgsql
+ # Linux distros that split 64-bit libs use lib64; most others use lib.
+ if [ -d "$base/lib64" ]; then
+ echo "$base/lib64"
+ else
+ echo "$base/lib"
+ fi
+}
+
+psql_as() {
+ local v=$1; shift
+ LD_LIBRARY_PATH="$(LD_of "$v")" "$(bin_of "$v")/psql" -h /tmp -p "$PORT" -U postgres -X "$@"
+}
+
+pgbench_as() {
+ local v=$1; shift
+ LD_LIBRARY_PATH="$(LD_of "$v")" "$(bin_of "$v")/pgbench" -h /tmp -p "$PORT" -U postgres "$@"
+}
+
+start_pg() {
+ local v=$1
+ local datadir=$BENCH/_data_$v
+ [ -d "$datadir" ] && find "$datadir" -mindepth 1 -delete && rmdir "$datadir"
+ mkdir -p "$datadir"
+
+ LD_LIBRARY_PATH="$(LD_of "$v")" "$(bin_of "$v")/initdb" -D "$datadir" -U postgres >"$LOGDIR/initdb_$v.log" 2>&1
+ local sb=${SHARED_BUFFERS:-512MB}
+ cat >> "$datadir/postgresql.conf" <<EOF
+shared_buffers = $sb
+work_mem = 32MB
+max_wal_size = 4GB
+synchronous_commit = on
+checkpoint_timeout = 10min
+wal_level = replica
+log_destination = 'stderr'
+logging_collector = off
+port = $PORT
+EOF
+ LD_LIBRARY_PATH="$(LD_of "$v")" "$(bin_of "$v")/pg_ctl" -D "$datadir" \
+ -o "-p $PORT" -l "$LOGDIR/pg_$v.log" start >/dev/null
+ sleep 2
+}
+
+stop_pg() {
+ local v=$1
+ local datadir=$BENCH/_data_$v
+ LD_LIBRARY_PATH="$(LD_of "$v")" "$(bin_of "$v")/pg_ctl" -D "$datadir" stop -m fast >/dev/null 2>&1 || true
+}
+
+postmaster_pid() {
+ local v=$1
+ head -1 "$BENCH/_data_$v/postmaster.pid" 2>/dev/null
+}
+
+setup_schemas() {
+ local v=$1
+ seed_siu_table "$v"
+ seed_wide_table "$v"
+ # pgbench schema for built-in simple_update.
+ LD_LIBRARY_PATH="$(LD_of "$v")" "$(bin_of "$v")/pgbench" -h /tmp -p "$PORT" -U postgres \
+ -i -s "$SCALE" -q postgres >"$LOGDIR/pgbench_init_$v.log" 2>&1
+}
+
+# seed_siu_table: (re)create the narrow table used by the siu_* workloads.
+seed_siu_table() {
+ local v=$1
+ local rows=$((SCALE * 100000))
+ psql_as "$v" <<SQL
+DROP TABLE IF EXISTS siu_table;
+CREATE TABLE siu_table(a int PRIMARY KEY, b int, c int, d int, e text);
+CREATE INDEX siu_b ON siu_table(b);
+CREATE INDEX siu_c ON siu_table(c);
+CREATE INDEX siu_d ON siu_table(d);
+INSERT INTO siu_table
+ SELECT i, i, i, i, repeat('x', 20) FROM generate_series(1, $rows) AS i;
+VACUUM (FULL, ANALYZE) siu_table;
+CHECKPOINT;
+SQL
+}
+
+# seed_wide_table: (re)create the wide table with WIDE_COLS indexed columns.
+seed_wide_table() {
+ local v=$1
+ local coldefs="" insertcols="" insertvals="" idxlist=""
+ for i in $(seq 1 "$WIDE_COLS"); do
+ coldefs+=", c$i int"
+ insertcols+=", c$i"
+ insertvals+=", i"
+ idxlist+="CREATE INDEX wide_c$i ON wide_table(c$i); "
+ done
+ local wide_rows=$((SCALE * 1000))
+ psql_as "$v" <<SQL
+DROP TABLE IF EXISTS wide_table;
+CREATE TABLE wide_table(id int PRIMARY KEY $coldefs);
+$idxlist
+INSERT INTO wide_table(id $insertcols) SELECT i $insertvals FROM generate_series(1, $wide_rows) AS i;
+VACUUM (FULL, ANALYZE) wide_table;
+CHECKPOINT;
+SQL
+}
+
+# reset_state: restore a workload's target table to its seeded baseline.
+# Used between workloads so per-workload bloat/idx_size deltas are not
+# polluted by carryover from earlier workloads in the same variant run.
+# For pgbench_accounts we re-initialise via `pgbench -i`; for our
+# hand-rolled tables we drop + recreate + reseed.
+reset_state() {
+ local v=$1 table=$2
+ case "$table" in
+ pgbench_accounts)
+ LD_LIBRARY_PATH="$(LD_of "$v")" "$(bin_of "$v")/pgbench" -h /tmp -p "$PORT" -U postgres \
+ -i -s "$SCALE" -q postgres >>"$LOGDIR/pgbench_init_$v.log" 2>&1
+ psql_as "$v" -c "CHECKPOINT" >/dev/null
+ ;;
+ siu_table)
+ seed_siu_table "$v"
+ ;;
+ wide_table)
+ seed_wide_table "$v"
+ ;;
+ *)
+ echo "reset_state: unknown table $table" >&2
+ return 1
+ ;;
+ esac
+ psql_as "$v" -c "SELECT pg_stat_reset_single_table_counters('$table'::regclass::oid)" >/dev/null
+}
+
+bloat_stats() {
+ local v=$1 table=$2
+ psql_as "$v" -Atc "SELECT pg_table_size('$table')/8192 || ',' || pg_indexes_size('$table')"
+}
+
+# siu_count: number of HOT-indexed updates observed on $table since its
+# pgstat counters were last reset. Returns "0" on master (where the
+# counter column does not exist) so the CSV column stays numeric.
+siu_count() {
+ local v=$1 table=$2
+ local val
+ val=$(psql_as "$v" -Atc \
+ "SELECT coalesce(n_tup_hot_indexed_upd, 0) FROM pg_stat_user_tables WHERE relname='$table'" 2>/dev/null)
+ [[ "$val" =~ ^[0-9]+$ ]] || val=0
+ echo "$val"
+}
+
+# per_index_sizes: emit "idx1=bytes;idx2=bytes;..." for the indexes on
+# $table, sorted by indexrelid. Used by the wide_* workloads so we can
+# see per-column index growth rather than just the aggregate. Returns
+# the literal "none" when $table has no indexes.
+per_index_sizes() {
+ local v=$1 table=$2
+ local out
+ out=$(psql_as "$v" -Atc "SELECT string_agg(
+ i.relname || '=' || pg_relation_size(i.oid)::text,
+ ';' ORDER BY i.oid)
+ FROM pg_class t
+ JOIN pg_index ix ON ix.indrelid = t.oid
+ JOIN pg_class i ON i.oid = ix.indexrelid
+ WHERE t.relname = '$table'")
+ [ -n "$out" ] || out="none"
+ echo "$out"
+}
+
+sample_peak() {
+ # Sample CPU / RSS of the postmaster tree for $DURATION+5 seconds.
+ # Writes "peak_cpu_pct,peak_rss_mib" to the given outfile. Portable across
+ # Linux / FreeBSD (falls back to pgrep + per-pid ps where --ppid isn't
+ # available). Returns 'NA,NA' if the sampler can't collect useful data.
+ local outfile=$1 v=$2
+ local leader
+ leader=$(postmaster_pid "$v")
+ [ -z "$leader" ] && { echo "NA,NA" > "$outfile"; return; }
+ local dur=$(( DURATION + 5 ))
+ (
+ local max_cpu=0
+ local max_rss=0
+ local t0=$(date +%s)
+ while :; do
+ # Children of the leader + the leader itself.
+ local pids
+ pids=$( (pgrep -P "$leader" 2>/dev/null; echo "$leader") | tr '\n' ' ')
+ local sample
+ sample=$(ps -o pcpu=,rss= -p $pids 2>/dev/null | \
+ awk '{cpu+=$1; rss+=$2} END{printf "%.1f %d\n", cpu+0, rss+0}')
+ local c r
+ read -r c r <<<"$sample"
+ if [ -n "${c:-}" ] && [ -n "${r:-}" ]; then
+ awk -v m="$max_cpu" -v c="$c" 'BEGIN{exit !(c>m)}' && max_cpu=$c
+ [ "$r" -gt "$max_rss" ] 2>/dev/null && max_rss=$r
+ fi
+ local now=$(date +%s)
+ [ $((now - t0)) -ge "$dur" ] && break
+ sleep 1
+ done
+ local rss_mib=$(( max_rss / 1024 ))
+ echo "$max_cpu,$rss_mib" > "$outfile"
+ ) &
+ echo $!
+}
+
+run_one() {
+ local v=$1 workload=$2 script=$3 table=${4:-siu_table} extra_set=${5:-}
+
+ local wal_start wal_end hot_start hot_end total_start total_end tps lat
+ local siu_start siu_end
+ local bloat_before bloat_after idx_before idx_after
+ local per_idx_before per_idx_after
+ read -r bloat_before idx_before <<<"$(bloat_stats "$v" "$table" | tr , ' ')"
+ per_idx_before=$(per_index_sizes "$v" "$table")
+
+ wal_start=$(psql_as "$v" -Atc "SELECT pg_current_wal_lsn()::text")
+ hot_start=$(psql_as "$v" -Atc "SELECT coalesce(n_tup_hot_upd,0) FROM pg_stat_user_tables WHERE relname='$table'")
+ siu_start=$(siu_count "$v" "$table")
+ total_start=$(psql_as "$v" -Atc "SELECT coalesce(n_tup_upd,0) FROM pg_stat_user_tables WHERE relname='$table'")
+
+ local out="$LOGDIR/${v}_${workload}.log"
+ local cpu_rss_file=$LOGDIR/${v}_${workload}.cpu
+ local sampler_pid
+ sampler_pid=$(sample_peak "$cpu_rss_file" "$v")
+
+ set +e
+ case "$workload" in
+ simple_update)
+ pgbench_as "$v" -N -c "$CLIENTS" -j "$THREADS" -T "$DURATION" \
+ -n postgres >"$out" 2>&1
+ ;;
+ wide_*)
+ # build the SET clause from extra_set which is "c1=:v,c2=:v,..."
+ pgbench_as "$v" -f <(sed "s/:wide_set_clause/$extra_set/" "$script") \
+ -c "$CLIENTS" -j "$THREADS" -T "$DURATION" \
+ -D "scale=$SCALE" -n postgres >"$out" 2>&1
+ ;;
+ read_indexscan)
+ # read-only; pass the row count so the script can pick random keys
+ pgbench_as "$v" -f "$script" -c "$CLIENTS" -j "$THREADS" -T "$DURATION" \
+ -D "rows=$((SCALE * 100000))" -n postgres >"$out" 2>&1
+ ;;
+ *)
+ pgbench_as "$v" -f "$script" -c "$CLIENTS" -j "$THREADS" -T "$DURATION" \
+ -n postgres >"$out" 2>&1
+ ;;
+ esac
+ set -e
+
+ wait "$sampler_pid" 2>/dev/null || true
+ local cpu_rss
+ cpu_rss=$(cat "$cpu_rss_file" 2>/dev/null || echo "NA,NA")
+
+ tps=$(awk '/tps = /{print $3; exit}' "$out")
+ lat=$(awk '/latency average = /{print $4; exit}' "$out")
+ tps=${tps:-NA}
+ lat=${lat:-NA}
+
+ wal_end=$(psql_as "$v" -Atc "SELECT pg_current_wal_lsn()::text")
+ hot_end=$(psql_as "$v" -Atc "SELECT coalesce(n_tup_hot_upd,0) FROM pg_stat_user_tables WHERE relname='$table'")
+ siu_end=$(siu_count "$v" "$table")
+ total_end=$(psql_as "$v" -Atc "SELECT coalesce(n_tup_upd,0) FROM pg_stat_user_tables WHERE relname='$table'")
+
+ local wal_bytes
+ wal_bytes=$(psql_as "$v" -Atc "SELECT pg_wal_lsn_diff('$wal_end'::pg_lsn, '$wal_start'::pg_lsn)::bigint")
+
+ # Capture a WAL record-type histogram for this workload. pg_waldump's
+ # --stats=record output is rich (~60 lines) so stash it in LOGDIR
+ # rather than trying to fold into the CSV. Tolerate failures: if the
+ # segment containing wal_start has been recycled (rare with
+ # max_wal_size=4GB but possible under long chained runs), we emit a
+ # note and move on instead of aborting the whole run.
+ local wal_stats_file=$LOGDIR/${v}_${workload}.walstats
+ LD_LIBRARY_PATH="$(LD_of "$v")" "$(bin_of "$v")/pg_waldump" \
+ --stats=record -p "$BENCH/_data_$v/pg_wal" \
+ --start="$wal_start" --end="$wal_end" \
+ > "$wal_stats_file" 2> "${wal_stats_file}.err" \
+ || echo "pg_waldump unavailable for this range; see ${wal_stats_file}.err" > "$wal_stats_file"
+
+ read -r bloat_after idx_after <<<"$(bloat_stats "$v" "$table" | tr , ' ')"
+ per_idx_after=$(per_index_sizes "$v" "$table")
+
+ local hot=$((hot_end - hot_start))
+ local siu=$((siu_end - siu_start))
+ local tot=$((total_end - total_start))
+ local classic_hot=$((hot - siu))
+ local non_hot=$((tot - hot))
+
+ printf '%s,%s,%s,%s,%d,%d,%d,%d,%s,%s,%s,%s,%s,%s,%s,%s\n' \
+ "$v" "$workload" "$tps" "$lat" "$classic_hot" "$siu" "$non_hot" "$tot" \
+ "$wal_bytes" \
+ "$bloat_before" "$bloat_after" \
+ "$idx_before" "$idx_after" \
+ "$cpu_rss" "$per_idx_before" "$per_idx_after" >> "$OUT"
+ printf ' %-8s %-14s tps=%10s lat=%6s classic_hot=%7d hi=%7d non_hot=%7d tot=%-7d wal=%12s bloat=%s->%s idx=%s->%s cpu_rss=%s\n' \
+ "$v" "$workload" "$tps" "$lat" "$classic_hot" "$siu" "$non_hot" "$tot" "$wal_bytes" \
+ "$bloat_before" "$bloat_after" "$idx_before" "$idx_after" "$cpu_rss"
+}
+
+build_wide_set_clause() {
+ # emit e.g. "c1=:v,c2=:v,...,cN=:v" for first N cols.
+ local n=$1
+ if [ "$n" -eq 0 ]; then
+ # No indexed-col update; touch a non-indexed column (id % 1 so it's a no-op)
+ echo "id=id"
+ return
+ fi
+ local clauses=""
+ for i in $(seq 1 "$n"); do
+ [ -n "$clauses" ] && clauses+=","
+ clauses+="c$i=:v"
+ done
+ echo "$clauses"
+}
+
+for v in master tepid; do
+ echo "--- variant: $v"
+ stop_pg "$v" || true
+ start_pg "$v"
+ setup_schemas "$v"
+
+ run_one "$v" simple_update '' pgbench_accounts
+ reset_state "$v" siu_table
+ run_one "$v" hot_indexed_update "$BENCH/scripts/hot_indexed_update.sql" siu_table
+ reset_state "$v" siu_table
+ run_one "$v" hot_indexed_mixed "$BENCH/scripts/hot_indexed_mixed.sql" siu_table
+ reset_state "$v" siu_table
+ run_one "$v" read_indexscan "$BENCH/scripts/read_indexscan.sql" siu_table
+
+ for n in ${WIDE_STEPS//,/ }; do
+ reset_state "$v" wide_table
+ run_one "$v" "wide_${n}" "$BENCH/scripts/wide_update.sql" wide_table \
+ "$(build_wide_set_clause "$n")"
+ done
+
+ stop_pg "$v"
+done
+
+echo "=== results: $OUT"
+column -t -s, "$OUT" | head -50
diff --git a/src/test/benchmarks/siu/scripts/soak.sh b/src/test/benchmarks/siu/scripts/soak.sh
new file mode 100755
index 00000000000..6d127f1c012
--- /dev/null
+++ b/src/test/benchmarks/siu/scripts/soak.sh
@@ -0,0 +1,128 @@
+#!/usr/bin/env bash
+# tepid soak: run hot_indexed_update for $DURATION seconds on each variant, sampling
+# TPS / HOT-rate / WAL volume / table+index bloat every $SAMPLE seconds.
+# Emits a CSV with one sample row per tick per variant.
+set -euo pipefail
+
+BENCH=${BENCH:-/scratch/siu-bench}
+SCALE=${SCALE:-50}
+CLIENTS=${CLIENTS:-16}
+THREADS=${THREADS:-8}
+DURATION=${DURATION:-900} # 15 minutes
+SAMPLE=${SAMPLE:-60} # every 60 s
+PORT=${PORT:-57503}
+SHARED_BUFFERS=${SHARED_BUFFERS:-2GB}
+
+TS=$(date -u +%Y%m%dT%H%M%SZ)
+OUT=$BENCH/results/soak_$TS.csv
+LOGDIR=$BENCH/logs/soak_$TS
+mkdir -p "$LOGDIR"
+echo "variant,t_secs,tps_instant,hot_pct_instant,heap_pages,index_bytes,wal_bytes_since_start,n_dead_tup" > "$OUT"
+echo "=== soak $TS -> $OUT"
+
+bin_of() { echo "$BENCH/$1/usr/local/pgsql/bin"; }
+LD_of() { local b=$BENCH/$1/usr/local/pgsql; [ -d "$b/lib64" ] && echo "$b/lib64" || echo "$b/lib"; }
+
+psql_as() { local v=$1; shift; LD_LIBRARY_PATH="$(LD_of "$v")" "$(bin_of "$v")/psql" -h /tmp -p "$PORT" -U postgres -X "$@"; }
+pgbench_as() { local v=$1; shift; LD_LIBRARY_PATH="$(LD_of "$v")" "$(bin_of "$v")/pgbench" -h /tmp -p "$PORT" -U postgres "$@"; }
+
+start_pg() {
+ local v=$1 datadir=$BENCH/_data_$v
+ [ -d "$datadir" ] && find "$datadir" -mindepth 1 -delete && rmdir "$datadir"
+ mkdir -p "$datadir"
+ LD_LIBRARY_PATH="$(LD_of "$v")" "$(bin_of "$v")/initdb" -D "$datadir" -U postgres >"$LOGDIR/initdb_$v.log" 2>&1
+ cat >> "$datadir/postgresql.conf" <<EOF
+shared_buffers = $SHARED_BUFFERS
+work_mem = 32MB
+max_wal_size = 8GB
+synchronous_commit = on
+checkpoint_timeout = 10min
+wal_level = replica
+autovacuum = on
+autovacuum_naptime = 10s
+autovacuum_vacuum_threshold = 50
+autovacuum_vacuum_scale_factor = 0.1
+port = $PORT
+EOF
+ LD_LIBRARY_PATH="$(LD_of "$v")" "$(bin_of "$v")/pg_ctl" -D "$datadir" \
+ -o "-p $PORT" -l "$LOGDIR/pg_$v.log" start >/dev/null
+ sleep 2
+}
+
+stop_pg() {
+ local v=$1
+ LD_LIBRARY_PATH="$(LD_of "$v")" "$(bin_of "$v")/pg_ctl" -D "$BENCH/_data_$v" stop -m fast >/dev/null 2>&1 || true
+}
+
+setup() {
+ local v=$1 rows=$((SCALE * 100000))
+ psql_as "$v" <<SQL
+DROP TABLE IF EXISTS siu_table;
+CREATE TABLE siu_table(a int PRIMARY KEY, b int, c int, d int, e text);
+CREATE INDEX siu_b ON siu_table(b);
+CREATE INDEX siu_c ON siu_table(c);
+CREATE INDEX siu_d ON siu_table(d);
+INSERT INTO siu_table
+ SELECT i, i, i, i, repeat('x', 20) FROM generate_series(1, $rows) AS i;
+VACUUM (ANALYZE) siu_table;
+SQL
+}
+
+run_soak() {
+ local v=$1
+ echo "--- soak $v for ${DURATION}s, sampling every ${SAMPLE}s"
+ stop_pg "$v" || true
+ start_pg "$v"
+ setup "$v"
+ local wal0
+ wal0=$(psql_as "$v" -Atc "SELECT pg_current_wal_lsn()::text")
+ local hot0 tot0
+ hot0=$(psql_as "$v" -Atc "SELECT coalesce(n_tup_hot_upd,0) FROM pg_stat_user_tables WHERE relname='siu_table'")
+ tot0=$(psql_as "$v" -Atc "SELECT coalesce(n_tup_upd,0) FROM pg_stat_user_tables WHERE relname='siu_table'")
+ local prev_hot=$hot0 prev_tot=$tot0
+
+ # Drive pgbench in the background; sampler in foreground.
+ pgbench_as "$v" -f "$BENCH/scripts/hot_indexed_update.sql" \
+ -c "$CLIENTS" -j "$THREADS" -T "$DURATION" \
+ -P "$SAMPLE" -n postgres >"$LOGDIR/pgbench_$v.log" 2>&1 &
+ local pgb=$!
+
+ local t=0
+ while [ "$t" -lt "$DURATION" ]; do
+ sleep "$SAMPLE"
+ t=$((t + SAMPLE))
+ local now_hot now_tot wal_now wal_bytes heap_pages idx_bytes n_dead
+ now_hot=$(psql_as "$v" -Atc "SELECT coalesce(n_tup_hot_upd,0) FROM pg_stat_user_tables WHERE relname='siu_table'")
+ now_tot=$(psql_as "$v" -Atc "SELECT coalesce(n_tup_upd,0) FROM pg_stat_user_tables WHERE relname='siu_table'")
+ wal_now=$(psql_as "$v" -Atc "SELECT pg_current_wal_lsn()::text")
+ wal_bytes=$(psql_as "$v" -Atc "SELECT pg_wal_lsn_diff('$wal_now'::pg_lsn, '$wal0'::pg_lsn)::bigint")
+ heap_pages=$(psql_as "$v" -Atc "SELECT pg_table_size('siu_table')/8192")
+ idx_bytes=$(psql_as "$v" -Atc "SELECT pg_indexes_size('siu_table')")
+ n_dead=$(psql_as "$v" -Atc "SELECT coalesce(n_dead_tup,0) FROM pg_stat_user_tables WHERE relname='siu_table'")
+
+ local d_hot=$((now_hot - prev_hot))
+ local d_tot=$((now_tot - prev_tot))
+ local tps_i hot_pct
+ if [ "$d_tot" -gt 0 ]; then
+ tps_i=$(awk -v d="$d_tot" -v s="$SAMPLE" 'BEGIN{printf "%.1f", d/s}')
+ hot_pct=$(awk -v h="$d_hot" -v t="$d_tot" 'BEGIN{printf "%.1f", 100*h/t}')
+ else
+ tps_i=0; hot_pct=0
+ fi
+ printf '%s,%d,%s,%s,%s,%s,%s,%s\n' "$v" "$t" "$tps_i" "$hot_pct" "$heap_pages" "$idx_bytes" "$wal_bytes" "$n_dead" >> "$OUT"
+ printf ' %-6s t=%-5d tps=%8s hot=%-5s%% heap_pgs=%-7s idx=%-12s wal=%-12s dead=%s\n' \
+ "$v" "$t" "$tps_i" "$hot_pct" "$heap_pages" "$idx_bytes" "$wal_bytes" "$n_dead"
+ prev_hot=$now_hot
+ prev_tot=$now_tot
+ done
+
+ wait "$pgb" 2>/dev/null || true
+ stop_pg "$v"
+}
+
+for v in master tepid; do
+ run_soak "$v"
+done
+
+echo "=== soak results: $OUT"
+column -t -s, "$OUT" | head -80
diff --git a/src/test/benchmarks/siu/scripts/wide_update.sql b/src/test/benchmarks/siu/scripts/wide_update.sql
new file mode 100644
index 00000000000..c2c2ff14ac4
--- /dev/null
+++ b/src/test/benchmarks/siu/scripts/wide_update.sql
@@ -0,0 +1,7 @@
+-- Wide-table workload. The setup script creates a table with WIDE_COLS integer
+-- columns, each separately btree-indexed. The workload UPDATEs a
+-- configurable number of those indexed columns per transaction
+-- (WIDE_UPDCOLS env var) on a random row.
+\set rid random(1, :scale * 1000)
+\set v random(1, 1000000000)
+UPDATE wide_table SET :wide_set_clause WHERE id = :rid;
--
2.50.1
view thread (8+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected]
Subject: Re: Tepid: selective index updates for heap relations
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox