From: Manu <manuelreyesbravo@gmail.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Zhijie Hou <houzhijie22@gmail.com>
Cc: shihao zhong <zhong950419@gmail.com>
Cc: Takeshi Ideriha <iderihatakeshi@gmail.com>
Cc: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Cc: alpkeles99@gmail.com
Cc: pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction
Date: Mon, 26 Oct 2026 12:41:19 -0300
Message-ID: <179302927929.1603125.13910327766898839560@gmail.com> (raw)
In-Reply-To: <at6lcAt8xkkuUKSm@paquier.xyz>
References: <at6lcAt8xkkuUKSm@paquier.xyz>
Hi Michael,
On Mon, Oct 26, 2026 at 09:57:20AM +0900, Michael Paquier wrote:
> Thanks for the reviews. Applied and backpatched this part down to
> v14.
Thanks for taking care of it. The attached series adds the second
spot on top of e7f4463e869, as shihao asked upthread.
0001 sets HEAP_XMAX_INVALID in the branch of
HeapTupleSatisfiesVacuumHorizon() where the inserter is another
backend that is still running, once the updater has aborted. It
skips xmax == xmin and MultiXacts, and it asks the clog first and
then the proc array, so it does not open the window Zhijie found in
my clog-only version. It includes shihao's isolation spec, which
loses the pg_class row without the fix. 0002 is Zhijie's test for
that window.
VACUUM with your scripts, 64 holders, one run of 12 interleaved
rounds on 2026-10-25, ratios to HEAD:
delete.sql: 1.00x
subxact_updates.sql: 1.00x (with v2's check: 1.51x)
aborted updates, first VACUUM: 1.07x, same as v2's check
aborted updates, following VACUUMs: 1.00x
"Aborted updates" is a new script where each subtransaction rolls
back after its UPDATE. It is the only case where 0001 asks the proc
array, once per tuple, in the VACUUM that sets the hint. The
measured heapam_visibility.c matches 0001 except for two lines of a
test overlay. Method and the script are in measurements.txt.
0001 and 0002 apply with git am on master and REL_19_STABLE. On 14
to 18 the code applies as is, but the isolation_schedule line needs
another neighbor. 0002 only goes down to 17, where the injection
points it uses exist. I ran make check and the isolation and
injection_points suites on master and 19, and make check and
isolation on 14.
The xmin != xmax check and the race are Zhijie's, the MultiXact skip
is yours, and the spec is shihao's.
Regards,
Manu
Attachments:download all (3 files, .tar.gz)v3-0001-Set-HEAP_XMAX_INVALID-for-aborted-updates-of-tupl.patch (text/x-patch, 6.4K) download
From 4c12deb22a1516c9b8af4f279ba8ad7f8d98dc17 Mon Sep 17 00:00:00 2001
From: Manu <manuelreyesbravo@gmail.com>
Date: Mon, 26 Oct 2026 12:00:00 -0300
Subject: [PATCH v3 1/2] Set HEAP_XMAX_INVALID for aborted updates of tuples
with a running inserter
e7f4463e869 made HeapTupleSatisfiesVacuumHorizon() set HEAP_XMAX_INVALID
on a tuple inserted by the current transaction and updated by one of its
subtransactions that rolled back. The same tuple is seen by other
backends while the inserting transaction is still running, and there
the hint was not set either. rewriteheap.c then treats the tuple as
updated, so a VACUUM FULL or CLUSTER of a catalog run by another session
loses the row: for example, the pg_class row of a table created in an
open transaction, after an ALTER TABLE on it was rolled back to a
savepoint.
The hint is now also set in the branch where the inserter is in
progress, if the updater has aborted. This is skipped when xmax is
equal to xmin, as xmax is then in progress as well, and when xmax is a
MultiXact. The clog is asked first, as a cheap filter for an updater
that is still running, and the proc array has to confirm that the
updater is gone: an abort reaches the clog before the transaction
leaves the proc array, and until then the new version of the tuple,
whose xmin is this xmax, is still in progress. Setting the hint from
the clog alone would leave the old version marked as not updated while
its successor is still in progress.
The xmin != xmax check and the race with the proc array come from
Zhijie Hou, and the MultiXact skip from Michael Paquier. The isolation
test is from Shihao Zhong.
Author: Manu <manuelreyesbravo@gmail.com>
Author: Shihao Zhong <zhong950419@gmail.com>
Reviewed-by: Zhijie Hou <houzhijie22@gmail.com>
Discussion: https://postgr.es/m/19760-8f9a0eba17bc8c92@postgresql.org
Backpatch-through: 14
---
src/backend/access/heap/heapam_visibility.c | 29 +++++++++++++++++
.../cluster-aborted-subxact-update.out | 20 ++++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/cluster-aborted-subxact-update.spec | 32 +++++++++++++++++++
4 files changed, 82 insertions(+)
create mode 100644 src/test/isolation/expected/cluster-aborted-subxact-update.out
create mode 100644 src/test/isolation/specs/cluster-aborted-subxact-update.spec
diff --git a/src/backend/access/heap/heapam_visibility.c b/src/backend/access/heap/heapam_visibility.cindex 905444a..681aaed 100644--- a/src/backend/access/heap/heapam_visibility.c+++ b/src/backend/access/heap/heapam_visibility.c@@ -1184,6 +1184,35 @@ HeapTupleSatisfiesVacuumHorizon(HeapTuple htup, Buffer buffer, TransactionId *de
}
else if (TransactionIdIsInProgress(HeapTupleHeaderGetRawXmin(tuple)))
{
+ /*+ * If the updater of the tuple has already aborted (a rolled-back+ * subtransaction of the inserter), remember that, as callers like+ * rewriteheap.c rely on HEAP_XMAX_INVALID to tell a live tuple+ * from an updated one.+ */+ if (!(tuple->t_infomask & HEAP_XMAX_INVALID) &&+ !(tuple->t_infomask & HEAP_XMAX_IS_MULTI) &&+ !HEAP_XMAX_IS_LOCKED_ONLY(tuple->t_infomask) &&+ !HeapTupleHeaderIsOnlyLocked(tuple))+ {+ TransactionId xmax = HeapTupleHeaderGetUpdateXid(tuple);++ /*+ * xmax == xmin: xmax is in progress too, nothing to do.+ * The clog is only a cheap filter here: a running xmax is not+ * aborted there. An abort reaches the clog before the xid+ * leaves the proc array, and the new version of the tuple,+ * whose xmin is this xmax, is in progress until then. So the+ * proc array must confirm it, as the note at the top of this+ * file asks.+ */+ if (!TransactionIdEquals(xmax, HeapTupleHeaderGetRawXmin(tuple)) &&+ TransactionIdDidAbort(xmax) &&+ !TransactionIdIsInProgress(xmax))+ SetHintBits(tuple, buffer, HEAP_XMAX_INVALID,+ InvalidTransactionId);+ }+
/*
* It'd be possible to discern between INSERT/DELETE in progress
* here by looking at xmax - but that doesn't seem beneficial for
diff --git a/src/test/isolation/expected/cluster-aborted-subxact-update.out b/src/test/isolation/expected/cluster-aborted-subxact-update.outnew file mode 100644index 0000000..a3448c2--- /dev/null+++ b/src/test/isolation/expected/cluster-aborted-subxact-update.out@@ -0,0 +1,20 @@+Parsed test spec with 2 sessions++starting permutation: create_and_rollback_alter vacuum_full_pg_class commit check_pg_class+step create_and_rollback_alter: + BEGIN;+ CREATE TABLE vacfull_tab (a int);+ SAVEPOINT sp;+ ALTER TABLE vacfull_tab ADD COLUMN b int;+ ROLLBACK TO sp;++step vacuum_full_pg_class: VACUUM FULL pg_class;+step commit: COMMIT;+step check_pg_class: + SELECT relname, relnatts FROM pg_class WHERE relname = 'vacfull_tab';++relname |relnatts+-----------+--------+vacfull_tab| 1+(1 row)+diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_scheduleindex bf9a203..eee40f5 100644--- a/src/test/isolation/isolation_schedule+++ b/src/test/isolation/isolation_schedule@@ -122,6 +122,7 @@ test: plpgsql-toast
test: cluster-conflict
test: cluster-conflict-partition
test: cluster-toast-value-reuse
+test: cluster-aborted-subxact-update
test: truncate-conflict
test: serializable-parallel
test: serializable-parallel-2
diff --git a/src/test/isolation/specs/cluster-aborted-subxact-update.spec b/src/test/isolation/specs/cluster-aborted-subxact-update.specnew file mode 100644index 0000000..e2000ed--- /dev/null+++ b/src/test/isolation/specs/cluster-aborted-subxact-update.spec@@ -0,0 +1,32 @@+# VACUUM FULL must keep a row whose inserting transaction is still running,+# when a rolled-back subtransaction of that transaction updated the row.+#+# VACUUM FULL of a user table would wait for the inserter, so this uses+# pg_class. The inserter creates a table and adds a column in a subtransaction+# that it rolls back, which leaves an updated pg_class row whose updater+# aborted.++teardown+{+ DROP TABLE IF EXISTS vacfull_tab;+}++session creator+step create_and_rollback_alter+{+ BEGIN;+ CREATE TABLE vacfull_tab (a int);+ SAVEPOINT sp;+ ALTER TABLE vacfull_tab ADD COLUMN b int;+ ROLLBACK TO sp;+}+step commit { COMMIT; }++session vacuumer+step vacuum_full_pg_class { VACUUM FULL pg_class; }+step check_pg_class+{+ SELECT relname, relnatts FROM pg_class WHERE relname = 'vacfull_tab';+}++permutation create_and_rollback_alter vacuum_full_pg_class commit check_pg_class--
2.56.0
v3-0002-Test-the-clog-proc-array-window-in-HeapTupleSatis.patch (text/x-patch, 3.6K) download
From 378472c1b3b2ca9f756a7bb09032db5729b73801 Mon Sep 17 00:00:00 2001
From: Zhijie Hou <houzhijie22@gmail.com>
Date: Mon, 26 Oct 2026 12:00:00 -0300
Subject: [PATCH v3 2/2] Test the clog/proc array window in
HeapTupleSatisfiesVacuumHorizon()
A HOT update in a subtransaction, released, then a ROLLBACK stopped
after the clog update but before the proc array update, with the
injection point transaction-abort-after-clog. In that window ANALYZE
must not mark the old tuple HEAP_XMAX_INVALID while the new heap-only
tuple is still in progress, or brin_summarize_range() fails with
"failed to find parent tuple for heap-only tuple".
Author: Zhijie Hou <houzhijie22@gmail.com>
Discussion: https://postgr.es/m/19760-8f9a0eba17bc8c92@postgresql.org
Backpatch-through: 17
---
.../expected/multixact-aborted-updater.out | 30 +++++++++++++++++++
.../specs/multixact-aborted-updater.spec | 19 ++++++++++++
2 files changed, 49 insertions(+)
diff --git a/src/test/modules/injection_points/expected/multixact-aborted-updater.out b/src/test/modules/injection_points/expected/multixact-aborted-updater.outindex 4075071..abc3d0a 100644--- a/src/test/modules/injection_points/expected/multixact-aborted-updater.out+++ b/src/test/modules/injection_points/expected/multixact-aborted-updater.out@@ -82,3 +82,33 @@ id|filler
2|initial
(1 row)
++starting permutation: s1ddl s1abort s3brin wake+step s1ddl: + BEGIN;+ INSERT INTO mxact_abort VALUES (2, 'v1');+ SAVEPOINT sp;+ UPDATE mxact_abort SET filler = 'v2' WHERE id = 2;+ RELEASE SAVEPOINT sp;++step s1abort: ROLLBACK; <waiting ...>+step s3brin: + SELECT brin_desummarize_range('mxact_brin_idx', 0);+ ANALYZE mxact_abort;+ SELECT brin_summarize_range('mxact_brin_idx', 0);++brin_desummarize_range+----------------------+ +(1 row)++brin_summarize_range+--------------------+ 1+(1 row)++step wake: + SELECT FROM injection_points_detach('transaction-abort-after-clog');+ SELECT FROM injection_points_wakeup('transaction-abort-after-clog');++step s1abort: <... completed>diff --git a/src/test/modules/injection_points/specs/multixact-aborted-updater.spec b/src/test/modules/injection_points/specs/multixact-aborted-updater.specindex 98d265a..f4c6fc9 100644--- a/src/test/modules/injection_points/specs/multixact-aborted-updater.spec+++ b/src/test/modules/injection_points/specs/multixact-aborted-updater.spec@@ -20,6 +20,7 @@ setup
CREATE EXTENSION injection_points;
CREATE TABLE mxact_abort (id int PRIMARY KEY, filler text);
+ CREATE INDEX mxact_brin_idx ON mxact_abort USING brin (id);
INSERT INTO mxact_abort VALUES (1, 'initial');
}
@@ -38,6 +39,13 @@ setup {
}
step s1begin { BEGIN; }
step s1update { UPDATE mxact_abort SET filler = 's1' WHERE id = 1; }
+step s1ddl {+ BEGIN;+ INSERT INTO mxact_abort VALUES (2, 'v1');+ SAVEPOINT sp;+ UPDATE mxact_abort SET filler = 'v2' WHERE id = 2;+ RELEASE SAVEPOINT sp;+}
step s1abort { ROLLBACK; }
# Transaction 2 locks the row in KEY SHARE mode, to cause a multixact to be
@@ -52,6 +60,11 @@ step s3update { UPDATE mxact_abort SET filler = 's3' WHERE id = 1; }
step s3delete { DELETE FROM mxact_abort WHERE id = 1; }
step s3keyupdate { UPDATE mxact_abort SET id = 2 WHERE id = 1; }
step s3check { SELECT * FROM mxact_abort ORDER BY id; }
+step s3brin {+ SELECT brin_desummarize_range('mxact_brin_idx', 0);+ ANALYZE mxact_abort;+ SELECT brin_summarize_range('mxact_brin_idx', 0);+}
session waker
step wake {
@@ -105,3 +118,9 @@ permutation
s3keyupdate
wake
s3check
++permutation+ s1ddl+ s1abort+ s3brin+ wake--
2.56.0
measurements.txt (text/plain, 3.4K) download
BUG #19760, part 2: VACUUM cost of the second HEAP_XMAX_INVALID hint
One run on 2026-10-25, 12 interleaved rounds per build, with Michael's
scripts from upthread (setup.sql, holders.sql, delete.sql,
subxact_updates.sql, vacuum.sql), unchanged, plus subxact_aborts.sql
below. Same harness and method as the measurements.txt of 2026-10-22.
Base: master 871dee0ccbe, the base of those measurements.
Builds: configure --enable-depend, CFLAGS=-O2, no assertions.
HEAD master
v2+0001 v2-0001 from upthread + nocfbot-0001 (xmin != xmax,
MultiXact skip): the second hint with v2's check,
!TransactionIdIsInProgress() && !TransactionIdDidCommit()
old 0002 + the clog-only TransactionIdDidAbort() check
v3 + the fixed check: TransactionIdDidAbort(xmax) &&
!TransactionIdIsInProgress(xmax)
The heapam_visibility.c of "v3" is the one of v3-0001 on today's
master, byte for byte, except for two lines of a test-only injection
point that these builds do not compile in (the object code differs only
in the __LINE__ constants of two elog() calls).
Machine: Ryzen AI 9 HX 370 laptop, runs pinned to 4 Zen 5 cores
(8 threads); data directory on tmpfs. Server settings:
max_connections = 200, autovacuum = off, max_wal_size = 256MB.
64 holders. Metric: per sample, the mean of the 9 VACUUM times of
vacuum.sql (or the first one / the other 8, where said). Reported:
median of the samples, ratio to HEAD, bootstrap 95% CI of the ratio of
medians (4000 resamples). A sample is dropped and redone if a CPU
calibration loop run before and after it is more than 8% slower than
its idle time.
delete.sql:
build n median ms x HEAD 95% CI
HEAD 12 418.9 1.00
v2+0001 11 419.0 1.00 0.99..1.01
old 0002 11 417.8 1.00 0.99..1.00
v3 12 419.7 1.00 0.99..1.01
subxact_updates.sql:
build n median ms x HEAD 95% CI
HEAD 9 923.6 1.00
v2+0001 9 1396.6 1.51 1.50..1.55
old 0002 9 931.5 1.01 0.99..1.03
v3 9 924.8 1.00 0.99..1.02
subxact_aborts.sql, first VACUUM only:
build n median ms x HEAD 95% CI
HEAD 12 5869.6 1.00
v2+0001 12 6295.6 1.07 1.07..1.07
old 0002 12 5877.2 1.00 1.00..1.00
v3 12 6294.1 1.07 1.07..1.07
subxact_aborts.sql, VACUUMs 2-9:
build n median ms x HEAD 95% CI
HEAD 12 5430.5 1.00
v2+0001 12 5431.9 1.00 1.00..1.00
old 0002 12 5434.1 1.00 1.00..1.00
v3 12 5430.5 1.00 1.00..1.00
In subxact_updates.sql xmax is a released subtransaction that is still
running: it is not aborted in the clog, so v3 never asks the proc
array. In subxact_aborts.sql xmax has aborted: v3 pays the proc array
check once per tuple, in the first VACUUM, which sets the hint.
subxact_updates.sql had 3 rounds with no clean sample (n = 9).
subxact_aborts.sql (like subxact_updates.sql, but each subtransaction
rolls back):
\if :{?n}
\else
\set n 500000
\endif
BEGIN;
INSERT INTO t SELECT g, 0 FROM generate_series(1, :n) g;
DO $$
DECLARE
r record;
BEGIN
FOR r IN SELECT ctid AS c FROM t LOOP
BEGIN
UPDATE t SET v = 1 WHERE ctid = r.c;
RAISE EXCEPTION 'roll back this subtransaction';
EXCEPTION WHEN others THEN NULL;
END;
END LOOP;
END $$;
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-bugs@postgresql.org
Cc: manuelreyesbravo@gmail.com, michael@paquier.xyz, houzhijie22@gmail.com, zhong950419@gmail.com, iderihatakeshi@gmail.com, ayushtiwari.slg01@gmail.com, alpkeles99@gmail.com
Subject: Re: BUG #19760: CLUSTER loses a row the transaction inserted and then updated in a rolled-back subtransaction
In-Reply-To: <179302927929.1603125.13910327766898839560@gmail.com>
References: <at6lcAt8xkkuUKSm@paquier.xyz> <179302927929.1603125.13910327766898839560@gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox