agora inbox for pgsql-bugs@postgresql.org
help / color / mirror / Atom feedBUG #19620: pg_class index corruption caused by statement_timeout during VACUUM FULL
4+ messages / 4 participants
[nested] [flat]
* BUG #19620: pg_class index corruption caused by statement_timeout during VACUUM FULL
@ 2026-08-15 02:37 PG Bug reporting form <noreply@postgresql.org>
0 siblings, 1 reply; 4+ messages in thread
From: PG Bug reporting form @ 2026-08-15 02:37 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org; +Cc: hackerzheng666@gmail.com
The following bug has been logged on the website:
Bug reference: 19620
Logged by: Zheng Hacker
Email address: hackerzheng666@gmail.com
PostgreSQL version: 19beta3
Operating system: Linux x86_64
Description:
SUMMARY
On PostgreSQL 18devel (git master cc1ea71), a single backend that
repeatedly runs catalog-churning DDL/maintenance under a short
statement_timeout drives the system-catalog index
pg_class_relname_nsp_index into a corrupt state: one btree leaf page
acquires two index tuples pointing to the same pg_class heap TID.
A subsequent CREATE TABLE touches that leaf page, a btree
simple-delete/dedup pass collects the two identical TIDs, and the
cassert assertion in index_delete_sort_cmp (heapam.c:8668) fires.
On a production (non-cassert) build, the corrupt catalog index is
silent and can produce wrong catalog lookups (we separately observe
the Assert(relid == targetRelId) in relcache.c:1138 from the same
corruption when a pg_class OID-index scan returns the wrong row).
KEY DISTINCTION from known reports (#18490, #17386, #17255): this
reproducer is purely single-session, single-backend -- all parallelism
disabled (max_worker_processes=0), no concurrent connections. The
trigger is statement cancellation (CHECK_FOR_INTERRUPTS) during
catalog-index maintenance, not a concurrency race.
VERSION / BUILD
PostgreSQL master (ahead of 19beta3), git commit cc1ea71
("Remove unnecessary list_free() calls in OpenTableList()")
Build: clang -O1 -g --enable-cassert (no sanitizers, no sancov),
x86_64 Linux.
First observed on an ASan+sancov build; confirmed on this clean
build to rule out instrumentation artifacts.
REPRODUCTION
1) Server setup:
initdb -D $PGDATA --no-locale -A trust
cat >> $PGDATA/postgresql.conf << 'EOF'
fsync = off
debug_parallel_query = off
max_parallel_workers = 0
max_parallel_maintenance_workers = 0
max_worker_processes = 0
EOF
pg_ctl -D $PGDATA -l $PGDATA/server.log start
createdb testdb
2) Save as repro.sql:
CREATE TABLE t1 (i int, t text);
INSERT INTO t1 SELECT g, md5(g::text) FROM generate_series(1,100) g;
CREATE INDEX t1_i ON t1(i);
CREATE INDEX t1_t ON t1(t);
CREATE TABLE t2 (i int PRIMARY KEY, a int[], p point);
INSERT INTO t2 SELECT g, array[g,g+1], point(g,g) FROM
generate_series(1,100) g;
CREATE INDEX t2_hash ON t2 USING hash(i);
CREATE INDEX t2_gin ON t2 USING gin(a);
CREATE INDEX t2_gist ON t2 USING gist(p);
CREATE TABLE t3 (k int, v text);
INSERT INTO t3 SELECT g, repeat('x',200) FROM generate_series(1,100) g;
CREATE INDEX t3_k ON t3(k);
CREATE TABLE t4 (a int) PARTITION BY LIST(a);
CREATE TABLE t4p1 PARTITION OF t4 FOR VALUES IN (1);
CREATE TABLE t4p2 PARTITION OF t4 FOR VALUES IN (2);
CREATE TABLE t4p3 PARTITION OF t4 FOR VALUES IN (3);
INSERT INTO t4 SELECT (g%3)+1 FROM generate_series(1,60) g;
CREATE TABLE t5 (i int PRIMARY KEY, t text);
ALTER TABLE t5 ALTER COLUMN t SET STORAGE EXTERNAL;
INSERT INTO t5(i,t) VALUES (generate_series(1,20),
repeat('1234567890',269));
VACUUM FULL t1;
VACUUM FULL t2;
VACUUM FULL t3;
VACUUM FULL t5;
DELETE FROM t1 WHERE i < 50;
DELETE FROM t3 WHERE k < 50;
VACUUM t1;
VACUUM t3;
DROP TABLE t1;
DROP TABLE t2;
DROP TABLE t3;
DROP TABLE t4 CASCADE;
DROP TABLE t5;
VACUUM FULL pg_class;
VACUUM FULL pg_am;
VACUUM FULL pg_database;
3) Save as repro.sh and run:
#!/bin/bash
Q="psql -d testdb -q"
for round in $(seq 1 200); do
to=$(( (round % 12) + 1 ))
{ echo "SET statement_timeout='${to}ms';"
cat repro.sql
echo "SET statement_timeout=0;"
echo "CREATE TABLE probe_${round}(a int, t text);"
echo "DROP TABLE IF EXISTS probe_${round};"
} | $Q 2>/dev/null
if grep -q "TRAP:" "$PGDATA/server.log" 2>/dev/null; then
echo "CRASH at round $round"
grep -m1 "TRAP:" "$PGDATA/server.log"
break
fi
done
RESULT: crashes 5/5 runs within 37 rounds (rounds 13, 25, 37, 37, 37).
Using the first 187 lines of src/test/regress/sql/vacuum.sql instead
(heavier pg_class churn) accelerates the crash to 3/3 within round 24.
Without statement_timeout (timeout = 0), the same workload runs for
hundreds of rounds without corruption -- query cancellation is the
essential trigger.
ASSERTION + BACKTRACE
TRAP: failed Assert("false"), File: "heapam.c", Line: 8668
#5 ExceptionalCondition assert.c:65
#6 index_delete_sort_cmp heapam.c:8668
#7 index_delete_sort heapam.c:8708
#8 heap_index_delete_tuples heapam.c:8366
#10 _bt_delitems_delete_check nbtpage.c:1536
#11 _bt_simpledel_pass nbtinsert.c:2959
#12 _bt_delete_or_dedup_one_page nbtinsert.c:2770
#14 _bt_doinsert nbtinsert.c:261
#15 btinsert nbtree.c:219
#16 index_insert indexam.c:231
#17 CatalogIndexInsert indexing.c:170
#18 CatalogTupleInsert indexing.c:243
#19 InsertPgClassTuple heap.c:989
#20 AddNewRelationTuple heap.c:1049
#21 heap_create_with_catalog heap.c:1447
#22 create_toast_table toasting.c:254
#25 ProcessUtilitySlow utility.c:1197
#31 exec_simple_query
"CREATE TABLE covering_index_heap (f1 int, f2 int, f3 text);"
DIRECT CORRUPTION EVIDENCE (core dump)
frame _bt_delitems_delete_check:
rel (index) = OID 2663 "pg_class_relname_nsp_index"
heapRel (heap) = OID 1259 "pg_class"
delstate->ndeltids = 119
delstate->deltids:
entry[3] = { tid=(block 0, offset 5), id=45 }
entry[6] = { tid=(block 0, offset 5), id=45 } <-- DUPLICATE
Two entries carry the identical heap TID (0,5). This can only happen
if pg_class_relname_nsp_index contains two live index pointers to the
same pg_class heap tuple -- the catalog index is corrupt.
SERIAL CONFIRMATION
Re-run with all parallelism disabled (debug_parallel_query=off,
max_parallel_workers=0, max_parallel_maintenance_workers=0,
max_worker_processes=0): still crashes at rounds 13-24 across
multiple runs. The crashing backend has no parallel/bgworker frames
anywhere in the stack. This rules out any parallel-index-build or
concurrent-reindex race condition.
A second core dump from a serial run shows a different crashing
statement (REINDEX TABLE CONCURRENTLY testcomment) hitting the same
corrupt pg_class_relname_nsp_index leaf via CatalogTupleUpdate ->
_bt_simpledel_pass. The corruption is latent in the index; any
subsequent catalog write that hits the corrupt page triggers the
assert.
ROOT CAUSE ANALYSIS
The corruption is a pg_class_relname_nsp_index btree leaf page
holding two pointers to one heap TID. Bisection of the workload
identifies three necessary ingredients:
1. VACUUM FULL pg_class -- rewrites the entire pg_class heap and
rebuilds all pg_class btree indexes from scratch. This is the
essential catalog operation; removing it eliminates the crash.
2. Heavy DDL churn (CREATE TABLE/INDEX, DROP TABLE, VACUUM FULL on
user tables) -- generates many pg_class inserts, updates, and
deletes, filling btree pages and creating conditions for
dedup/simple-delete passes during subsequent inserts.
3. statement_timeout cancellation -- interrupts (1) or (2) mid-flight
via CHECK_FOR_INTERRUPTS(). Without cancellation, no corruption
occurs even after hundreds of rounds.
The most likely mechanism: VACUUM FULL pg_class calls cluster_rel()
which rewrites the heap and rebuilds all indexes via index_build().
If statement_timeout fires during this rebuild at a
CHECK_FOR_INTERRUPTS() site, the transaction is aborted, but a
catalog-index btree page may be left in an inconsistent state with a
duplicate pointer to the same heap TID. The abort/rollback path
does not fully undo the partial btree page modification.
PRIOR ART
The "duplicate heap TID in a pg_class index" corruption class is
known:
- BUG #18490 (2024) -- same symptom, index
pg_class_tblspc_relfilenode_index, trigger = concurrent REINDEX
during table creation; Peter Geoghegan: "a known issue", no fix
committed.
- BUG #17386 (2022) -- btree corruption after REINDEX CONCURRENTLY.
- BUG #17255 (2021) -- index_delete_sort_cmp via parallel-vacuum
race; fixed 2022.
What is new: all known reports require concurrency (parallel workers,
concurrent connections, concurrent REINDEX). This reproducer triggers
the same corruption in a single session with all parallelism disabled,
via statement cancellation alone. This is an interrupt-safety bug in
catalog index maintenance, distinct from the concurrency races in the
known reports.
DISCOVERY
Credit: Zheng Wang, Yanjie Zhao, Yiyang Liu.
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: BUG #19620: pg_class index corruption caused by statement_timeout during VACUUM FULL
@ 2026-08-16 16:01 Andrey Borodin <x4mmm@yandex-team.ru>
parent: PG Bug reporting form <noreply@postgresql.org>
0 siblings, 1 reply; 4+ messages in thread
From: Andrey Borodin @ 2026-08-16 16:01 UTC (permalink / raw)
To: hackerzheng666@gmail.com, PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
> On 15 Aug 2026, at 05:37, PG Bug reporting form <noreply@postgresql.org> wrote:
>
> What is new: all known reports require concurrency (parallel workers,
> concurrent connections, concurrent REINDEX). This reproducer triggers
> the same corruption in a single session with all parallelism disabled,
> via statement cancellation alone. This is an interrupt-safety bug in
> catalog index maintenance, distinct from the concurrency races in the
> known reports.
Hi Zheng,
Thanks for the report. This is worth fixing for sure, thanks for raising
attention to this.
I believe this is a known bug, unfixed since July 2020 [0]. Justin
Pryzby hit the same duplicate-TID corruption in a pg_class index, and
Peter Geoghegan and Tom Lane diagnosed it there; Peter also pointed at
that thread when BUG #18490 arrived in 2024 [1]. The mechanism is in
heapam_index_build_range_scan(): for a HEAPTUPLE_INSERT_IN_PROGRESS
tuple inserted by another transaction, the build waits only when it is
checking uniqueness. So for a non-unique index our opinion of which HOT
chain member is live can change mid-scan, and the build emits two
entries for one chain, both carrying the chain root's TID.
Worth adding, since that thread only ever discussed the assertion: this
is not confined to assert builds. I reproduced on REL_18_STABLE with
two sessions, one looping REINDEX INDEX
pg_class_tblspc_relfilenode_index and one looping CREATE TABLE /
CREATE INDEX / DROP TABLE, with no statement_timeout anywhere. A
cassert build trips comparetup_index_btree_tiebreak() in 0.4 seconds. A
build without assertions completes the rebuild and leaves the catalog
index corrupt on disk, where amcheck finds it after 23 iterations:
ERROR: posting list contains misplaced TID in index
"pg_class_tblspc_relfilenode_index"
DETAIL: Index tid=(1,210) posting list offset=1 page lsn=0/1C2A950.
So bt_index_check() on catalog indexes seems worth running in
production....
Now the thing to check. I suspect your reproducer is concurrent after
all, which would make this the same bug rather than a new one.
max_worker_processes = 0 does not turn autovacuum off. With that
setting autovacuum and autoanalyze are still running, I see 6
on pg_class itself within half a minute. Some of those updates
a pg_class row without touching an indexed column, which is a HOT
update on pg_class, and that is precisely the concurrent writer the
2020 mechanism needs. The code also seems to require one: both branches
that can emit a second entry for a chain sit inside
"if (!TransactionIdIsCurrentTransactionId(xwait))", so a backend acting
alone should not be able to reach them.
Could you set autovacuum = off in postgresql.conf, rather than relying
on max_worker_processes, and re-run your loop? If it still corrupts,
then you do have something new and I would very much like to see it. It
would also help to know the xmin of the two heap tuples your duplicate
pointers reference, and whether that XID belongs to your session.
I could not reproduce a single-session failure myself, but that is weak
evidence either way, so I would rather have your answer than my guess.
Thank you!
Best regards, Andrey Borodin.
[0] https://www.postgresql.org/message-id/20200728151002.GE20393%40telsasoft.com
[1] https://www.postgresql.org/message-id/CAH2-WzkaOCUV_0JYmbv%3DZKH5cs6X27eKZLouWq0d6_meGyNJtg%40mail.g...
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: BUG #19620: pg_class index corruption caused by statement_timeout during VACUUM FULL
@ 2026-08-17 02:20 Zheng Hacker <hackerzheng666@gmail.com>
parent: Andrey Borodin <x4mmm@yandex-team.ru>
0 siblings, 0 replies; 4+ messages in thread
From: Zheng Hacker @ 2026-08-17 02:20 UTC (permalink / raw)
To: Andrey Borodin <x4mmm@yandex-team.ru>; +Cc: PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
Hi Andrey,
Thanks for pointing that out.
We re-ran the reproducer for 2,000 rounds with autovacuum=off,
together with max_worker_processes=0,
max_parallel_workers_per_gather=0, and
max_parallel_maintenance_workers=0. We saw no corruption or crashes,
and autovacuum_count for pg_class stayed at 0 throughout the test.
So it looks like the issue only reproduces when autovacuum can run
concurrently with the index build, which matches the behavior
described in #18490 and #17386.
Thanks again for the clarification.
Best,
Zheng Wang
Andrey Borodin <x4mmm@yandex-team.ru> 于2026年8月17日周一 00:01写道：
>
>
>
> > On 15 Aug 2026, at 05:37, PG Bug reporting form <noreply@postgresql.org> wrote:
> >
> > What is new: all known reports require concurrency (parallel workers,
> > concurrent connections, concurrent REINDEX). This reproducer triggers
> > the same corruption in a single session with all parallelism disabled,
> > via statement cancellation alone. This is an interrupt-safety bug in
> > catalog index maintenance, distinct from the concurrency races in the
> > known reports.
>
>
> Hi Zheng,
>
> Thanks for the report. This is worth fixing for sure, thanks for raising
> attention to this.
>
> I believe this is a known bug, unfixed since July 2020 [0]. Justin
> Pryzby hit the same duplicate-TID corruption in a pg_class index, and
> Peter Geoghegan and Tom Lane diagnosed it there; Peter also pointed at
> that thread when BUG #18490 arrived in 2024 [1]. The mechanism is in
> heapam_index_build_range_scan(): for a HEAPTUPLE_INSERT_IN_PROGRESS
> tuple inserted by another transaction, the build waits only when it is
> checking uniqueness. So for a non-unique index our opinion of which HOT
> chain member is live can change mid-scan, and the build emits two
> entries for one chain, both carrying the chain root's TID.
>
> Worth adding, since that thread only ever discussed the assertion: this
> is not confined to assert builds. I reproduced on REL_18_STABLE with
> two sessions, one looping REINDEX INDEX
> pg_class_tblspc_relfilenode_index and one looping CREATE TABLE /
> CREATE INDEX / DROP TABLE, with no statement_timeout anywhere. A
> cassert build trips comparetup_index_btree_tiebreak() in 0.4 seconds. A
> build without assertions completes the rebuild and leaves the catalog
> index corrupt on disk, where amcheck finds it after 23 iterations:
>
> ERROR: posting list contains misplaced TID in index
> "pg_class_tblspc_relfilenode_index"
> DETAIL: Index tid=(1,210) posting list offset=1 page lsn=0/1C2A950.
>
> So bt_index_check() on catalog indexes seems worth running in
> production....
>
> Now the thing to check. I suspect your reproducer is concurrent after
> all, which would make this the same bug rather than a new one.
> max_worker_processes = 0 does not turn autovacuum off. With that
> setting autovacuum and autoanalyze are still running, I see 6
> on pg_class itself within half a minute. Some of those updates
> a pg_class row without touching an indexed column, which is a HOT
> update on pg_class, and that is precisely the concurrent writer the
> 2020 mechanism needs. The code also seems to require one: both branches
> that can emit a second entry for a chain sit inside
> "if (!TransactionIdIsCurrentTransactionId(xwait))", so a backend acting
> alone should not be able to reach them.
>
> Could you set autovacuum = off in postgresql.conf, rather than relying
> on max_worker_processes, and re-run your loop? If it still corrupts,
> then you do have something new and I would very much like to see it. It
> would also help to know the xmin of the two heap tuples your duplicate
> pointers reference, and whether that XID belongs to your session.
>
> I could not reproduce a single-session failure myself, but that is weak
> evidence either way, so I would rather have your answer than my guess.
>
> Thank you!
>
> Best regards, Andrey Borodin.
>
> [0] https://www.postgresql.org/message-id/20200728151002.GE20393%40telsasoft.com
> [1] https://www.postgresql.org/message-id/CAH2-WzkaOCUV_0JYmbv%3DZKH5cs6X27eKZLouWq0d6_meGyNJtg%40mail.g...
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: BUG #19620: pg_class index corruption caused by statement_timeout during VACUUM FULL
@ 2026-09-12 18:51 Tesla V. <maah.celo.v@gmail.com>
0 siblings, 0 replies; 4+ messages in thread
From: Tesla V. @ 2026-09-12 18:51 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org; +Cc: hackerzheng666@gmail.com; x4mmm@yandex-team.ru
Hi Zheng, hi Andrey,
Zheng's autovacuum=off result matches what I see: this is the July 2020
catalog HOT / index-build race, not a new interrupt-safety bug.
statement_timeout only causes VACUUM FULL of pg_class to retry while
autovacuum is still HOT-updating catalog rows.
Waiting for INSERT_IN_PROGRESS is not usable on catalogs. Commit
1ddc2703a936 stopped those waits on non-unique builds because VACUUM
FULL / CLUSTER takes AccessExclusiveLock on the catalog and then
deadlocks with a backend whose in-progress insert we would wait for.
VACUUM FULL also sets skip_constraint_checks, so even unique pg_class
indexes take the non-waiting path. That is why the original report
could corrupt pg_class_relname_nsp_index.
The attached patch keeps that decision. heapam_index_build_range_scan
records which root offsets on the current page have already been
handed to the AM, and skips a second callback for the same root TID.
The mark is taken only after a partial-index predicate would accept
the tuple. Refreshing root_offsets for an unknown HOT parent does not
clear the map.
I reproduced this on current master with cassert. With about 14000
pg_class rows, REINDEX INDEX pg_class_tblspc_relfilenode_index against
GRANT/REVOKE on one table (relacl is not indexed, so the update is
HOT) traps without the patch in comparetup_index_btree_tiebreak
("ItemPointer values should never be equal") within a few seconds.
The same workload with the patch does not trap. bt_index_check and
bt_index_parent_check pass, and VACUUM FULL pg_class still completes
while another transaction has an open catalog insert.
contrib/amcheck/t/007_catalog_reindex_hot.pl covers the deadlock
invariant and the concurrent REINDEX + GRANT case.
The patch is against master.
Thanks,
Marcelo Tesla
Attachments:
[text/x-patch] v1-0001-Avoid-duplicate-root-TIDs-in-catalog-index-builds.patch (11.0K, ../../CABr23Z-d8qvOkR+ZY6NUCD13pmJasLDd9eP9KDS82Va8jpw=eQ@mail.gmail.com/2-v1-0001-Avoid-duplicate-root-TIDs-in-catalog-index-builds.patch)
download | inline diff:
diff --git a/contrib/amcheck/meson.build b/contrib/amcheck/meson.build
index d5137ef691d..d31dfe99be6 100644
--- a/contrib/amcheck/meson.build
+++ b/contrib/amcheck/meson.build
@@ -50,6 +50,7 @@ tests += {
't/004_verify_nbtree_unique.pl',
't/005_pitr.pl',
't/006_verify_gin.pl',
+ 't/007_catalog_reindex_hot.pl',
],
},
}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index dd2218bbbaf..0f62e4989e8 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -1155,6 +1155,7 @@ heapam_index_build_range_scan(Relation heapRelation,
BlockNumber previous_blkno = InvalidBlockNumber;
BlockNumber root_blkno = InvalidBlockNumber;
OffsetNumber root_offsets[MaxHeapTuplesPerPage];
+ bool indexed_root[MaxHeapTuplesPerPage];
/*
* sanity checks
@@ -1331,9 +1332,15 @@ heapam_index_build_range_scan(Relation heapRelation,
* the chain root locations won't, so this info doesn't need to be
* rebuilt after waiting for another transaction.
*
- * Note the implied assumption that there is no more than one live
- * tuple per HOT-chain --- else we could create more than one index
- * entry pointing to the same root tuple.
+ * That liveness change is not hypothetical. System catalog
+ * modifications release their relation lock before commit, so a
+ * non-unique index build can observe both a still-live root and a
+ * heap-only HOT update from another transaction. Indexing both
+ * would emit two entries for the same root TID. We do not wait for
+ * the inserting transaction: that reintroduces VACUUM FULL/CLUSTER
+ * deadlocks on catalogs (commit 1ddc2703a936). Instead,
+ * indexed_root[] records which root offsets we already handed to the
+ * AM on this page.
*/
if (hscan->rs_cblock != root_blkno)
{
@@ -1343,6 +1350,7 @@ heapam_index_build_range_scan(Relation heapRelation,
heap_get_root_tuples(page, root_offsets);
LockBuffer(hscan->rs_cbuf, BUFFER_LOCK_UNLOCK);
+ MemSet(indexed_root, 0, sizeof(indexed_root));
root_blkno = hscan->rs_cblock;
}
@@ -1445,6 +1453,12 @@ heapam_index_build_range_scan(Relation heapRelation,
* such a tuple could lead to a bogus uniqueness
* failure. In that case we wait for the inserting
* transaction to finish and check again.
+ *
+ * We do not wait merely because the tuple is part of
+ * a HOT chain. That would re-introduce catalog
+ * deadlocks with VACUUM FULL/CLUSTER (1ddc2703a936).
+ * Duplicate root TIDs from a mid-scan HOT update are
+ * suppressed via indexed_root[] instead.
*/
if (checking_uniqueness)
{
@@ -1616,52 +1630,77 @@ heapam_index_build_range_scan(Relation heapRelation,
* pass the values[] and isnull[] arrays, instead.
*/
- if (HeapTupleIsHeapOnly(heapTuple))
{
- /*
- * For a heap-only tuple, pretend its TID is that of the root. See
- * src/backend/access/heap/README.HOT for discussion.
- */
ItemPointerData tid;
- OffsetNumber offnum;
-
- offnum = ItemPointerGetOffsetNumber(&heapTuple->t_self);
+ ItemPointer tidptr;
+ OffsetNumber emitoff;
- /*
- * If a HOT tuple points to a root that we don't know about,
- * obtain root items afresh. If that still fails, report it as
- * corruption.
- */
- if (root_offsets[offnum - 1] == InvalidOffsetNumber)
+ if (HeapTupleIsHeapOnly(heapTuple))
{
- Page page = BufferGetPage(hscan->rs_cbuf);
+ /*
+ * For a heap-only tuple, pretend its TID is that of the root.
+ * See src/backend/access/heap/README.HOT for discussion.
+ */
+ OffsetNumber offnum;
- LockBuffer(hscan->rs_cbuf, BUFFER_LOCK_SHARE);
- heap_get_root_tuples(page, root_offsets);
- LockBuffer(hscan->rs_cbuf, BUFFER_LOCK_UNLOCK);
- }
+ offnum = ItemPointerGetOffsetNumber(&heapTuple->t_self);
- if (!OffsetNumberIsValid(root_offsets[offnum - 1]))
- ereport(ERROR,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg_internal("failed to find parent tuple for heap-only tuple at (%u,%u) in table \"%s\"",
- ItemPointerGetBlockNumber(&heapTuple->t_self),
- offnum,
- RelationGetRelationName(heapRelation))));
+ /*
+ * If a HOT tuple points to a root that we don't know about,
+ * obtain root items afresh. If that still fails, report it
+ * as corruption.
+ */
+ if (root_offsets[offnum - 1] == InvalidOffsetNumber)
+ {
+ Page page = BufferGetPage(hscan->rs_cbuf);
- ItemPointerSet(&tid, ItemPointerGetBlockNumber(&heapTuple->t_self),
- root_offsets[offnum - 1]);
+ LockBuffer(hscan->rs_cbuf, BUFFER_LOCK_SHARE);
+ heap_get_root_tuples(page, root_offsets);
+ LockBuffer(hscan->rs_cbuf, BUFFER_LOCK_UNLOCK);
- /* Call the AM's callback routine to process the tuple */
- callback(indexRelation, &tid, values, isnull, tupleIsAlive,
+ /*
+ * Leave indexed_root[] unchanged; it tracks TIDs we
+ * already emitted on this page.
+ */
+ }
+
+ if (!OffsetNumberIsValid(root_offsets[offnum - 1]))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("failed to find parent tuple for heap-only tuple at (%u,%u) in table \"%s\"",
+ ItemPointerGetBlockNumber(&heapTuple->t_self),
+ offnum,
+ RelationGetRelationName(heapRelation))));
+
+ ItemPointerSet(&tid, ItemPointerGetBlockNumber(&heapTuple->t_self),
+ root_offsets[offnum - 1]);
+ tidptr = &tid;
+ emitoff = root_offsets[offnum - 1];
+ }
+ else
+ {
+ tidptr = &heapTuple->t_self;
+ emitoff = ItemPointerGetOffsetNumber(&heapTuple->t_self);
+ }
+
+ /*
+ * HOT chains are confined to one page and must produce at most
+ * one index entry. If we already emitted this root TID on the
+ * current page, skip; the earlier callback used the same TID
+ * (and, for a well-formed HOT chain, the same key).
+ *
+ * Record the emission only here, after the partial-index
+ * predicate has accepted the tuple, so a rejected member does not
+ * suppress a later one.
+ */
+ Assert(OffsetNumberIsValid(emitoff));
+ if (indexed_root[emitoff - 1])
+ continue;
+ indexed_root[emitoff - 1] = true;
+
+ callback(indexRelation, tidptr, values, isnull, tupleIsAlive,
callback_state);
}
- else
- {
- /* Call the AM's callback routine to process the tuple */
- callback(indexRelation, &heapTuple->t_self, values, isnull,
- tupleIsAlive, callback_state);
- }
}
/* Report scan progress one last time. */
diff --git a/contrib/amcheck/t/007_catalog_reindex_hot.pl b/contrib/amcheck/t/007_catalog_reindex_hot.pl
new file mode 100644
index 00000000000..69542b8337c
--- /dev/null
+++ b/contrib/amcheck/t/007_catalog_reindex_hot.pl
@@ -0,0 +1,128 @@
+
+# Copyright (c) 2026, PostgreSQL Global Development Group
+
+# BUG #19620: a REINDEX of a non-unique catalog index can emit two btree
+# entries for the same heap TID when it races with a HOT update. Catalog
+# writers release their lock before commit, so the index build (which holds
+# ShareLock) can still see INSERT_IN_PROGRESS heap-only tuples. Waiting for
+# those inserts would deadlock VACUUM FULL/CLUSTER on catalogs (1ddc2703);
+# heapam_index_build_range_scan instead skips a second emission of the same
+# root TID.
+#
+# pg_class_tblspc_relfilenode_index is the usual casualty: it is not unique,
+# so the build does not wait on INSERT_IN_PROGRESS.
+#
+# The race is easy to miss on an empty initdb (tiny pg_class, scan finishes
+# in microseconds). Inflate pg_class, then hammer GRANT/REVOKE on one
+# persistent row (relacl is not indexed, so the update is HOT) against
+# REINDEX. Unpatched cassert traps in comparetup_index_btree_tiebreak
+# ("ItemPointer values should never be equal").
+
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+
+use Test::More;
+
+my $node = PostgreSQL::Test::Cluster->new('catalog_reindex_hot');
+$node->init;
+$node->append_conf('postgresql.conf',
+ 'lock_timeout = ' . (1000 * $PostgreSQL::Test::Utils::timeout_default));
+$node->append_conf('postgresql.conf', 'deadlock_timeout = 1s');
+$node->append_conf('postgresql.conf', 'max_locks_per_transaction = 128');
+$node->start;
+$node->safe_psql('postgres', q(CREATE EXTENSION amcheck));
+
+# Extra pg_class rows lengthen the index-build heap scan. CREATE TYPE AS ()
+# is a cheap pg_class insert; COMMIT every batch so we do not exhaust the
+# lock table. persist_1 is the HOT target (relacl).
+$node->safe_psql(
+ 'postgres',
+ q(
+CREATE TABLE persist_1(i int);
+CREATE PROCEDURE catalog_hot_bloat(n_types int)
+LANGUAGE plpgsql AS $$
+DECLARE
+ i int;
+BEGIN
+ FOR i IN 1..n_types LOOP
+ EXECUTE format('CREATE TYPE catalog_hot_ty_%s AS ()', i);
+ IF i % 40 = 0 THEN
+ COMMIT;
+ END IF;
+ END LOOP;
+ COMMIT;
+END;
+$$;
+CALL catalog_hot_bloat(8000);
+));
+
+#
+# VACUUM FULL must not wait out an in-progress catalog insert. CREATE TABLE
+# inserts into pg_class and then releases that lock before commit; the rewrite
+# should finish while the inserting transaction is still open.
+#
+# While it runs, the inserting backend must still be able to do more catalog
+# work. Waiting for INSERT_IN_PROGRESS during the rewrite deadlocks here:
+# VACUUM FULL holds AccessExclusiveLock and waits for the inserter; the
+# inserter waits for that exclusive lock.
+#
+my $hold = $node->background_psql('postgres');
+$hold->query_safe(q(BEGIN; CREATE TABLE hold_open(i int);));
+
+my $vf = $node->background_psql('postgres');
+$vf->query_until(
+ qr/start/,
+ q(
+\echo start
+SET statement_timeout = '30s';
+VACUUM FULL pg_class;
+));
+$hold->query_safe(q(CREATE TABLE hold_open_2(i int);));
+$vf->query_safe(q(SELECT 1));
+pass('VACUUM FULL pg_class does not deadlock with in-progress catalog inserts');
+
+$hold->query_safe(q(COMMIT; DROP TABLE hold_open, hold_open_2;));
+$hold->quit;
+$vf->quit;
+
+#
+# Stress REINDEX against concurrent catalog HOT updates of a single pg_class
+# row. Serialize GRANT vs GRANT with an advisory lock so pgbench is not
+# aborted by "tuple concurrently updated"; REINDEX (ShareLock) still races
+# with GRANT (RowExclusiveLock).
+#
+$node->pgbench(
+ '--no-vacuum --client=4 --jobs=4 --time=12',
+ 0,
+ [qr{actually processed}],
+ [qr{^$}],
+ 'concurrent catalog REINDEX and HOT GRANT',
+ {
+ '007_reindex_catalog' => q(
+ REINDEX INDEX pg_class_tblspc_relfilenode_index;
+ ),
+ '007_catalog_acl_hot' => q(
+ SELECT pg_try_advisory_lock(43)::integer AS gotlock \gset
+ \if :gotlock
+ GRANT SELECT ON persist_1 TO PUBLIC;
+ REVOKE SELECT ON persist_1 FROM PUBLIC;
+ SELECT pg_advisory_unlock(43);
+ \endif
+ )
+ });
+
+$node->safe_psql(
+ 'postgres',
+ q(
+SELECT bt_index_check('pg_class_tblspc_relfilenode_index', true);
+SELECT bt_index_check('pg_class_relname_nsp_index', true);
+SELECT bt_index_check('pg_class_oid_index', true);
+SELECT bt_index_parent_check('pg_class_tblspc_relfilenode_index', true, true);
+));
+pass('pg_class indexes pass bt_index_check after concurrent REINDEX');
+
+$node->stop;
+done_testing();
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2026-09-12 18:51 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-08-15 02:37 BUG #19620: pg_class index corruption caused by statement_timeout during VACUUM FULL PG Bug reporting form <noreply@postgresql.org>
2026-08-16 16:01 ` Andrey Borodin <x4mmm@yandex-team.ru>
2026-08-17 02:20 ` Zheng Hacker <hackerzheng666@gmail.com>
2026-09-12 18:51 Re: BUG #19620: pg_class index corruption caused by statement_timeout during VACUUM FULL Tesla V. <maah.celo.v@gmail.com>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox