Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Subject: BUG #19620: pg_class index corruption caused by statement_timeout
 during VACUUM FULL
To: pgsql-bugs@lists.postgresql.org
From: PG Bug reporting form <noreply@postgresql.org>
Cc: hackerzheng666@gmail.com
Reply-To: hackerzheng666@gmail.com, pgsql-bugs@lists.postgresql.org
Date: Sat, 15 Aug 2026 02:37:43 +0000
Message-ID: <19620-3bc8f6c8d53fd8f2@postgresql.org>
Auto-Submitted: auto-generated
Archived-At: 
 <https://www.postgresql.org/message-id/19620-3bc8f6c8d53fd8f2%40postgresql.org>
Precedence: bulk

The following bug has been logged on the website:

Bug reference:      19620
Logged by:          Zheng Hacker
Email address:      hackerzheng666@gmail.com
PostgreSQL version: 19beta3
Operating system:   Linux x86_64
Description:       =20

SUMMARY

On PostgreSQL 18devel (git master cc1ea71), a single backend that
repeatedly runs catalog-churning DDL/maintenance under a short
statement_timeout drives the system-catalog index
pg_class_relname_nsp_index into a corrupt state: one btree leaf page
acquires two index tuples pointing to the same pg_class heap TID.

A subsequent CREATE TABLE touches that leaf page, a btree
simple-delete/dedup pass collects the two identical TIDs, and the
cassert assertion in index_delete_sort_cmp (heapam.c:8668) fires.

On a production (non-cassert) build, the corrupt catalog index is
silent and can produce wrong catalog lookups (we separately observe
the Assert(relid =3D=3D targetRelId) in relcache.c:1138 from the same
corruption when a pg_class OID-index scan returns the wrong row).

KEY DISTINCTION from known reports (#18490, #17386, #17255): this
reproducer is purely single-session, single-backend -- all parallelism
disabled (max_worker_processes=3D0), no concurrent connections.  The
trigger is statement cancellation (CHECK_FOR_INTERRUPTS) during
catalog-index maintenance, not a concurrency race.


VERSION / BUILD

  PostgreSQL master (ahead of 19beta3), git commit cc1ea71
  ("Remove unnecessary list_free() calls in OpenTableList()")
  Build: clang -O1 -g --enable-cassert (no sanitizers, no sancov),
  x86_64 Linux.
  First observed on an ASan+sancov build; confirmed on this clean
  build to rule out instrumentation artifacts.


REPRODUCTION

1) Server setup:

  initdb -D $PGDATA --no-locale -A trust
  cat >> $PGDATA/postgresql.conf << 'EOF'
  fsync =3D off
  debug_parallel_query =3D off
  max_parallel_workers =3D 0
  max_parallel_maintenance_workers =3D 0
  max_worker_processes =3D 0
  EOF
  pg_ctl -D $PGDATA -l $PGDATA/server.log start
  createdb testdb

2) Save as repro.sql:

  CREATE TABLE t1 (i int, t text);
  INSERT INTO t1 SELECT g, md5(g::text) FROM generate_series(1,100) g;
  CREATE INDEX t1_i ON t1(i);
  CREATE INDEX t1_t ON t1(t);
  CREATE TABLE t2 (i int PRIMARY KEY, a int[], p point);
  INSERT INTO t2 SELECT g, array[g,g+1], point(g,g) FROM
generate_series(1,100) g;
  CREATE INDEX t2_hash ON t2 USING hash(i);
  CREATE INDEX t2_gin ON t2 USING gin(a);
  CREATE INDEX t2_gist ON t2 USING gist(p);
  CREATE TABLE t3 (k int, v text);
  INSERT INTO t3 SELECT g, repeat('x',200) FROM generate_series(1,100) g;
  CREATE INDEX t3_k ON t3(k);
  CREATE TABLE t4 (a int) PARTITION BY LIST(a);
  CREATE TABLE t4p1 PARTITION OF t4 FOR VALUES IN (1);
  CREATE TABLE t4p2 PARTITION OF t4 FOR VALUES IN (2);
  CREATE TABLE t4p3 PARTITION OF t4 FOR VALUES IN (3);
  INSERT INTO t4 SELECT (g%3)+1 FROM generate_series(1,60) g;
  CREATE TABLE t5 (i int PRIMARY KEY, t text);
  ALTER TABLE t5 ALTER COLUMN t SET STORAGE EXTERNAL;
  INSERT INTO t5(i,t) VALUES (generate_series(1,20),
repeat('1234567890',269));
  VACUUM FULL t1;
  VACUUM FULL t2;
  VACUUM FULL t3;
  VACUUM FULL t5;
  DELETE FROM t1 WHERE i < 50;
  DELETE FROM t3 WHERE k < 50;
  VACUUM t1;
  VACUUM t3;
  DROP TABLE t1;
  DROP TABLE t2;
  DROP TABLE t3;
  DROP TABLE t4 CASCADE;
  DROP TABLE t5;
  VACUUM FULL pg_class;
  VACUUM FULL pg_am;
  VACUUM FULL pg_database;

3) Save as repro.sh and run:

  #!/bin/bash
  Q=3D"psql -d testdb -q"
  for round in $(seq 1 200); do
      to=3D$(( (round % 12) + 1 ))
      { echo "SET statement_timeout=3D'${to}ms';"
        cat repro.sql
        echo "SET statement_timeout=3D0;"
        echo "CREATE TABLE probe_${round}(a int, t text);"
        echo "DROP TABLE IF EXISTS probe_${round};"
      } | $Q 2>/dev/null
      if grep -q "TRAP:" "$PGDATA/server.log" 2>/dev/null; then
          echo "CRASH at round $round"
          grep -m1 "TRAP:" "$PGDATA/server.log"
          break
      fi
  done

RESULT: crashes 5/5 runs within 37 rounds (rounds 13, 25, 37, 37, 37).
Using the first 187 lines of src/test/regress/sql/vacuum.sql instead
(heavier pg_class churn) accelerates the crash to 3/3 within round 24.
Without statement_timeout (timeout =3D 0), the same workload runs for
hundreds of rounds without corruption -- query cancellation is the
essential trigger.


ASSERTION + BACKTRACE

  TRAP: failed Assert("false"), File: "heapam.c", Line: 8668
  #5  ExceptionalCondition                     assert.c:65
  #6  index_delete_sort_cmp                     heapam.c:8668
  #7  index_delete_sort                         heapam.c:8708
  #8  heap_index_delete_tuples                  heapam.c:8366
  #10 _bt_delitems_delete_check                 nbtpage.c:1536
  #11 _bt_simpledel_pass                        nbtinsert.c:2959
  #12 _bt_delete_or_dedup_one_page              nbtinsert.c:2770
  #14 _bt_doinsert                              nbtinsert.c:261
  #15 btinsert                                  nbtree.c:219
  #16 index_insert                              indexam.c:231
  #17 CatalogIndexInsert                        indexing.c:170
  #18 CatalogTupleInsert                        indexing.c:243
  #19 InsertPgClassTuple                        heap.c:989
  #20 AddNewRelationTuple                       heap.c:1049
  #21 heap_create_with_catalog                  heap.c:1447
  #22 create_toast_table                        toasting.c:254
  #25 ProcessUtilitySlow                        utility.c:1197
  #31 exec_simple_query
        "CREATE TABLE covering_index_heap (f1 int, f2 int, f3 text);"


DIRECT CORRUPTION EVIDENCE (core dump)

  frame _bt_delitems_delete_check:
    rel     (index) =3D OID 2663  "pg_class_relname_nsp_index"
    heapRel (heap)  =3D OID 1259  "pg_class"
    delstate->ndeltids =3D 119
    delstate->deltids:
      entry[3] =3D { tid=3D(block 0, offset 5), id=3D45 }
      entry[6] =3D { tid=3D(block 0, offset 5), id=3D45 }   <-- DUPLICATE

  Two entries carry the identical heap TID (0,5).  This can only happen
  if pg_class_relname_nsp_index contains two live index pointers to the
  same pg_class heap tuple -- the catalog index is corrupt.


SERIAL CONFIRMATION

  Re-run with all parallelism disabled (debug_parallel_query=3Doff,
  max_parallel_workers=3D0, max_parallel_maintenance_workers=3D0,
  max_worker_processes=3D0): still crashes at rounds 13-24 across
  multiple runs.  The crashing backend has no parallel/bgworker frames
  anywhere in the stack.  This rules out any parallel-index-build or
  concurrent-reindex race condition.

  A second core dump from a serial run shows a different crashing
  statement (REINDEX TABLE CONCURRENTLY testcomment) hitting the same
  corrupt pg_class_relname_nsp_index leaf via CatalogTupleUpdate ->
  _bt_simpledel_pass.  The corruption is latent in the index; any
  subsequent catalog write that hits the corrupt page triggers the
  assert.


ROOT CAUSE ANALYSIS

  The corruption is a pg_class_relname_nsp_index btree leaf page
  holding two pointers to one heap TID.  Bisection of the workload
  identifies three necessary ingredients:

  1. VACUUM FULL pg_class -- rewrites the entire pg_class heap and
     rebuilds all pg_class btree indexes from scratch.  This is the
     essential catalog operation; removing it eliminates the crash.

  2. Heavy DDL churn (CREATE TABLE/INDEX, DROP TABLE, VACUUM FULL on
     user tables) -- generates many pg_class inserts, updates, and
     deletes, filling btree pages and creating conditions for
     dedup/simple-delete passes during subsequent inserts.

  3. statement_timeout cancellation -- interrupts (1) or (2) mid-flight
     via CHECK_FOR_INTERRUPTS().  Without cancellation, no corruption
     occurs even after hundreds of rounds.

  The most likely mechanism: VACUUM FULL pg_class calls cluster_rel()
  which rewrites the heap and rebuilds all indexes via index_build().
  If statement_timeout fires during this rebuild at a
  CHECK_FOR_INTERRUPTS() site, the transaction is aborted, but a
  catalog-index btree page may be left in an inconsistent state with a
  duplicate pointer to the same heap TID.  The abort/rollback path
  does not fully undo the partial btree page modification.


PRIOR ART

  The "duplicate heap TID in a pg_class index" corruption class is
  known:

  - BUG #18490 (2024) -- same symptom, index
    pg_class_tblspc_relfilenode_index, trigger =3D concurrent REINDEX
    during table creation; Peter Geoghegan: "a known issue", no fix
    committed.
  - BUG #17386 (2022) -- btree corruption after REINDEX CONCURRENTLY.
  - BUG #17255 (2021) -- index_delete_sort_cmp via parallel-vacuum
    race; fixed 2022.

  What is new: all known reports require concurrency (parallel workers,
  concurrent connections, concurrent REINDEX).  This reproducer triggers
  the same corruption in a single session with all parallelism disabled,
  via statement cancellation alone.  This is an interrupt-safety bug in
  catalog index maintenance, distinct from the concurrency races in the
  known reports.


DISCOVERY

  Credit: Zheng Wang, Yanjie Zhao, Yiyang Liu.