pg.ddx.io pgsql-bugs@postgresql.org mailing list archive
help / color / mirror / Atom feedBUG #19829: BTREE pivot tuple length overflows t_info when BLCKSZ=32768
3+ messages / 2 participants
[nested] [flat]
* BUG #19829: BTREE pivot tuple length overflows t_info when BLCKSZ=32768
@ 2026-11-03 07:46 PG Bug reporting form <noreply@postgresql.org>
0 siblings, 1 reply; 3+ messages in thread
From: PG Bug reporting form @ 2026-11-03 07:46 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org; +Cc: wf13934952799@gmail.com
The following bug has been logged on the website:
Bug reference: 19829
Logged by: wangfei
Email address: wf13934952799@gmail.com
PostgreSQL version: 18.6
Operating system: ubuntu-24.04
Description:
PostgreSQL version: 19devel (also reproducible on 18.x)
Operating system: Linux x86_64
Configure options: --with-blocksize=32
With BLCKSZ=32768, BTMaxItemSize() evaluates to roughly 10896 bytes, but the
tuple length field reserved by INDEX_SIZE_MASK in IndexTupleData.t_info can
only represent 8191 bytes. index_form_tuple() enforces the 8191 limit, so a
leaf tuple is at most 8184 bytes after MAXALIGN. _bt_truncate() may then
need
to append a tiebreaker heap TID:
newsize = MAXALIGN(IndexTupleSize(pivot)) +
MAXALIGN(sizeof(ItemPointerData));
...
tidpivot->t_info &= ~INDEX_SIZE_MASK;
tidpivot->t_info |= newsize;
For a maximally sized leaf tuple this is 8184 + 8 = 8192, i.e. exactly
INDEX_SIZE_MASK + 1, so the length field wraps to zero. The result is a
zero-length pivot tuple used as a leaf high key; PageAddItem() and
XLogRegisterBufData() both accept it without complaint, so the corruption is
written out and WAL-logged silently.
BTMaxItemSize() is supposed to keep leaf tuples small enough that this
cannot
happen, but the one-third-of-a-page rule scales with BLCKSZ while
INDEX_SIZE_MASK does not, so at BLCKSZ=32768 the one-third rule is no longer
the binding constraint. Block sizes of 16384 and below are unaffected.
Steps to reproduce:
< SQL 脚本>
Expected: either the insert is rejected, or the pivot tuple is stored
correctly.
Actual: high key with length 0 (amcheck reports: <具体报错>).
A patch against master is attached. It takes Min() of the one-third rule
and
what t_info can represent in BTMaxItemSize(), and adds a defensive
elog(ERROR)
in _bt_truncate() so that a future miscalculation cannot corrupt an index
silently. Since index_form_tuple() already caps index tuples at 8184 bytes,
lowering the BLCKSZ=32768 limit cannot invalidate any index that could
actually exist on disk, so there is no on-disk compatibility concern.
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: BUG #19829: BTREE pivot tuple length overflows t_info when BLCKSZ=32768
@ 2026-11-03 08:10 王飞 <wf13934952799@gmail.com>
parent: PG Bug reporting form <noreply@postgresql.org>
0 siblings, 1 reply; 3+ messages in thread
From: 王飞 @ 2026-11-03 08:10 UTC (permalink / raw)
To: wf13934952799@gmail.com; pgsql-bugs@lists.postgresql.org
CREATE EXTENSION amcheck;
-- One incompressible 8172-byte value. The resulting index tuple is
-- MAXALIGN(sizeof(IndexTupleData) + VARHDRSZ + 8172) =
MAXALIGN(8+4+8172) = 8184
-- bytes, which is the largest index tuple index_form_tuple() will ever
produce.
CREATE TABLE pad AS
SELECT substr((SELECT string_agg(md5(random()::text), '')
FROM generate_series(1, 300)), 1, 8172) AS v;
CREATE TABLE t (v text);
-- Must come before CREATE INDEX: the index inherits attstorage, and this
stops
-- TOAST_INDEX_HACK in index_form_tuple() from compressing the value.
ALTER TABLE t ALTER COLUMN v SET STORAGE EXTERNAL;
CREATE INDEX t_v_idx ON t (v);
-- Four identical keys. Three fill the leaf page; the fourth splits it.
Since
-- all keys are equal, _bt_truncate() cannot truncate away any key
attribute and
-- must append a tiebreaker heap TID: newsize = 8184 + 8 = 8192.
INSERT INTO t SELECT v FROM pad, generate_series(1, 4);
SELECT bt_index_check('t_v_idx', true);
ERROR: invalid line pointer storage in index "t_v_idx"
DETAIL: Index tid=(3,2) lp_off=32744, lp_len=0 lp_flags=1.
PG Bug reporting form <noreply@postgresql.org> 于2026年11月3日周二 15:47写道:
> The following bug has been logged on the website:
>
> Bug reference: 19829
> Logged by: wangfei
> Email address: wf13934952799@gmail.com
> PostgreSQL version: 18.6
> Operating system: ubuntu-24.04
> Description:
>
> PostgreSQL version: 19devel (also reproducible on 18.x)
> Operating system: Linux x86_64
> Configure options: --with-blocksize=32
>
> With BLCKSZ=32768, BTMaxItemSize() evaluates to roughly 10896 bytes, but
> the
> tuple length field reserved by INDEX_SIZE_MASK in IndexTupleData.t_info can
> only represent 8191 bytes. index_form_tuple() enforces the 8191 limit, so
> a
> leaf tuple is at most 8184 bytes after MAXALIGN. _bt_truncate() may then
> need
> to append a tiebreaker heap TID:
>
> newsize = MAXALIGN(IndexTupleSize(pivot)) +
> MAXALIGN(sizeof(ItemPointerData));
> ...
> tidpivot->t_info &= ~INDEX_SIZE_MASK;
> tidpivot->t_info |= newsize;
>
> For a maximally sized leaf tuple this is 8184 + 8 = 8192, i.e. exactly
> INDEX_SIZE_MASK + 1, so the length field wraps to zero. The result is a
> zero-length pivot tuple used as a leaf high key; PageAddItem() and
> XLogRegisterBufData() both accept it without complaint, so the corruption
> is
> written out and WAL-logged silently.
>
> BTMaxItemSize() is supposed to keep leaf tuples small enough that this
> cannot
> happen, but the one-third-of-a-page rule scales with BLCKSZ while
> INDEX_SIZE_MASK does not, so at BLCKSZ=32768 the one-third rule is no
> longer
> the binding constraint. Block sizes of 16384 and below are unaffected.
>
> Steps to reproduce:
> < SQL 脚本>
>
> Expected: either the insert is rejected, or the pivot tuple is stored
> correctly.
> Actual: high key with length 0 (amcheck reports: <具体报错>).
>
> A patch against master is attached. It takes Min() of the one-third rule
> and
> what t_info can represent in BTMaxItemSize(), and adds a defensive
> elog(ERROR)
> in _bt_truncate() so that a future miscalculation cannot corrupt an index
> silently. Since index_form_tuple() already caps index tuples at 8184
> bytes,
> lowering the BLCKSZ=32768 limit cannot invalidate any index that could
> actually exist on disk, so there is no on-disk compatibility concern.
>
>
>
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: BUG #19829: BTREE pivot tuple length overflows t_info when BLCKSZ=32768
@ 2026-11-03 08:12 王飞 <wf13934952799@gmail.com>
parent: 王飞 <wf13934952799@gmail.com>
0 siblings, 0 replies; 3+ messages in thread
From: 王飞 @ 2026-11-03 08:12 UTC (permalink / raw)
To: wf13934952799@gmail.com; pgsql-bugs@lists.postgresql.org
#!/bin/bash
#
=============================================================================
# B-tree pivot tuple length overflows t_info when BLCKSZ = 32768
#
# BTMaxItemSize() scales with BLCKSZ (10896 at 32kB pages), but the tuple
# length field reserved by INDEX_SIZE_MASK in IndexTupleData.t_info is a
fixed
# 13 bits (8191). index_form_tuple() caps index tuples at 8184 bytes after
# MAXALIGN, so when _bt_truncate() has to append a tiebreaker heap TID it
# computes newsize = 8184 + 8 = 8192 = INDEX_SIZE_MASK + 1 and
# `pivot->t_info |= newsize` wraps the length field to zero.
#
# Needs a build with --with-blocksize=32. At BLCKSZ 8192/16384 the
# one-third-of-a-page rule (2704/5432) binds first, so those are
unaffected.
#
=============================================================================
set -u
PGBIN=${PGBIN:-/home/coder/postgres_github/release/REL_18_STABLE_32/bin}
PGDATA=${PGDATA:-/tmp/pgdata32_repro}
SOCKDIR=${SOCKDIR:-/tmp/s32repro} # keep short: socket path limit is 107
bytes
PGPORT=${PGPORT:-55432}
PSQL="$PGBIN/psql -h $SOCKDIR -p $PGPORT -U postgres -d postgres"
echo "### 0. confirm the binary uses 32kB pages"
$PGBIN/pg_config --configure | tr ' ' '\n' | grep blocksize || {
echo "!! not a 32kB build, cannot reproduce"; exit 1; }
echo "### 1. init + start"
rm -rf "$PGDATA"; mkdir -p "$SOCKDIR"
$PGBIN/initdb -D "$PGDATA" --locale=C --encoding=UTF8 -U postgres >/dev/null
2>&1
cat >> "$PGDATA/postgresql.conf" <<EOF
port = $PGPORT
listen_addresses = ''
unix_socket_directories = '$SOCKDIR'
fsync = off
restart_after_crash = off
EOF
$PGBIN/pg_controldata -D "$PGDATA" | grep -i "Database block size"
$PGBIN/pg_ctl -D "$PGDATA" -l "$PGDATA/server.log" start -w >/dev/null 2>&1
echo "### 2. build the corrupt index (this is the whole repro)"
$PSQL -v ON_ERROR_STOP=1 <<'EOF'
CREATE EXTENSION amcheck;
CREATE EXTENSION pageinspect;
-- One incompressible 8172-byte value. The resulting index tuple is
-- MAXALIGN(sizeof(IndexTupleData) + VARHDRSZ + 8172) = MAXALIGN(8+4+8172)
-- = 8184 bytes
-- which is the largest index tuple index_form_tuple() will ever produce.
CREATE TABLE pad AS
SELECT substr((SELECT string_agg(md5(random()::text), '')
FROM generate_series(1, 300)), 1, 8172) AS v;
CREATE TABLE t (v text);
-- Must come before CREATE INDEX: the index inherits attstorage, and this
stops
-- TOAST_INDEX_HACK in index_form_tuple() from compressing the value.
ALTER TABLE t ALTER COLUMN v SET STORAGE EXTERNAL;
CREATE INDEX t_v_idx ON t (v);
CHECKPOINT;
-- Four identical keys. Three fill the leaf page; the fourth splits it.
Since
-- all the keys are equal, _bt_truncate() cannot truncate away any key
-- attribute and must append a tiebreaker heap TID: newsize = 8184 + 8 =
8192.
INSERT INTO t SELECT v FROM pad, generate_series(1, 4);
EOF
echo
echo "### 3. amcheck"
$PSQL -c "SELECT bt_index_check('t_v_idx', true);" 2>&1 | sed 's/^/
/'
$PSQL -c "SELECT bt_index_parent_check('t_v_idx', true);" 2>&1 | sed 's/^/
/'
echo "### 4. page layout: zero-length pivot as leaf high key AND as root
downlink"
$PSQL -c "SELECT b AS blkno, s.type, s.live_items, s.avg_item_size
FROM generate_series(1,3) b, bt_page_stats('t_v_idx', b) s;"
$PSQL -c "
WITH g AS (SELECT blk, get_raw_page('t_v_idx', blk) AS pg FROM
generate_series(1,3) blk),
w AS (SELECT blk, off,
get_byte(pg,24+(off-1)*4) + get_byte(pg,25+(off-1)*4)*256
+ get_byte(pg,26+(off-1)*4)*65536 +
get_byte(pg,27+(off-1)*4)*16777216 AS word
FROM g, generate_series(1,3) off)
SELECT blk, off, word & 32767 AS lp_off, (word/32768) & 3 AS lp_flags,
word/131072 AS lp_len
FROM w ORDER BY blk, off;"
echo "### 5. pageinspect underflows on the zero-length item"
$PSQL -Atc "SELECT itemlen FROM bt_page_items('t_v_idx', 1);" 2>&1 | head -2
| sed 's/^/ /'
echo "### 6. silently wrong query results"
echo -n " indexscan: "; $PSQL -Atc \
"SET enable_seqscan=off; SELECT count(*) FROM t WHERE v = (SELECT v FROM
pad);"
echo -n " seqscan: "; $PSQL -Atc \
"SET enable_indexscan=off; SET enable_bitmapscan=off;
SELECT count(*) FROM t WHERE v = (SELECT v FROM pad);"
echo "### 7. the split record is WAL-logged without a full-page image"
$PSQL -Atc "SELECT pg_switch_wal();" >/dev/null
for f in $(ls "$PGDATA/pg_wal" | grep -E '^[0-9A-F]{24}$'); do
$PGBIN/pg_waldump -p "$PGDATA/pg_wal" "$f" 2>/dev/null | grep "SPLIT_R"
done | tail -1 | sed 's/^/ /'
echo "### 8. crash + restart => recovery segfaults, cluster will not come
up"
$PGBIN/pg_ctl -D "$PGDATA" stop -m immediate >/dev/null 2>&1
: > "$PGDATA/server.log"
$PGBIN/pg_ctl -D "$PGDATA" -l "$PGDATA/server.log" start -w -t 20 >/dev/null
2>&1
sed 's/^/ /' "$PGDATA/server.log"
echo "### 9. gdb backtrace of the recovery crash"
if command -v gdb >/dev/null; then
cd "$SOCKDIR" && timeout 180 gdb --batch -q \
-ex 'run' -ex 'bt 4' \
-ex 'p $_siginfo._sifields._sigfault.si_addr' \
--args "$PGBIN/postgres" --single -D "$PGDATA" postgres </dev/null
2>&1 \
| grep -E "SIGSEGV|^#|\\\$1" | sed 's/^/ /'
fi
王飞 <wf13934952799@gmail.com> 于2026年11月3日周二 16:10写道:
> CREATE EXTENSION amcheck;
>
> -- One incompressible 8172-byte value. The resulting index tuple is
> -- MAXALIGN(sizeof(IndexTupleData) + VARHDRSZ + 8172) =
> MAXALIGN(8+4+8172) = 8184
> -- bytes, which is the largest index tuple index_form_tuple() will ever
> produce.
> CREATE TABLE pad AS
> SELECT substr((SELECT string_agg(md5(random()::text), '')
> FROM generate_series(1, 300)), 1, 8172) AS v;
>
> CREATE TABLE t (v text);
> -- Must come before CREATE INDEX: the index inherits attstorage, and this
> stops
> -- TOAST_INDEX_HACK in index_form_tuple() from compressing the value.
> ALTER TABLE t ALTER COLUMN v SET STORAGE EXTERNAL;
> CREATE INDEX t_v_idx ON t (v);
>
> -- Four identical keys. Three fill the leaf page; the fourth splits it.
> Since
> -- all keys are equal, _bt_truncate() cannot truncate away any key
> attribute and
> -- must append a tiebreaker heap TID: newsize = 8184 + 8 = 8192.
> INSERT INTO t SELECT v FROM pad, generate_series(1, 4);
>
> SELECT bt_index_check('t_v_idx', true);
> ERROR: invalid line pointer storage in index "t_v_idx"
> DETAIL: Index tid=(3,2) lp_off=32744, lp_len=0 lp_flags=1.
>
> PG Bug reporting form <noreply@postgresql.org> 于2026年11月3日周二 15:47写道:
>
>> The following bug has been logged on the website:
>>
>> Bug reference: 19829
>> Logged by: wangfei
>> Email address: wf13934952799@gmail.com
>> PostgreSQL version: 18.6
>> Operating system: ubuntu-24.04
>> Description:
>>
>> PostgreSQL version: 19devel (also reproducible on 18.x)
>> Operating system: Linux x86_64
>> Configure options: --with-blocksize=32
>>
>> With BLCKSZ=32768, BTMaxItemSize() evaluates to roughly 10896 bytes, but
>> the
>> tuple length field reserved by INDEX_SIZE_MASK in IndexTupleData.t_info
>> can
>> only represent 8191 bytes. index_form_tuple() enforces the 8191 limit,
>> so a
>> leaf tuple is at most 8184 bytes after MAXALIGN. _bt_truncate() may then
>> need
>> to append a tiebreaker heap TID:
>>
>> newsize = MAXALIGN(IndexTupleSize(pivot)) +
>> MAXALIGN(sizeof(ItemPointerData));
>> ...
>> tidpivot->t_info &= ~INDEX_SIZE_MASK;
>> tidpivot->t_info |= newsize;
>>
>> For a maximally sized leaf tuple this is 8184 + 8 = 8192, i.e. exactly
>> INDEX_SIZE_MASK + 1, so the length field wraps to zero. The result is a
>> zero-length pivot tuple used as a leaf high key; PageAddItem() and
>> XLogRegisterBufData() both accept it without complaint, so the corruption
>> is
>> written out and WAL-logged silently.
>>
>> BTMaxItemSize() is supposed to keep leaf tuples small enough that this
>> cannot
>> happen, but the one-third-of-a-page rule scales with BLCKSZ while
>> INDEX_SIZE_MASK does not, so at BLCKSZ=32768 the one-third rule is no
>> longer
>> the binding constraint. Block sizes of 16384 and below are unaffected.
>>
>> Steps to reproduce:
>> < SQL 脚本>
>>
>> Expected: either the insert is rejected, or the pivot tuple is stored
>> correctly.
>> Actual: high key with length 0 (amcheck reports: <具体报错>).
>>
>> A patch against master is attached. It takes Min() of the one-third rule
>> and
>> what t_info can represent in BTMaxItemSize(), and adds a defensive
>> elog(ERROR)
>> in _bt_truncate() so that a future miscalculation cannot corrupt an index
>> silently. Since index_form_tuple() already caps index tuples at 8184
>> bytes,
>> lowering the BLCKSZ=32768 limit cannot invalidate any index that could
>> actually exist on disk, so there is no on-disk compatibility concern.
>>
>>
>>
^ permalink raw reply [nested|flat] 3+ messages in thread
end of thread, other threads:[~2026-11-03 08:12 UTC | newest]
Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-11-03 07:46 BUG #19829: BTREE pivot tuple length overflows t_info when BLCKSZ=32768 PG Bug reporting form <noreply@postgresql.org>
2026-11-03 08:10 ` 王飞 <wf13934952799@gmail.com>
2026-11-03 08:12 ` 王飞 <wf13934952799@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox