public inbox for [email protected]
help / color / mirror / Atom feedFrom: Ewan Young <[email protected]>
To: [email protected]
To: [email protected]
Subject: Re: BUG #19545: Integer truncation of `GinTuple.keylen` causes out-of-bounds read in parallel GIN index build
Date: Wed, 8 Jul 2026 14:27:55 +0800
Message-ID: <CAON2xHMXYLwkq4UaNbUtmaw1ZKne6OqdyyiACFQUbCj+XRBvQQ@mail.gmail.com> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
Hi Yuelin,
Thanks for the very precise report -- I reproduced it on master and your
analysis is exactly right. _gin_build_tuple() builds the whole GinTuple
(palloc size, key memcpy, TID-list offset) from the int keylen, but the
stored GinTuple.keylen is uint16, so a key wider than 65535 bytes has its
stored length truncated. On read-back GinTupleGetFirst() and
_gin_parse_tuple_items() recompute the posting-list offset from the
truncated value, and ginPostingListDecodeAllSegments() then walks the key
bytes, aborting (or reading past the allocation on non-assert builds)
exactly as you saw. It's parallel-only because only the parallel path
serializes a GinTuple.
I went with your fix A -- widening keylen to uint32 (attached). It's the
minimal root-cause fix: the stored length now matches the length the rest
of the function already uses. I preferred it over an explicit ereport at
UINT16_MAX, since 65535 isn't a meaningful GIN limit -- the entry-tree item
limit is much smaller and is applied to the (compressed) tuple by
GinFormTuple() -- so rejecting there would be an arbitrary cutoff.
Verified on master with your PoC:
- Before: the parallel build aborts in ginPostingListDecodeAllSegments
(Assert at ginpostinglist.c:324), as reported.
- After: the parallel build succrrect
results -- your 100000-byte element compresses well under the
entry-tree item limit, so it irial build.
- A genuinely incompressible key wider than the item limit now fails
cleanly ("index row requires N) in both
the parallel and serial paths, instead of crashing.
I didn't add a regression test -- forcing parallel maintenance workers plus
a >64 kB key deterministically in -- but I'm
happy to add one if preferred.
Attaching v1 against master; this should go back to the branches that have
parallel GIN builds.
On Wed, Jul 8, 2026 at 7:58 AM PG Bug reporting form
<[email protected]> wrote:
>
> The following bug has been logged on the website:
>
> Bug reference: 19545
> Logged by: Yuelin Wang
> Email address: [email protected]
> PostgreSQL version: 19beta1
> Operating system: Linux (Ubuntu 24.04, x86_64)
> Description:
>
> ### Summary
>
> `GinTuple.keylen` is declared `uint16` (max 65535), but in
> `_gin_build_tuple()` the local `keylen` is an `int` taken from
> `VARSIZE_ANY(key)` (up to ~1 GB for a varlena `NORM_KEY`). The store
> `tuple->keylen = keylen` **truncates** when the key exceeds 65535 bytes. The
> tuple's *memory layout* (palloc size, key memcpy, TID-array pointer) uses
> the full `int` keylen and is correct. Only the stored field is truncated. On
> read-back the truncated value mis-computes the posting-list region, so the
> decoder walks attacker-controlled bytes past the allocation, which is a heap
> out-of-bounds read.
>
> ### Details
>
> ```c
> int keylen; /* 2251 */
> keylen = VARSIZE_ANY(DatumGetPointer(key)); /* 2278: up to ~1GB */
> tuple->keylen = keylen; /* 2327: stored into uint16 ->
> truncation */
> ```
>
> On read-back `_gin_parse_tuple_items()` (2419-2420) uses the truncated
> `a->keylen`, so the posting-list pointer lands ~64 KB early inside the
> attacker's key content and `ginPostingListDecodeAllSegments()` decodes
> attacker-controlled bytes off the end of the allocation. This is
> parallel-only because the `GinMaxItemSize` guard lives in the leader's
> `GinFormTuple`, whereas a parallel worker reparses the serialized `GinTuple`
> *before* any size gate (present in 19devel).
>
> ### Proof of Concept
>
> `array_ops`' `extractValue` returns full-size array-element datums, so a
> `text[]` element > 65535 bytes flows in as `NORM_KEY`:
>
> ```sql
> CREATE SCHEMA vuln_001_sch;
> CREATE TABLE vuln_001_sch.vuln_001_t (a text[]);
>
> -- one 100000-byte element per row (> 65535 -> keylen truncation), enough
> rows for real sort work
> INSERT INTO vuln_001_sch.vuln_001_t
> SELECT ARRAY[repeat('x', 100000)] FROM generate_series(1, 300);
>
> -- force a parallel maintenance build (all settable by a non-superuser)
> ALTER TABLE vuln_001_sch.vuln_001_t SET (parallel_workers = 4);
> SET max_parallel_maintenance_workers = 4;
> SET min_parallel_table_scan_size = '0';
>
> CREATE INDEX vuln_001_idx ON vuln_001_sch.vuln_001_t USING gin (a);
> ```
>
> Run:
>
> ```
> psql -h /tmp -p 36901 -U ylwang -d postgres -f vuln_001.sql
> ```
>
> ### Result
>
> The parallel build crashed 3 workers (PIDs 114475 to 114477) during `CREATE
> INDEX ... USING gin`, on exactly the predicted path
> (`ginPostingListDecodeAllSegments`):
>
> ```
> DEBUG: building index "vuln_001_idx" ... with request for 4 parallel
> workers
> server closed the connection unexpectedly
> parallel worker ... ExceptionalCondition ...
> parallel worker ... ginPostingListDecodeAllSegments+0x163
> TRAP: failed
> Assert("OffsetNumberIsValid(ItemPointerGetOffsetNumber(&segment->first))"),
> File: "ginpostinglist.c", Line: 324
> ```
>
> ### Fix
>
> Widen the field to hold real key lengths by declaring `GinTuple.keylen` as
> `int`/`uint32` in `gin_tuple.h`. Alternatively, reject oversized keys
> explicitly in `_gin_build_tuple()` by calling `ereport(ERROR)` when `keylen
> > UINT16_MAX`, so the parallel path fails as cleanly as the serial one.
>
>
>
>
--
Regards,
Ewan Young
Attachments:
[application/octet-stream] v1-0001-Widen-GinTuple.keylen-to-uint32-to-fix-parallel-G.patch (2.4K, ../CAON2xHMXYLwkq4UaNbUtmaw1ZKne6OqdyyiACFQUbCj+XRBvQQ@mail.gmail.com/2-v1-0001-Widen-GinTuple.keylen-to-uint32-to-fix-parallel-G.patch)
download | inline diff:
From b78b6b6f0e71403cf93fb15d4ed4470a7dbb9ad9 Mon Sep 17 00:00:00 2001
From: Ewan Young <[email protected]>
Date: Wed, 8 Jul 2026 22:18:56 +0800
Subject: [PATCH v1] Widen GinTuple.keylen to uint32 to fix parallel GIN builds
with large keys
During a parallel GIN index build, _gin_build_tuple() computes the key
length as an int from VARSIZE_ANY() (up to ~1GB for a varlena key) and
lays out the serialized GinTuple -- its palloc size, the key memcpy, and
the offset of the compressed TID list -- using that full length. But
GinTuple.keylen was declared uint16, so the stored field wrapped around
for keys larger than 65535 bytes while the rest of the tuple used the
untruncated length.
On read-back GinTupleGetFirst() and _gin_parse_tuple_items() recompute
the TID-list offset from the truncated keylen, so it points into the key
data instead of at the compressed posting list.
ginPostingListDecodeAllSegments() then decodes arbitrary key bytes, which
aborts the build (Assert in ginpostinglist.c) or reads past the
allocation on non-assert builds. Any user able to run CREATE INDEX with
parallel workers can hit this, e.g. a gin index on a text[] column with
an element wider than 65535 bytes.
This is specific to parallel builds: the serial path never materializes a
GinTuple. The general index-tuple size limit is still enforced by
GinFormTuple() in both cases.
Widen keylen to uint32 so the stored length matches the length used to
build the tuple. A parallel build then correctly indexes large keys that
compress below the entry-tree item limit (as a serial build already does),
and still rejects genuinely oversized keys cleanly via GinFormTuple()
instead of crashing.
Reported-by: Yuelin Wang <[email protected]>
Bug: #19545
---
src/include/access/gin_tuple.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/include/access/gin_tuple.h b/src/include/access/gin_tuple.h
index 7bde05e2de4..a116f3a1a77 100644
--- a/src/include/access/gin_tuple.h
+++ b/src/include/access/gin_tuple.h
@@ -23,7 +23,7 @@ typedef struct GinTuple
{
int tuplen; /* length of the whole tuple */
OffsetNumber attrnum; /* attnum of index key */
- uint16 keylen; /* bytes in data for key value */
+ uint32 keylen; /* bytes in data for key value */
int16 typlen; /* typlen for key */
bool typbyval; /* typbyval for key */
signed char category; /* category: normal or NULL? */
--
2.47.3
view thread (9+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected]
Subject: Re: BUG #19545: Integer truncation of `GinTuple.keylen` causes out-of-bounds read in parallel GIN index build
In-Reply-To: <CAON2xHMXYLwkq4UaNbUtmaw1ZKne6OqdyyiACFQUbCj+XRBvQQ@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox