agora inbox for pgsql-committers@postgresql.org
help / color / mirror / Atom feedFrom: Peter Eisentraut <peter@eisentraut.org>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Fix parallel GIN index build with keys larger than 65535 bytes
Date: Mon, 14 Sep 2026 14:55:51 +0000
Message-ID: <E1x6863-00000000Ijf-2rKi@gemulon.postgresql.org> (raw)
Fix parallel GIN index build with keys larger than 65535 bytes
During a parallel GIN build, each key is serialized into a GinTuple, a
transient representation used only while sorting. _gin_build_tuple()
lays out the whole tuple (the palloc size, the key memcpy, and the
offset of the posting list) from a local variable holding the real key
length, but then stored that length in GinTuple.keylen, which was
uint16. For a key longer than 65535 bytes, the stored length was thus
silently truncated.
On read-back, GinTupleGetFirst() and _gin_parse_tuple_items()
recompute the posting-list offset from the truncated keylen and land
inside the key data, so ginPostingListDecodeAllSegments() decodes
garbage: an assertion failure with assertions enabled, and a read past
the end of the allocation without, which could in turn lead to a crash
or garbage being written into the index. Only parallel builds are
affected, because only they materialize a GinTuple; a serial build of
the same data succeeds, as index_form_tuple() compresses large keys
before the GinMaxItemSize check.
To fix, widen GinTuple.keylen to Size, which is what VARSIZE_ANY()
returns and what GinBuffer.keylen already uses, and use Size for the
local in _gin_build_tuple() too, which was an int that truncated
VARSIZE_ANY() the same way.
Widening keylen moves GinTuple.data. The key value is accessed in
place in data, so data must be MAXALIGN'ed; before, that was only true
by accident of the field layout. Make that explicit with
alignas(MAXIMUM_ALIGNOF) on data, so that the layout stays correct on
32-bit platforms, where Size is 4 bytes and would otherwise leave data
under-aligned when MAXIMUM_ALIGNOF is 8.
Bug: #19545
Author: Ewan Young <kdbase.hack@gmail.com>
Reported-by: Yuelin Wang <1217816127@qq.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://www.postgresql.org/message-id/flat/19545-0f25b7e47351e8fc%40postgresql.org
Branch
------
REL_19_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/05c4b10fc526a549223d963030057c2e5f709e43
Modified Files
--------------
src/backend/access/gin/gininsert.c | 2 +-
src/include/access/gin_tuple.h | 9 +++++++--
2 files changed, 8 insertions(+), 3 deletions(-)
view thread (3+ messages) latest in thread
Message-ID: <E1x6863-00000000Ijf-2rKi@gemulon.postgresql.org>
Permalink: ../E1x6863-00000000Ijf-2rKi@gemulon.postgresql.org/
Also on: postgresql.org/message-id/E1x6863-00000000Ijf-2rKi@gemulon.postgresql.org
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-committers@postgresql.org
Cc: peter@eisentraut.org, pgsql-committers@lists.postgresql.org
Subject: Re: pgsql: Fix parallel GIN index build with keys larger than 65535 bytes
In-Reply-To: <E1x6863-00000000Ijf-2rKi@gemulon.postgresql.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox