agora inbox for pgsql-committers@postgresql.org
help / color / mirror / Atom feedFrom: Heikki Linnakangas <heikki.linnakangas@iki.fi>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Fix int32 overflow in ltree_compare()
Date: Tue, 16 Jun 2026 06:37:31 +0000
Message-ID: <E1wZNQR-000WT5-1n@gemulon.postgresql.org> (raw)
Fix int32 overflow in ltree_compare()
The expression (len_diff * 10 * (an + 1)) used as the return value of
ltree_compare() is computed at int32 width. With LTREE_MAX_LEVELS =
65535, the product can exceed INT32_MAX once an ltree has more than
~14,653 levels, which causes the result to wrap and invert its sign.
That corrupts btree ordering as well as the "magnitude" consumed by
ltree_penalty() for GiST page splits.
To fix, split ltree_compare() into two functions. The new
ltree_compare_distance() function returns a float, which won't
overflow. It's used by the ltree_penalty() caller. All the other
callers only care about the sign of the return value, i.e. which of
the arguments is greater, so change ltree_compare() to not multiply
the result with (10 * (an + 1)), which avoids the overflow for those
callers.
Existing btree or GiST indexes on ltree columns containing values with
more than ~14,653 levels may be corrupt and should be REINDEXed.
Add a regression test based on the reporter's PoC.
Author: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Reported-by: 王跃林 <violin0613@tju.edu.cn>
Discussion: https://www.postgresql.org/message-id/AI6AnABgKW93Qbx1jVzi84r9.8.1781322625756.Hmail.3020001251%40tj...
Backpatch-through: 14
Branch
------
REL_14_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/f528a5606a836289c8cf3faa09874439b6f79c8f
Modified Files
--------------
contrib/ltree/expected/ltree.out | 10 ++++++++
contrib/ltree/ltree.h | 1 +
contrib/ltree/ltree_gist.c | 6 ++---
contrib/ltree/ltree_op.c | 49 +++++++++++++++++++++++++++++++++++-----
contrib/ltree/sql/ltree.sql | 6 +++++
5 files changed, 63 insertions(+), 9 deletions(-)
view thread (6+ messages)
Message-ID: <E1wZNQR-000WT5-1n@gemulon.postgresql.org>
Permalink: ../E1wZNQR-000WT5-1n@gemulon.postgresql.org/
Also on: postgresql.org/message-id/E1wZNQR-000WT5-1n@gemulon.postgresql.org
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-committers@postgresql.org
Cc: heikki.linnakangas@iki.fi, pgsql-committers@lists.postgresql.org
Subject: Re: pgsql: Fix int32 overflow in ltree_compare()
In-Reply-To: <E1wZNQR-000WT5-1n@gemulon.postgresql.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox