agora inbox for pgsql-committers@postgresql.orghelp / color / mirror / Atom feed
pgsql: Fix int32 overflow in ltree_compare() 6+ messages / 1 participants [nested] [flat]
* pgsql: Fix int32 overflow in ltree_compare() @ 2026-06-16 06:37 Heikki Linnakangas <heikki.linnakangas@iki.fi> 0 siblings, 0 replies; 6+ messages in thread From: Heikki Linnakangas @ 2026-06-16 06:37 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix int32 overflow in ltree_compare() The expression (len_diff * 10 * (an + 1)) used as the return value of ltree_compare() is computed at int32 width. With LTREE_MAX_LEVELS = 65535, the product can exceed INT32_MAX once an ltree has more than ~14,653 levels, which causes the result to wrap and invert its sign. That corrupts btree ordering as well as the "magnitude" consumed by ltree_penalty() for GiST page splits. To fix, split ltree_compare() into two functions. The new ltree_compare_distance() function returns a float, which won't overflow. It's used by the ltree_penalty() caller. All the other callers only care about the sign of the return value, i.e. which of the arguments is greater, so change ltree_compare() to not multiply the result with (10 * (an + 1)), which avoids the overflow for those callers. Existing btree or GiST indexes on ltree columns containing values with more than ~14,653 levels may be corrupt and should be REINDEXed. Add a regression test based on the reporter's PoC. Author: Ayush Tiwari <ayushtiwari.slg01@gmail.com> Reported-by: 王跃林 <violin0613@tju.edu.cn> Discussion: https://www.postgresql.org/message-id/AI6AnABgKW93Qbx1jVzi84r9.8.1781322625756.Hmail.3020001251%40tj... Backpatch-through: 14 Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/3f32804918383e48e9ce1d0f1b396775ab312d52 Modified Files -------------- contrib/ltree/expected/ltree.out | 10 ++++++++ contrib/ltree/ltree.h | 1 + contrib/ltree/ltree_gist.c | 6 ++--- contrib/ltree/ltree_op.c | 49 +++++++++++++++++++++++++++++++++++----- contrib/ltree/sql/ltree.sql | 6 +++++ 5 files changed, 63 insertions(+), 9 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix int32 overflow in ltree_compare() @ 2026-06-16 06:37 Heikki Linnakangas <heikki.linnakangas@iki.fi> 0 siblings, 0 replies; 6+ messages in thread From: Heikki Linnakangas @ 2026-06-16 06:37 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix int32 overflow in ltree_compare() The expression (len_diff * 10 * (an + 1)) used as the return value of ltree_compare() is computed at int32 width. With LTREE_MAX_LEVELS = 65535, the product can exceed INT32_MAX once an ltree has more than ~14,653 levels, which causes the result to wrap and invert its sign. That corrupts btree ordering as well as the "magnitude" consumed by ltree_penalty() for GiST page splits. To fix, split ltree_compare() into two functions. The new ltree_compare_distance() function returns a float, which won't overflow. It's used by the ltree_penalty() caller. All the other callers only care about the sign of the return value, i.e. which of the arguments is greater, so change ltree_compare() to not multiply the result with (10 * (an + 1)), which avoids the overflow for those callers. Existing btree or GiST indexes on ltree columns containing values with more than ~14,653 levels may be corrupt and should be REINDEXed. Add a regression test based on the reporter's PoC. Author: Ayush Tiwari <ayushtiwari.slg01@gmail.com> Reported-by: 王跃林 <violin0613@tju.edu.cn> Discussion: https://www.postgresql.org/message-id/AI6AnABgKW93Qbx1jVzi84r9.8.1781322625756.Hmail.3020001251%40tj... Backpatch-through: 14 Branch ------ REL_18_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/c3e36a9a5f19bb7c2df07bc70d799a8bca682d8b Modified Files -------------- contrib/ltree/expected/ltree.out | 10 ++++++++ contrib/ltree/ltree.h | 1 + contrib/ltree/ltree_gist.c | 6 ++--- contrib/ltree/ltree_op.c | 49 +++++++++++++++++++++++++++++++++++----- contrib/ltree/sql/ltree.sql | 6 +++++ 5 files changed, 63 insertions(+), 9 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix int32 overflow in ltree_compare() @ 2026-06-16 06:37 Heikki Linnakangas <heikki.linnakangas@iki.fi> 0 siblings, 0 replies; 6+ messages in thread From: Heikki Linnakangas @ 2026-06-16 06:37 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix int32 overflow in ltree_compare() The expression (len_diff * 10 * (an + 1)) used as the return value of ltree_compare() is computed at int32 width. With LTREE_MAX_LEVELS = 65535, the product can exceed INT32_MAX once an ltree has more than ~14,653 levels, which causes the result to wrap and invert its sign. That corrupts btree ordering as well as the "magnitude" consumed by ltree_penalty() for GiST page splits. To fix, split ltree_compare() into two functions. The new ltree_compare_distance() function returns a float, which won't overflow. It's used by the ltree_penalty() caller. All the other callers only care about the sign of the return value, i.e. which of the arguments is greater, so change ltree_compare() to not multiply the result with (10 * (an + 1)), which avoids the overflow for those callers. Existing btree or GiST indexes on ltree columns containing values with more than ~14,653 levels may be corrupt and should be REINDEXed. Add a regression test based on the reporter's PoC. Author: Ayush Tiwari <ayushtiwari.slg01@gmail.com> Reported-by: 王跃林 <violin0613@tju.edu.cn> Discussion: https://www.postgresql.org/message-id/AI6AnABgKW93Qbx1jVzi84r9.8.1781322625756.Hmail.3020001251%40tj... Backpatch-through: 14 Branch ------ REL_17_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/c391c00d9dd792e7f8d385ea05400c50eca82d78 Modified Files -------------- contrib/ltree/expected/ltree.out | 10 ++++++++ contrib/ltree/ltree.h | 1 + contrib/ltree/ltree_gist.c | 6 ++--- contrib/ltree/ltree_op.c | 49 +++++++++++++++++++++++++++++++++++----- contrib/ltree/sql/ltree.sql | 6 +++++ 5 files changed, 63 insertions(+), 9 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix int32 overflow in ltree_compare() @ 2026-06-16 06:37 Heikki Linnakangas <heikki.linnakangas@iki.fi> 0 siblings, 0 replies; 6+ messages in thread From: Heikki Linnakangas @ 2026-06-16 06:37 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix int32 overflow in ltree_compare() The expression (len_diff * 10 * (an + 1)) used as the return value of ltree_compare() is computed at int32 width. With LTREE_MAX_LEVELS = 65535, the product can exceed INT32_MAX once an ltree has more than ~14,653 levels, which causes the result to wrap and invert its sign. That corrupts btree ordering as well as the "magnitude" consumed by ltree_penalty() for GiST page splits. To fix, split ltree_compare() into two functions. The new ltree_compare_distance() function returns a float, which won't overflow. It's used by the ltree_penalty() caller. All the other callers only care about the sign of the return value, i.e. which of the arguments is greater, so change ltree_compare() to not multiply the result with (10 * (an + 1)), which avoids the overflow for those callers. Existing btree or GiST indexes on ltree columns containing values with more than ~14,653 levels may be corrupt and should be REINDEXed. Add a regression test based on the reporter's PoC. Author: Ayush Tiwari <ayushtiwari.slg01@gmail.com> Reported-by: 王跃林 <violin0613@tju.edu.cn> Discussion: https://www.postgresql.org/message-id/AI6AnABgKW93Qbx1jVzi84r9.8.1781322625756.Hmail.3020001251%40tj... Backpatch-through: 14 Branch ------ REL_16_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/aca944e3378614bacd2eb6ceb20afab7c4892de9 Modified Files -------------- contrib/ltree/expected/ltree.out | 10 ++++++++ contrib/ltree/ltree.h | 1 + contrib/ltree/ltree_gist.c | 6 ++--- contrib/ltree/ltree_op.c | 49 +++++++++++++++++++++++++++++++++++----- contrib/ltree/sql/ltree.sql | 6 +++++ 5 files changed, 63 insertions(+), 9 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix int32 overflow in ltree_compare() @ 2026-06-16 06:37 Heikki Linnakangas <heikki.linnakangas@iki.fi> 0 siblings, 0 replies; 6+ messages in thread From: Heikki Linnakangas @ 2026-06-16 06:37 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix int32 overflow in ltree_compare() The expression (len_diff * 10 * (an + 1)) used as the return value of ltree_compare() is computed at int32 width. With LTREE_MAX_LEVELS = 65535, the product can exceed INT32_MAX once an ltree has more than ~14,653 levels, which causes the result to wrap and invert its sign. That corrupts btree ordering as well as the "magnitude" consumed by ltree_penalty() for GiST page splits. To fix, split ltree_compare() into two functions. The new ltree_compare_distance() function returns a float, which won't overflow. It's used by the ltree_penalty() caller. All the other callers only care about the sign of the return value, i.e. which of the arguments is greater, so change ltree_compare() to not multiply the result with (10 * (an + 1)), which avoids the overflow for those callers. Existing btree or GiST indexes on ltree columns containing values with more than ~14,653 levels may be corrupt and should be REINDEXed. Add a regression test based on the reporter's PoC. Author: Ayush Tiwari <ayushtiwari.slg01@gmail.com> Reported-by: 王跃林 <violin0613@tju.edu.cn> Discussion: https://www.postgresql.org/message-id/AI6AnABgKW93Qbx1jVzi84r9.8.1781322625756.Hmail.3020001251%40tj... Backpatch-through: 14 Branch ------ REL_15_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/1bec6b1c146cfd9ef52623178da0e873f1238d31 Modified Files -------------- contrib/ltree/expected/ltree.out | 10 ++++++++ contrib/ltree/ltree.h | 1 + contrib/ltree/ltree_gist.c | 6 ++--- contrib/ltree/ltree_op.c | 49 +++++++++++++++++++++++++++++++++++----- contrib/ltree/sql/ltree.sql | 6 +++++ 5 files changed, 63 insertions(+), 9 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix int32 overflow in ltree_compare() @ 2026-06-16 06:37 Heikki Linnakangas <heikki.linnakangas@iki.fi> 0 siblings, 0 replies; 6+ messages in thread From: Heikki Linnakangas @ 2026-06-16 06:37 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix int32 overflow in ltree_compare() The expression (len_diff * 10 * (an + 1)) used as the return value of ltree_compare() is computed at int32 width. With LTREE_MAX_LEVELS = 65535, the product can exceed INT32_MAX once an ltree has more than ~14,653 levels, which causes the result to wrap and invert its sign. That corrupts btree ordering as well as the "magnitude" consumed by ltree_penalty() for GiST page splits. To fix, split ltree_compare() into two functions. The new ltree_compare_distance() function returns a float, which won't overflow. It's used by the ltree_penalty() caller. All the other callers only care about the sign of the return value, i.e. which of the arguments is greater, so change ltree_compare() to not multiply the result with (10 * (an + 1)), which avoids the overflow for those callers. Existing btree or GiST indexes on ltree columns containing values with more than ~14,653 levels may be corrupt and should be REINDEXed. Add a regression test based on the reporter's PoC. Author: Ayush Tiwari <ayushtiwari.slg01@gmail.com> Reported-by: 王跃林 <violin0613@tju.edu.cn> Discussion: https://www.postgresql.org/message-id/AI6AnABgKW93Qbx1jVzi84r9.8.1781322625756.Hmail.3020001251%40tj... Backpatch-through: 14 Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/f528a5606a836289c8cf3faa09874439b6f79c8f Modified Files -------------- contrib/ltree/expected/ltree.out | 10 ++++++++ contrib/ltree/ltree.h | 1 + contrib/ltree/ltree_gist.c | 6 ++--- contrib/ltree/ltree_op.c | 49 +++++++++++++++++++++++++++++++++++----- contrib/ltree/sql/ltree.sql | 6 +++++ 5 files changed, 63 insertions(+), 9 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2026-06-16 06:37 UTC | newest] Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2026-06-16 06:37 pgsql: Fix int32 overflow in ltree_compare() Heikki Linnakangas <heikki.linnakangas@iki.fi> 2026-06-16 06:37 pgsql: Fix int32 overflow in ltree_compare() Heikki Linnakangas <heikki.linnakangas@iki.fi> 2026-06-16 06:37 pgsql: Fix int32 overflow in ltree_compare() Heikki Linnakangas <heikki.linnakangas@iki.fi> 2026-06-16 06:37 pgsql: Fix int32 overflow in ltree_compare() Heikki Linnakangas <heikki.linnakangas@iki.fi> 2026-06-16 06:37 pgsql: Fix int32 overflow in ltree_compare() Heikki Linnakangas <heikki.linnakangas@iki.fi> 2026-06-16 06:37 pgsql: Fix int32 overflow in ltree_compare() Heikki Linnakangas <heikki.linnakangas@iki.fi>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox