pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: Fix selectivity estimation integer overflow in contrib/intarray 6+ messages / 1 participants [nested] [flat]
* pgsql: Fix selectivity estimation integer overflow in contrib/intarray @ 2026-01-04 07:33 David Rowley <drowley@postgresql.org> 0 siblings, 0 replies; 6+ messages in thread From: David Rowley @ 2026-01-04 07:33 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix selectivity estimation integer overflow in contrib/intarray This fixes a poorly written integer comparison function which was performing subtraction in an attempt to return a negative value when a < b and a positive value when a > b, and 0 when the values were equal. Unfortunately that didn't always work correctly due to two's complement having the INT_MIN 1 further from zero than INT_MAX. This could result in an overflow and cause the comparison function to return an incorrect result, which would result in the binary search failing to find the value being searched for. This could cause poor selectivity estimates when the statistics stored the value of INT_MAX (2147483647) and the value being searched for was large enough to result in the binary search doing a comparison with that INT_MAX value. Author: Chao Li <li.evan.chao@gmail.com> Reviewed-by: David Rowley <dgrowleyml@gmail.com> Discussion: https://postgr.es/m/CAEoWx2ng1Ot5LoKbVU-Dh---dFTUZWJRH8wv2chBu29fnNDMaQ@mail.gmail.com Backpatch-through: 14 Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/4f49e4b55e709e9aa758312da5b84bbbb674f30a Modified Files -------------- contrib/intarray/_int_selfuncs.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix selectivity estimation integer overflow in contrib/intarray @ 2026-01-04 07:33 David Rowley <drowley@postgresql.org> 0 siblings, 0 replies; 6+ messages in thread From: David Rowley @ 2026-01-04 07:33 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix selectivity estimation integer overflow in contrib/intarray This fixes a poorly written integer comparison function which was performing subtraction in an attempt to return a negative value when a < b and a positive value when a > b, and 0 when the values were equal. Unfortunately that didn't always work correctly due to two's complement having the INT_MIN 1 further from zero than INT_MAX. This could result in an overflow and cause the comparison function to return an incorrect result, which would result in the binary search failing to find the value being searched for. This could cause poor selectivity estimates when the statistics stored the value of INT_MAX (2147483647) and the value being searched for was large enough to result in the binary search doing a comparison with that INT_MAX value. Author: Chao Li <li.evan.chao@gmail.com> Reviewed-by: David Rowley <dgrowleyml@gmail.com> Discussion: https://postgr.es/m/CAEoWx2ng1Ot5LoKbVU-Dh---dFTUZWJRH8wv2chBu29fnNDMaQ@mail.gmail.com Backpatch-through: 14 Branch ------ REL_18_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/07c1c6ec51a4474c22abbee731cfc8111fc09a43 Modified Files -------------- contrib/intarray/_int_selfuncs.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix selectivity estimation integer overflow in contrib/intarray @ 2026-01-04 07:33 David Rowley <drowley@postgresql.org> 0 siblings, 0 replies; 6+ messages in thread From: David Rowley @ 2026-01-04 07:33 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix selectivity estimation integer overflow in contrib/intarray This fixes a poorly written integer comparison function which was performing subtraction in an attempt to return a negative value when a < b and a positive value when a > b, and 0 when the values were equal. Unfortunately that didn't always work correctly due to two's complement having the INT_MIN 1 further from zero than INT_MAX. This could result in an overflow and cause the comparison function to return an incorrect result, which would result in the binary search failing to find the value being searched for. This could cause poor selectivity estimates when the statistics stored the value of INT_MAX (2147483647) and the value being searched for was large enough to result in the binary search doing a comparison with that INT_MAX value. Author: Chao Li <li.evan.chao@gmail.com> Reviewed-by: David Rowley <dgrowleyml@gmail.com> Discussion: https://postgr.es/m/CAEoWx2ng1Ot5LoKbVU-Dh---dFTUZWJRH8wv2chBu29fnNDMaQ@mail.gmail.com Backpatch-through: 14 Branch ------ REL_17_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/a5f2dc421f7f8ed9588cf0a32566c71b9d8f52c6 Modified Files -------------- contrib/intarray/_int_selfuncs.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix selectivity estimation integer overflow in contrib/intarray @ 2026-01-04 07:34 David Rowley <drowley@postgresql.org> 0 siblings, 0 replies; 6+ messages in thread From: David Rowley @ 2026-01-04 07:34 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix selectivity estimation integer overflow in contrib/intarray This fixes a poorly written integer comparison function which was performing subtraction in an attempt to return a negative value when a < b and a positive value when a > b, and 0 when the values were equal. Unfortunately that didn't always work correctly due to two's complement having the INT_MIN 1 further from zero than INT_MAX. This could result in an overflow and cause the comparison function to return an incorrect result, which would result in the binary search failing to find the value being searched for. This could cause poor selectivity estimates when the statistics stored the value of INT_MAX (2147483647) and the value being searched for was large enough to result in the binary search doing a comparison with that INT_MAX value. Author: Chao Li <li.evan.chao@gmail.com> Reviewed-by: David Rowley <dgrowleyml@gmail.com> Discussion: https://postgr.es/m/CAEoWx2ng1Ot5LoKbVU-Dh---dFTUZWJRH8wv2chBu29fnNDMaQ@mail.gmail.com Backpatch-through: 14 Branch ------ REL_16_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/54f82c4aae768037fe14d909b2b535cfbe89f900 Modified Files -------------- contrib/intarray/_int_selfuncs.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix selectivity estimation integer overflow in contrib/intarray @ 2026-01-04 07:34 David Rowley <drowley@postgresql.org> 0 siblings, 0 replies; 6+ messages in thread From: David Rowley @ 2026-01-04 07:34 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix selectivity estimation integer overflow in contrib/intarray This fixes a poorly written integer comparison function which was performing subtraction in an attempt to return a negative value when a < b and a positive value when a > b, and 0 when the values were equal. Unfortunately that didn't always work correctly due to two's complement having the INT_MIN 1 further from zero than INT_MAX. This could result in an overflow and cause the comparison function to return an incorrect result, which would result in the binary search failing to find the value being searched for. This could cause poor selectivity estimates when the statistics stored the value of INT_MAX (2147483647) and the value being searched for was large enough to result in the binary search doing a comparison with that INT_MAX value. Author: Chao Li <li.evan.chao@gmail.com> Reviewed-by: David Rowley <dgrowleyml@gmail.com> Discussion: https://postgr.es/m/CAEoWx2ng1Ot5LoKbVU-Dh---dFTUZWJRH8wv2chBu29fnNDMaQ@mail.gmail.com Backpatch-through: 14 Branch ------ REL_15_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/c89510431af2efe447d3f069d1f6ea417efdf62e Modified Files -------------- contrib/intarray/_int_selfuncs.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix selectivity estimation integer overflow in contrib/intarray @ 2026-01-04 07:35 David Rowley <drowley@postgresql.org> 0 siblings, 0 replies; 6+ messages in thread From: David Rowley @ 2026-01-04 07:35 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix selectivity estimation integer overflow in contrib/intarray This fixes a poorly written integer comparison function which was performing subtraction in an attempt to return a negative value when a < b and a positive value when a > b, and 0 when the values were equal. Unfortunately that didn't always work correctly due to two's complement having the INT_MIN 1 further from zero than INT_MAX. This could result in an overflow and cause the comparison function to return an incorrect result, which would result in the binary search failing to find the value being searched for. This could cause poor selectivity estimates when the statistics stored the value of INT_MAX (2147483647) and the value being searched for was large enough to result in the binary search doing a comparison with that INT_MAX value. Author: Chao Li <li.evan.chao@gmail.com> Reviewed-by: David Rowley <dgrowleyml@gmail.com> Discussion: https://postgr.es/m/CAEoWx2ng1Ot5LoKbVU-Dh---dFTUZWJRH8wv2chBu29fnNDMaQ@mail.gmail.com Backpatch-through: 14 Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/67a6fd5074812af4af93a08f823a0885584fd6f1 Modified Files -------------- contrib/intarray/_int_selfuncs.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2026-01-04 07:35 UTC | newest] Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2026-01-04 07:33 pgsql: Fix selectivity estimation integer overflow in contrib/intarray David Rowley <drowley@postgresql.org> 2026-01-04 07:33 pgsql: Fix selectivity estimation integer overflow in contrib/intarray David Rowley <drowley@postgresql.org> 2026-01-04 07:33 pgsql: Fix selectivity estimation integer overflow in contrib/intarray David Rowley <drowley@postgresql.org> 2026-01-04 07:34 pgsql: Fix selectivity estimation integer overflow in contrib/intarray David Rowley <drowley@postgresql.org> 2026-01-04 07:34 pgsql: Fix selectivity estimation integer overflow in contrib/intarray David Rowley <drowley@postgresql.org> 2026-01-04 07:35 pgsql: Fix selectivity estimation integer overflow in contrib/intarray David Rowley <drowley@postgresql.org>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox