agora inbox for pgsql-committers@postgresql.orghelp / color / mirror / Atom feed
pgsql: btree_gist: fix NaN handling in float4/float8 opclasses. 7+ messages / 1 participants [nested] [flat]
* pgsql: btree_gist: fix NaN handling in float4/float8 opclasses. @ 2026-07-01 17:27 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 7+ messages in thread From: Tom Lane @ 2026-07-01 17:27 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org btree_gist: fix NaN handling in float4/float8 opclasses. The float4 and float8 btree_gist opclasses compared keys with raw C operators (==, <, >). IEEE 754 makes every comparison involving NaN false, so GiST disagreed with the regular float comparison operators and with the btree opclass, which uses float[4|8]_cmp_internal() (so that all NaNs are equal and NaN sorts after every non-NaN value). In addition, the penalty and distance functions were not careful about NaNs, and the penalty functions could also misbehave for IEEE infinities. Wrong answers from the penalty functions would probably do no more than make the index non-optimal, but the distance mistakes were visible from SQL. To fix, make the comparison functions rely on the same NaN-aware comparison functions the core code uses, and rewrite the penalty and distance functions to follow the rules that NaNs are equal but maximally far away from non-NaNs. The penalty_num() code was formerly shared between integral and float cases, but I chose to make two copies so that the integral cases are not saddled with the extra logic for NaNs and infinities/overflows. I also rewrote it as static inline functions instead of an unreadable and uncommented macro. The float penalty functions were previously unreached by the regression tests, so add new test cases to exercise them. There's no on-disk format change, but users who have NaN entries in a btree_gist index would be well advised to reindex it. Bug: #19501 Bug: #19524 Reported-by: Man Zeng <zengman@halodbtech.com> Reported-by: Yuelin Wang <3020001251@tju.edu.cn> Author: Bill Kim <billkimjh@gmail.com> Co-authored-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/19501-3bff3bbc97f1e7c9@postgresql.org Discussion: https://postgr.es/m/19524-9559d302c8455664@postgresql.org Discussion: https://postgr.es/m/CAMQXxcgbtD2LXfX0tpgvOizxP-XxrCHV2ZDy4By_TZnJMsxXWQ@mail.gmail.com Backpatch-through: 14 Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/7d3448961da3f8cb5c78b9d58c5e03b6bff53364 Modified Files -------------- contrib/btree_gist/btree_float4.c | 59 +++++++++----- contrib/btree_gist/btree_float8.c | 51 +++++++++---- contrib/btree_gist/btree_utils_num.h | 131 ++++++++++++++++++++++++++++---- contrib/btree_gist/data/float4.data | 3 + contrib/btree_gist/data/float8.data | 3 + contrib/btree_gist/expected/float4.out | 51 +++++++++++-- contrib/btree_gist/expected/float8.out | 51 +++++++++++-- contrib/btree_gist/expected/numeric.out | 48 ++++++------ contrib/btree_gist/sql/float4.sql | 17 +++++ contrib/btree_gist/sql/float8.sql | 17 +++++ 10 files changed, 345 insertions(+), 86 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: btree_gist: fix NaN handling in float4/float8 opclasses. @ 2026-07-01 17:27 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 7+ messages in thread From: Tom Lane @ 2026-07-01 17:27 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org btree_gist: fix NaN handling in float4/float8 opclasses. The float4 and float8 btree_gist opclasses compared keys with raw C operators (==, <, >). IEEE 754 makes every comparison involving NaN false, so GiST disagreed with the regular float comparison operators and with the btree opclass, which uses float[4|8]_cmp_internal() (so that all NaNs are equal and NaN sorts after every non-NaN value). In addition, the penalty and distance functions were not careful about NaNs, and the penalty functions could also misbehave for IEEE infinities. Wrong answers from the penalty functions would probably do no more than make the index non-optimal, but the distance mistakes were visible from SQL. To fix, make the comparison functions rely on the same NaN-aware comparison functions the core code uses, and rewrite the penalty and distance functions to follow the rules that NaNs are equal but maximally far away from non-NaNs. The penalty_num() code was formerly shared between integral and float cases, but I chose to make two copies so that the integral cases are not saddled with the extra logic for NaNs and infinities/overflows. I also rewrote it as static inline functions instead of an unreadable and uncommented macro. The float penalty functions were previously unreached by the regression tests, so add new test cases to exercise them. There's no on-disk format change, but users who have NaN entries in a btree_gist index would be well advised to reindex it. Bug: #19501 Bug: #19524 Reported-by: Man Zeng <zengman@halodbtech.com> Reported-by: Yuelin Wang <3020001251@tju.edu.cn> Author: Bill Kim <billkimjh@gmail.com> Co-authored-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/19501-3bff3bbc97f1e7c9@postgresql.org Discussion: https://postgr.es/m/19524-9559d302c8455664@postgresql.org Discussion: https://postgr.es/m/CAMQXxcgbtD2LXfX0tpgvOizxP-XxrCHV2ZDy4By_TZnJMsxXWQ@mail.gmail.com Backpatch-through: 14 Branch ------ REL_19_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/a47005f0b11df3e456c755c73b6f41fd26b27cad Modified Files -------------- contrib/btree_gist/btree_float4.c | 59 +++++++++----- contrib/btree_gist/btree_float8.c | 51 +++++++++---- contrib/btree_gist/btree_utils_num.h | 131 ++++++++++++++++++++++++++++---- contrib/btree_gist/data/float4.data | 3 + contrib/btree_gist/data/float8.data | 3 + contrib/btree_gist/expected/float4.out | 51 +++++++++++-- contrib/btree_gist/expected/float8.out | 51 +++++++++++-- contrib/btree_gist/expected/numeric.out | 48 ++++++------ contrib/btree_gist/sql/float4.sql | 17 +++++ contrib/btree_gist/sql/float8.sql | 17 +++++ 10 files changed, 345 insertions(+), 86 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: btree_gist: fix NaN handling in float4/float8 opclasses. @ 2026-07-01 17:27 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 7+ messages in thread From: Tom Lane @ 2026-07-01 17:27 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org btree_gist: fix NaN handling in float4/float8 opclasses. The float4 and float8 btree_gist opclasses compared keys with raw C operators (==, <, >). IEEE 754 makes every comparison involving NaN false, so GiST disagreed with the regular float comparison operators and with the btree opclass, which uses float[4|8]_cmp_internal() (so that all NaNs are equal and NaN sorts after every non-NaN value). In addition, the penalty and distance functions were not careful about NaNs, and the penalty functions could also misbehave for IEEE infinities. Wrong answers from the penalty functions would probably do no more than make the index non-optimal, but the distance mistakes were visible from SQL. To fix, make the comparison functions rely on the same NaN-aware comparison functions the core code uses, and rewrite the penalty and distance functions to follow the rules that NaNs are equal but maximally far away from non-NaNs. The penalty_num() code was formerly shared between integral and float cases, but I chose to make two copies so that the integral cases are not saddled with the extra logic for NaNs and infinities/overflows. I also rewrote it as static inline functions instead of an unreadable and uncommented macro. The float penalty functions were previously unreached by the regression tests, so add new test cases to exercise them. There's no on-disk format change, but users who have NaN entries in a btree_gist index would be well advised to reindex it. Bug: #19501 Bug: #19524 Reported-by: Man Zeng <zengman@halodbtech.com> Reported-by: Yuelin Wang <3020001251@tju.edu.cn> Author: Bill Kim <billkimjh@gmail.com> Co-authored-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/19501-3bff3bbc97f1e7c9@postgresql.org Discussion: https://postgr.es/m/19524-9559d302c8455664@postgresql.org Discussion: https://postgr.es/m/CAMQXxcgbtD2LXfX0tpgvOizxP-XxrCHV2ZDy4By_TZnJMsxXWQ@mail.gmail.com Backpatch-through: 14 Branch ------ REL_18_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/1e1d07792e0827ca84685784c0c958127f5853eb Modified Files -------------- contrib/btree_gist/btree_float4.c | 59 +++++++++----- contrib/btree_gist/btree_float8.c | 51 +++++++++---- contrib/btree_gist/btree_utils_num.h | 131 ++++++++++++++++++++++++++++---- contrib/btree_gist/data/float4.data | 3 + contrib/btree_gist/data/float8.data | 3 + contrib/btree_gist/expected/float4.out | 51 +++++++++++-- contrib/btree_gist/expected/float8.out | 51 +++++++++++-- contrib/btree_gist/expected/numeric.out | 48 ++++++------ contrib/btree_gist/sql/float4.sql | 17 +++++ contrib/btree_gist/sql/float8.sql | 17 +++++ 10 files changed, 345 insertions(+), 86 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: btree_gist: fix NaN handling in float4/float8 opclasses. @ 2026-07-01 17:27 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 7+ messages in thread From: Tom Lane @ 2026-07-01 17:27 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org btree_gist: fix NaN handling in float4/float8 opclasses. The float4 and float8 btree_gist opclasses compared keys with raw C operators (==, <, >). IEEE 754 makes every comparison involving NaN false, so GiST disagreed with the regular float comparison operators and with the btree opclass, which uses float[4|8]_cmp_internal() (so that all NaNs are equal and NaN sorts after every non-NaN value). In addition, the penalty and distance functions were not careful about NaNs, and the penalty functions could also misbehave for IEEE infinities. Wrong answers from the penalty functions would probably do no more than make the index non-optimal, but the distance mistakes were visible from SQL. To fix, make the comparison functions rely on the same NaN-aware comparison functions the core code uses, and rewrite the penalty and distance functions to follow the rules that NaNs are equal but maximally far away from non-NaNs. The penalty_num() code was formerly shared between integral and float cases, but I chose to make two copies so that the integral cases are not saddled with the extra logic for NaNs and infinities/overflows. I also rewrote it as static inline functions instead of an unreadable and uncommented macro. The float penalty functions were previously unreached by the regression tests, so add new test cases to exercise them. There's no on-disk format change, but users who have NaN entries in a btree_gist index would be well advised to reindex it. Bug: #19501 Bug: #19524 Reported-by: Man Zeng <zengman@halodbtech.com> Reported-by: Yuelin Wang <3020001251@tju.edu.cn> Author: Bill Kim <billkimjh@gmail.com> Co-authored-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/19501-3bff3bbc97f1e7c9@postgresql.org Discussion: https://postgr.es/m/19524-9559d302c8455664@postgresql.org Discussion: https://postgr.es/m/CAMQXxcgbtD2LXfX0tpgvOizxP-XxrCHV2ZDy4By_TZnJMsxXWQ@mail.gmail.com Backpatch-through: 14 Branch ------ REL_17_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/d215d2cc2ae181dbb201c017b43a9778c4bcf7f4 Modified Files -------------- contrib/btree_gist/btree_float4.c | 59 +++++++++----- contrib/btree_gist/btree_float8.c | 51 +++++++++---- contrib/btree_gist/btree_utils_num.h | 131 ++++++++++++++++++++++++++++---- contrib/btree_gist/data/float4.data | 3 + contrib/btree_gist/data/float8.data | 3 + contrib/btree_gist/expected/float4.out | 51 +++++++++++-- contrib/btree_gist/expected/float8.out | 51 +++++++++++-- contrib/btree_gist/expected/numeric.out | 48 ++++++------ contrib/btree_gist/sql/float4.sql | 17 +++++ contrib/btree_gist/sql/float8.sql | 17 +++++ 10 files changed, 345 insertions(+), 86 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: btree_gist: fix NaN handling in float4/float8 opclasses. @ 2026-07-01 17:27 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 7+ messages in thread From: Tom Lane @ 2026-07-01 17:27 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org btree_gist: fix NaN handling in float4/float8 opclasses. The float4 and float8 btree_gist opclasses compared keys with raw C operators (==, <, >). IEEE 754 makes every comparison involving NaN false, so GiST disagreed with the regular float comparison operators and with the btree opclass, which uses float[4|8]_cmp_internal() (so that all NaNs are equal and NaN sorts after every non-NaN value). In addition, the penalty and distance functions were not careful about NaNs, and the penalty functions could also misbehave for IEEE infinities. Wrong answers from the penalty functions would probably do no more than make the index non-optimal, but the distance mistakes were visible from SQL. To fix, make the comparison functions rely on the same NaN-aware comparison functions the core code uses, and rewrite the penalty and distance functions to follow the rules that NaNs are equal but maximally far away from non-NaNs. The penalty_num() code was formerly shared between integral and float cases, but I chose to make two copies so that the integral cases are not saddled with the extra logic for NaNs and infinities/overflows. I also rewrote it as static inline functions instead of an unreadable and uncommented macro. The float penalty functions were previously unreached by the regression tests, so add new test cases to exercise them. There's no on-disk format change, but users who have NaN entries in a btree_gist index would be well advised to reindex it. Bug: #19501 Bug: #19524 Reported-by: Man Zeng <zengman@halodbtech.com> Reported-by: Yuelin Wang <3020001251@tju.edu.cn> Author: Bill Kim <billkimjh@gmail.com> Co-authored-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/19501-3bff3bbc97f1e7c9@postgresql.org Discussion: https://postgr.es/m/19524-9559d302c8455664@postgresql.org Discussion: https://postgr.es/m/CAMQXxcgbtD2LXfX0tpgvOizxP-XxrCHV2ZDy4By_TZnJMsxXWQ@mail.gmail.com Backpatch-through: 14 Branch ------ REL_16_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/d569ccd408587b488cb33b8fdfb76341225fbbc2 Modified Files -------------- contrib/btree_gist/btree_float4.c | 59 +++++++++----- contrib/btree_gist/btree_float8.c | 51 +++++++++---- contrib/btree_gist/btree_utils_num.h | 131 ++++++++++++++++++++++++++++---- contrib/btree_gist/data/float4.data | 3 + contrib/btree_gist/data/float8.data | 3 + contrib/btree_gist/expected/float4.out | 51 +++++++++++-- contrib/btree_gist/expected/float8.out | 51 +++++++++++-- contrib/btree_gist/expected/numeric.out | 48 ++++++------ contrib/btree_gist/sql/float4.sql | 17 +++++ contrib/btree_gist/sql/float8.sql | 17 +++++ 10 files changed, 345 insertions(+), 86 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: btree_gist: fix NaN handling in float4/float8 opclasses. @ 2026-07-01 17:27 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 7+ messages in thread From: Tom Lane @ 2026-07-01 17:27 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org btree_gist: fix NaN handling in float4/float8 opclasses. The float4 and float8 btree_gist opclasses compared keys with raw C operators (==, <, >). IEEE 754 makes every comparison involving NaN false, so GiST disagreed with the regular float comparison operators and with the btree opclass, which uses float[4|8]_cmp_internal() (so that all NaNs are equal and NaN sorts after every non-NaN value). In addition, the penalty and distance functions were not careful about NaNs, and the penalty functions could also misbehave for IEEE infinities. Wrong answers from the penalty functions would probably do no more than make the index non-optimal, but the distance mistakes were visible from SQL. To fix, make the comparison functions rely on the same NaN-aware comparison functions the core code uses, and rewrite the penalty and distance functions to follow the rules that NaNs are equal but maximally far away from non-NaNs. The penalty_num() code was formerly shared between integral and float cases, but I chose to make two copies so that the integral cases are not saddled with the extra logic for NaNs and infinities/overflows. I also rewrote it as static inline functions instead of an unreadable and uncommented macro. The float penalty functions were previously unreached by the regression tests, so add new test cases to exercise them. There's no on-disk format change, but users who have NaN entries in a btree_gist index would be well advised to reindex it. Bug: #19501 Bug: #19524 Reported-by: Man Zeng <zengman@halodbtech.com> Reported-by: Yuelin Wang <3020001251@tju.edu.cn> Author: Bill Kim <billkimjh@gmail.com> Co-authored-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/19501-3bff3bbc97f1e7c9@postgresql.org Discussion: https://postgr.es/m/19524-9559d302c8455664@postgresql.org Discussion: https://postgr.es/m/CAMQXxcgbtD2LXfX0tpgvOizxP-XxrCHV2ZDy4By_TZnJMsxXWQ@mail.gmail.com Backpatch-through: 14 Branch ------ REL_15_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/98dd4406f7a44da9af56473fa22a03f885ea5e78 Modified Files -------------- contrib/btree_gist/btree_float4.c | 59 +++++++++----- contrib/btree_gist/btree_float8.c | 51 +++++++++---- contrib/btree_gist/btree_utils_num.h | 131 ++++++++++++++++++++++++++++---- contrib/btree_gist/data/float4.data | 3 + contrib/btree_gist/data/float8.data | 3 + contrib/btree_gist/expected/float4.out | 51 +++++++++++-- contrib/btree_gist/expected/float8.out | 51 +++++++++++-- contrib/btree_gist/expected/numeric.out | 48 ++++++------ contrib/btree_gist/sql/float4.sql | 17 +++++ contrib/btree_gist/sql/float8.sql | 17 +++++ 10 files changed, 345 insertions(+), 86 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: btree_gist: fix NaN handling in float4/float8 opclasses. @ 2026-07-01 17:27 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 7+ messages in thread From: Tom Lane @ 2026-07-01 17:27 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org btree_gist: fix NaN handling in float4/float8 opclasses. The float4 and float8 btree_gist opclasses compared keys with raw C operators (==, <, >). IEEE 754 makes every comparison involving NaN false, so GiST disagreed with the regular float comparison operators and with the btree opclass, which uses float[4|8]_cmp_internal() (so that all NaNs are equal and NaN sorts after every non-NaN value). In addition, the penalty and distance functions were not careful about NaNs, and the penalty functions could also misbehave for IEEE infinities. Wrong answers from the penalty functions would probably do no more than make the index non-optimal, but the distance mistakes were visible from SQL. To fix, make the comparison functions rely on the same NaN-aware comparison functions the core code uses, and rewrite the penalty and distance functions to follow the rules that NaNs are equal but maximally far away from non-NaNs. The penalty_num() code was formerly shared between integral and float cases, but I chose to make two copies so that the integral cases are not saddled with the extra logic for NaNs and infinities/overflows. I also rewrote it as static inline functions instead of an unreadable and uncommented macro. The float penalty functions were previously unreached by the regression tests, so add new test cases to exercise them. There's no on-disk format change, but users who have NaN entries in a btree_gist index would be well advised to reindex it. Bug: #19501 Bug: #19524 Reported-by: Man Zeng <zengman@halodbtech.com> Reported-by: Yuelin Wang <3020001251@tju.edu.cn> Author: Bill Kim <billkimjh@gmail.com> Co-authored-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/19501-3bff3bbc97f1e7c9@postgresql.org Discussion: https://postgr.es/m/19524-9559d302c8455664@postgresql.org Discussion: https://postgr.es/m/CAMQXxcgbtD2LXfX0tpgvOizxP-XxrCHV2ZDy4By_TZnJMsxXWQ@mail.gmail.com Backpatch-through: 14 Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/255bce44884ec5e12414760e5865a95dedb39925 Modified Files -------------- contrib/btree_gist/btree_float4.c | 59 +++++++++----- contrib/btree_gist/btree_float8.c | 52 ++++++++----- contrib/btree_gist/btree_utils_num.h | 131 ++++++++++++++++++++++++++++---- contrib/btree_gist/data/float4.data | 3 + contrib/btree_gist/data/float8.data | 3 + contrib/btree_gist/expected/float4.out | 51 +++++++++++-- contrib/btree_gist/expected/float8.out | 51 +++++++++++-- contrib/btree_gist/expected/numeric.out | 48 ++++++------ contrib/btree_gist/sql/float4.sql | 17 +++++ contrib/btree_gist/sql/float8.sql | 17 +++++ 10 files changed, 345 insertions(+), 87 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2026-07-01 17:27 UTC | newest] Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2026-07-01 17:27 pgsql: btree_gist: fix NaN handling in float4/float8 opclasses. Tom Lane <tgl@sss.pgh.pa.us> 2026-07-01 17:27 pgsql: btree_gist: fix NaN handling in float4/float8 opclasses. Tom Lane <tgl@sss.pgh.pa.us> 2026-07-01 17:27 pgsql: btree_gist: fix NaN handling in float4/float8 opclasses. Tom Lane <tgl@sss.pgh.pa.us> 2026-07-01 17:27 pgsql: btree_gist: fix NaN handling in float4/float8 opclasses. Tom Lane <tgl@sss.pgh.pa.us> 2026-07-01 17:27 pgsql: btree_gist: fix NaN handling in float4/float8 opclasses. Tom Lane <tgl@sss.pgh.pa.us> 2026-07-01 17:27 pgsql: btree_gist: fix NaN handling in float4/float8 opclasses. Tom Lane <tgl@sss.pgh.pa.us> 2026-07-01 17:27 pgsql: btree_gist: fix NaN handling in float4/float8 opclasses. Tom Lane <tgl@sss.pgh.pa.us>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox