public inbox for [email protected]  

help / color / mirror / Atom feed

From: PG Bug reporting form <[email protected]>
To: [email protected]
Cc: [email protected]
Subject: BUG #19524: In `contrib/btree_gist` float4/float8 GiST index operations, handling NaN values with raw C operator
Date: Thu, 18 Jun 2026 07:52:50 +0000
Message-ID: <[email protected]> (raw)

The following bug has been logged on the website:

Bug reference:      19524
Logged by:          Yuelin Wang
Email address:      [email protected]
PostgreSQL version: 19beta1
Operating system:   Linux (Ubuntu 24.04, x86_64)
Description:        

**Component**: `contrib/btree_gist/btree_float4.c`, `btree_float8.c`,
`btree_utils_num.c`

```sql
CREATE EXTENSION btree_gist;

-- Effect 1: EXCLUDE constraint bypass (float4)
CREATE TABLE reservations (
    room   float4,
    during tsrange,
    EXCLUDE USING gist (room WITH =, during WITH &&)
);
INSERT INTO reservations VALUES ('NaN'::float4, '[2025-01-01,2025-01-02)');
INSERT INTO reservations VALUES ('NaN'::float4, '[2025-01-01,2025-01-02)');
SELECT COUNT(*) FROM reservations;

-- Effect 2: RLS bypass (float8)
CREATE TABLE measurements (id int, val float8);
CREATE INDEX ON measurements USING gist (val);
INSERT INTO measurements VALUES (1, 'NaN'), (2, 1.5);
ALTER TABLE measurements ENABLE ROW LEVEL SECURITY;
ALTER TABLE measurements FORCE ROW LEVEL SECURITY;
CREATE POLICY hide_nan ON measurements FOR SELECT USING (val !=
'NaN'::float8);
CREATE ROLE lowpriv LOGIN;
GRANT SELECT ON measurements TO lowpriv;
SET ROLE lowpriv;
SET enable_seqscan = off;
SET enable_bitmapscan = off;
SELECT * FROM measurements ORDER BY id;
RESET ROLE;

-- Effect 3: index corruption after page split (float8)
CREATE TABLE t (val float8);
CREATE INDEX ON t USING gist (val);
INSERT INTO t SELECT 'NaN'::float8 FROM generate_series(1, 2000);
SET enable_indexscan = off; SET enable_bitmapscan = off;
SELECT COUNT(*) AS seqscan_count FROM t WHERE val = 'NaN';
RESET ALL;
SET enable_seqscan = off; SET enable_bitmapscan = off;
SELECT COUNT(*) AS indexscan_count FROM t WHERE val = 'NaN';
RESET ALL;
```

Expected vs actual output:

| Query | Expected | Actual |
|---|---|---|
| `SELECT COUNT(*) FROM reservations` | `1` (second insert blocked by
EXCLUDE) | `2` |
| `SELECT * FROM measurements ORDER BY id` (lowpriv, index scan) | `(2,
1.5)` only | `(1, NaN)` and `(2, 1.5)` |
| `seqscan_count` (Effect 3, seq scan forced) | `2000` | `2000` |
| `indexscan_count` (Effect 3, index scan forced) | `2000` | `0` |

The comparison functions in `btree_float4.c` and `btree_float8.c` use raw C
operators (`==`, `<`, `>`) on `float4`/`float8` values. Under IEEE 754, all
comparisons involving NaN return false, including `NaN == NaN`, which
diverges from PostgreSQL's semantic ordering where NaN equals itself and
sorts above all finite values. This causes two independent failures:
`gbt_float8eq(NaN, NaN)` returns false, so `BtreeGistNotEqualStrategyNumber`
in `btree_utils_num.c:300` incorrectly concludes that NaN satisfies `!=
NaN`, and since `gbt_float8_consistent()` unconditionally sets `*recheck =
false`, the heap-level filter is never applied. Separately,
`gbt_float8key_cmp(NaN, NaN)` returns -1 instead of 0, violating strict weak
ordering and producing a corrupted GiST tree during `picksplit` that cannot
locate NaN entries.

The fix is to replace the raw C operators in `gbt_float4gt/ge/eq/le/lt` and
`gbt_float8gt/ge/eq/le/lt` with PostgreSQL's `float4_cmp_internal()` and
`float8_cmp_internal()`, which handle NaN correctly. The key comparators
`gbt_float4key_cmp()` and `gbt_float8key_cmp()` need the same update. As a
defensive measure, `*recheck` in the consistent functions should be set to
`true` when the query or key involves NaN.

view thread (6+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected]
  Subject: Re: BUG #19524: In `contrib/btree_gist` float4/float8 GiST index operations, handling NaN values with raw C operator
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox