agora inbox for pgsql-bugs@postgresql.org
help / color / mirror / Atom feedBUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows
11+ messages / 5 participants
[nested] [flat]
* BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows
@ 2026-09-19 17:37 PG Bug reporting form <noreply@postgresql.org>
2026-09-21 23:49 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
0 siblings, 1 reply; 11+ messages in thread
From: PG Bug reporting form @ 2026-09-19 17:37 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org; +Cc: kehan5800@gmail.com
The following bug has been logged on the website:
Bug reference: 19705
Logged by: Ke
Email address: kehan5800@gmail.com
PostgreSQL version: 18.6
Operating system: Ubuntu 22.04.2 x86_64
Description:
Summary
-------
A single box value with a NaN coordinate, stored anywhere in a table, makes
a
BRIN box_inclusion_ops index stop returning rows that have nothing to do
with
it. The rows that disappear contain no NaN, and the queries that lose them
mention no NaN. There is no error and nothing unusual in the plan; the count
is
simply smaller than the heap says it should be.
The loss is one BRIN page range per NaN row. With the default
pages_per_range = 128 that is up to 128 pages of ordinary rows made
invisible
by one unrelated value.
Minimal case (fresh database, initdb defaults, nothing set but
enable_seqscan
to force the two plans to be compared):
CREATE TABLE b (v box);
INSERT INTO b SELECT box '(0,0),(1,1)' FROM generate_series(1, 1000);
INSERT INTO b VALUES (box '(NaN,NaN),(0,0)'); -- one row
CREATE INDEX bi ON b USING brin (v);
SELECT count(*) FROM b WHERE v && box '(-2,-2),(2,2)'; -- 1000
SET enable_seqscan = off;
SELECT count(*) FROM b WHERE v && box '(-2,-2),(2,2)'; -- 0
EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) of the
second one:
Aggregate (actual rows=1.00 loops=1)
-> Bitmap Heap Scan on b (actual rows=0.00 loops=1)
Recheck Cond: (v && '(2,2),(-2,-2)'::box)
-> Bitmap Index Scan on bi (actual rows=0.00 loops=1)
Index Cond: (v && '(2,2),(-2,-2)'::box)
Index Searches: 1
Every strategy in the opclass behaves the same way on that table (seq scan
first, BRIN second):
v && box '(-2,-2),(2,2)' 1000 / 0
v @> point '(0,0)' 1000 / 0
v @> box '(0,0),(0,0)' 1000 / 0
v ~= box '(0,0),(1,1)' 1000 / 0
Deleting the NaN row and reindexing restores the correct answers.
What I expected
---------------
The same answer with and without the index. An index may hand back extra
rows
for the recheck above it to drop; it may not hand back fewer rows than the
predicate selects.
Why it happens
--------------
BRIN's inclusion framework summarises a page range with the union of the
values
on it and, at scan time, asks the strategy operator whether that summary
could
match the query, skipping the range if it could not. That is sound only
while
the summary is a true superset of the range's contents.
For box the two halves use different NaN conventions:
* The union, amproc 11 = boxes_bound_box() in
src/backend/utils/adt/geo_ops.c,
is computed with float8_max()/float8_min() from
src/include/utils/float.h.
Those follow the float8 convention, in which NaN is larger than
everything.
So a NaN corner propagates into the union and becomes its high corner.
* The consistent side, brin_inclusion_consistent() calling box_overlap(),
box_contain() and friends, is computed with the FPlt/FPle/FPgt/FPge
macros
in src/include/utils/geo_decls.h. Those are plain C comparisons with an
epsilon, under which every comparison with a NaN is false.
Put together, the union of one ordinary box and one NaN box is a box that
overlaps nothing -- not even the boxes it was built from:
SELECT bound_box(box '(0,0),(1,1)', box '(NaN,NaN),(0,0)') AS
u,
bound_box(box '(0,0),(1,1)', box '(NaN,NaN),(0,0)') @> box
'(0,0),(1,1)',
bound_box(box '(0,0),(1,1)', box '(NaN,NaN),(0,0)') && box
'(0,0),(1,1)';
u | ?column? | ?column?
----------------+----------+----------
(NaN,NaN),(0,0) | f | f
and that is exactly what gets stored as the range summary (pageinspect on
the
minimal case above):
SELECT blknum, value FROM brin_page_items(get_raw_page('bi', 2),
'bi'::regclass);
blknum | value
-------+-----------------------------
0 | {(NaN,NaN),(0,0) .. f .. f}
brin_inclusion_consistent() then correctly concludes that this summary
cannot
match the query and skips the range. The consistency function is doing its
job;
the summary is the lie.
The framework already has the escape hatch for values that cannot be merged
into a bounding value -- the INCLUSION_UNMERGEABLE flag, set when the
optional
support function PROCNUM_MERGEABLE (amproc 12) says two values cannot be
merged. A range marked unmergeable is always scanned. box_inclusion_ops does
not have that function:
opfname | amprocnum | amproc
----------------------+-----------+---------------------------
box_inclusion_ops | 11 | bound_box
box_inclusion_ops | 13 | box_contain
network_inclusion_ops | 11 | inet_merge
network_inclusion_ops | 12 | inet_same_family <-- the hatch
network_inclusion_ops | 13 | network_supeq
inet gets this right for the analogous case (an IPv6 address among IPv4
addresses, where no meaningful union exists) and box does not, which is why
inet is unaffected and box is not.
What else I measured
--------------------
* Not only the build path. Insert the NaN row after CREATE INDEX and let
brin_summarize_new_values() fold it into the summary: 4896 of 5000 rows
(pages_per_range = 1, so one page lost).
* Parallel index build: 382976 of 400000 rows.
* Order does not matter: NaN row first or last, same result.
* NaN specifically, not non-finite generally. The same table with
box '(Infinity,Infinity),(0,0)' instead returns 5001 of 5001. float8_max
/
float8_min and the FP* macros agree about infinities; they disagree only
about NaN.
* One NaN coordinate out of the four is enough.
* Realistic shape, 1,000,000 rows of box(point(g,g), point(g+1,g+1)) plus
one
NaN row, default pages_per_range = 128: 1 of 58 ranges is poisoned, and
a
query whose rows live in that range returns 0 where the heap has 102. At
this scale the wrong answer is plausible rather than obviously empty,
which
is what makes it worth reporting.
* Other BRIN opclasses on the same shape of data are fine: float8 minmax
with
a NaN among 3000 finite values returns 3000 = 3000 (it compares with
float8_lt/float8_gt, which know about NaN), and inet inclusion with a
::1
among 3000 IPv4 addresses returns 3000 = 3000 (amproc 12).
* The NaN row itself is also lost, not just its neighbours: v ~= box
'(NaN,NaN),(0,0)' finds it on a seq scan -- box_same routes through
float8_eq, where NaN = NaN is true -- and finds nothing through BRIN.
* The same disagreement is visible on GiST, at smaller blast radius: the
same 5001-row table loses roughly a leaf page's worth through a GiST
index
-- I have measured 4835, 4839, 4876 and 4897 of 5000 across four builds,
the exact figure depending on page packing -- and 5000 of 5000 through
SP-GiST. gistproc.c's rt_box_union() was already converted to
float8_max/float8_min by 1acf7572554, so GiST's union is NaN-propagating
in the same way, and it loses rows for the same reason: the search side
still uses the epsilon macros. I am reporting the BRIN case because
there
the unit of loss is a whole page range and because BRIN has the
ready-made
fix below, but the two are one defect.
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows
2026-09-19 17:37 BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows PG Bug reporting form <noreply@postgresql.org>
@ 2026-09-21 23:49 ` shihao zhong <zhong950419@gmail.com>
2026-09-22 12:12 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Kirill Reshke <reshkekirill@gmail.com>
0 siblings, 1 reply; 11+ messages in thread
From: shihao zhong @ 2026-09-21 23:49 UTC (permalink / raw)
To: kehan5800@gmail.com; pgsql-bugs@lists.postgresql.org
Hi,
I can reproduce this on master, and your analysis is right. bound_box()
lets the NaN into the range summary, and then every box operator says
the range cannot match.
You suggested adding a mergeable support function, like inet has. That
works, but it needs a new pg_amproc row, so it cannot go to the back
branches. The same code is in 14 and later.
The attached 0001 fixes bound_box() instead. When an input coordinate is
NaN, the result is infinite on that side. The summary then matches any
query on that axis and the recheck does the rest. Only the NaN axis
becomes lossy, the other axis still prunes.
Existing summaries that hold a NaN stay broken until REINDEX.
0002 adds tests and is optional.
Thanks,
Shihao
Attachments:
[application/octet-stream] v1-0002-Add-tests-for-NaN-handling-in-bound_box-and-BRIN-.patch (4.5K, ../../CAGRkXqS=KcB0EKNpC31PnEEuP75mXOxYvYE3M80q78pZK=3TzA@mail.gmail.com/3-v1-0002-Add-tests-for-NaN-handling-in-bound_box-and-BRIN-.patch)
download | inline diff:
From 7749aa23cd5573a35d7edb9fbc883e6c2a967c13 Mon Sep 17 00:00:00 2001
From: Shihao <zhong950419@gmail.com>
Date: Mon, 21 Sep 2026 19:06:37 -0400
Subject: [PATCH v1 2/2] Add tests for NaN handling in bound_box() and BRIN
box_inclusion_ops
Check that a box with a NaN coordinate does not hide the other rows of
its page range from a BRIN index scan, and that bound_box() returns an
infinite bound for a NaN coordinate.
Discussion: https://postgr.es/m/19705-548fda77321e062d@postgresql.org
---
src/test/regress/expected/brin.out | 37 ++++++++++++++++++++++++++
src/test/regress/expected/geometry.out | 7 +++++
src/test/regress/sql/brin.sql | 14 ++++++++++
src/test/regress/sql/geometry.sql | 3 +++
4 files changed, 61 insertions(+)
diff --git a/src/test/regress/expected/brin.out b/src/test/regress/expected/brin.out
index e1db2280cf9..5ebf9e74faf 100644
--- a/src/test/regress/expected/brin.out
+++ b/src/test/regress/expected/brin.out
@@ -589,3 +589,40 @@ CREATE INDEX brin_insert_optimization_idx ON brin_insert_optimization USING brin
UPDATE brin_insert_optimization SET a = a;
REINDEX INDEX CONCURRENTLY brin_insert_optimization_idx;
DROP TABLE brin_insert_optimization;
+-- A box with a NaN coordinate must not hide the other rows of its page range
+CREATE TABLE brin_box_nan (b box);
+INSERT INTO brin_box_nan SELECT box '(0,0),(1,1)' FROM generate_series(1, 100);
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+CREATE INDEX ON brin_box_nan USING brin (b);
+SET enable_seqscan = off;
+EXPLAIN (COSTS OFF)
+SELECT count(*) FROM brin_box_nan WHERE b && box '(-2,-2),(2,2)';
+ QUERY PLAN
+-------------------------------------------------------
+ Aggregate
+ -> Bitmap Heap Scan on brin_box_nan
+ Recheck Cond: (b && '(2,2),(-2,-2)'::box)
+ -> Bitmap Index Scan on brin_box_nan_b_idx
+ Index Cond: (b && '(2,2),(-2,-2)'::box)
+(5 rows)
+
+SELECT count(*) FROM brin_box_nan WHERE b && box '(-2,-2),(2,2)';
+ count
+-------
+ 100
+(1 row)
+
+SELECT count(*) FROM brin_box_nan WHERE b @> point '(0.5,0.5)';
+ count
+-------
+ 100
+(1 row)
+
+SELECT count(*) FROM brin_box_nan WHERE b ~= box '(0,0),(1,1)';
+ count
+-------
+ 100
+(1 row)
+
+RESET enable_seqscan;
+DROP TABLE brin_box_nan;
diff --git a/src/test/regress/expected/geometry.out b/src/test/regress/expected/geometry.out
index 1d168b21cbc..fa9c746fedd 100644
--- a/src/test/regress/expected/geometry.out
+++ b/src/test/regress/expected/geometry.out
@@ -5321,3 +5321,10 @@ SELECT * FROM pg_input_error_info('(1,2),-1', 'circle');
invalid input syntax for type circle: "(1,2),-1" | | | 22P02
(1 row)
+-- A NaN coordinate makes the bound infinite on that side
+SELECT bound_box(box '(0,0),(1,1)', box '(NaN,3),(0,2)');
+ bound_box
+--------------------
+ (Infinity,3),(0,0)
+(1 row)
+
diff --git a/src/test/regress/sql/brin.sql b/src/test/regress/sql/brin.sql
index 7ea97f47c8d..11c93177cda 100644
--- a/src/test/regress/sql/brin.sql
+++ b/src/test/regress/sql/brin.sql
@@ -534,3 +534,17 @@ CREATE INDEX brin_insert_optimization_idx ON brin_insert_optimization USING brin
UPDATE brin_insert_optimization SET a = a;
REINDEX INDEX CONCURRENTLY brin_insert_optimization_idx;
DROP TABLE brin_insert_optimization;
+
+-- A box with a NaN coordinate must not hide the other rows of its page range
+CREATE TABLE brin_box_nan (b box);
+INSERT INTO brin_box_nan SELECT box '(0,0),(1,1)' FROM generate_series(1, 100);
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+CREATE INDEX ON brin_box_nan USING brin (b);
+SET enable_seqscan = off;
+EXPLAIN (COSTS OFF)
+SELECT count(*) FROM brin_box_nan WHERE b && box '(-2,-2),(2,2)';
+SELECT count(*) FROM brin_box_nan WHERE b && box '(-2,-2),(2,2)';
+SELECT count(*) FROM brin_box_nan WHERE b @> point '(0.5,0.5)';
+SELECT count(*) FROM brin_box_nan WHERE b ~= box '(0,0),(1,1)';
+RESET enable_seqscan;
+DROP TABLE brin_box_nan;
diff --git a/src/test/regress/sql/geometry.sql b/src/test/regress/sql/geometry.sql
index c3ea368da5e..5d8a1749c35 100644
--- a/src/test/regress/sql/geometry.sql
+++ b/src/test/regress/sql/geometry.sql
@@ -529,3 +529,6 @@ SELECT pg_input_is_valid('(1', 'circle');
SELECT * FROM pg_input_error_info('1,', 'circle');
SELECT pg_input_is_valid('(1,2),-1', 'circle');
SELECT * FROM pg_input_error_info('(1,2),-1', 'circle');
+
+-- A NaN coordinate makes the bound infinite on that side
+SELECT bound_box(box '(0,0),(1,1)', box '(NaN,3),(0,2)');
--
2.37.1 (Apple Git-137.1)
[application/octet-stream] v1-0001-Fix-BRIN-box_inclusion_ops-losing-rows-when-a-box.patch (3.1K, ../../CAGRkXqS=KcB0EKNpC31PnEEuP75mXOxYvYE3M80q78pZK=3TzA@mail.gmail.com/4-v1-0001-Fix-BRIN-box_inclusion_ops-losing-rows-when-a-box.patch)
download | inline diff:
From b3a6658da4d2a133636c0fbe94503e66b7541b9a Mon Sep 17 00:00:00 2001
From: Shihao <zhong950419@gmail.com>
Date: Mon, 21 Sep 2026 19:06:37 -0400
Subject: [PATCH v1 1/2] Fix BRIN box_inclusion_ops losing rows when a box has
a NaN coordinate
BRIN summarizes each page range of a box column with bound_box(). That
function used float8_max(), which treats NaN as larger than any other
value, so one box with a NaN coordinate put NaN into the summary. The
box operators compare with plain C comparisons, which are all false for
NaN. So brin_inclusion_consistent() decided that the range could not
match any query, and every row of the range was skipped, including the
rows that have no NaN at all.
Make bound_box() return an infinite bound on any side where an input
coordinate is NaN. The summary then matches every query on that axis,
the range gets scanned, and the recheck sorts out the rows.
Summaries that already hold a NaN are not repaired by this. Users with
NaN values in a box column under a BRIN index need to REINDEX it.
Bug: #19705
Reported-by: Ke <kehan5800@gmail.com>
Discussion: https://postgr.es/m/19705-548fda77321e062d@postgresql.org
Backpatch-through: 14
---
src/backend/utils/adt/geo_ops.c | 34 +++++++++++++++++++++++++++++----
1 file changed, 30 insertions(+), 4 deletions(-)
diff --git a/src/backend/utils/adt/geo_ops.c b/src/backend/utils/adt/geo_ops.c
index 73324b91fe5..09ae2bdaad2 100644
--- a/src/backend/utils/adt/geo_ops.c
+++ b/src/backend/utils/adt/geo_ops.c
@@ -4407,6 +4407,32 @@ point_box(PG_FUNCTION_ARGS)
PG_RETURN_BOX_P(box);
}
+/*
+ * Helpers for boxes_bound_box
+ *
+ * A NaN coordinate has no position, so no finite bound can be said to include
+ * it. We make the bound infinite on that side instead of letting the NaN
+ * through. This matters because the box operators use plain C comparisons,
+ * which are all false for NaN. BRIN's box_inclusion_ops uses bound_box to
+ * summarize a page range, and a NaN in the summary would make every operator
+ * say that the range cannot match, hiding all rows of the range.
+ */
+static inline float8
+bound_box_high(float8 val1, float8 val2)
+{
+ if (unlikely(isnan(val1) || isnan(val2)))
+ return get_float8_infinity();
+ return float8_max(val1, val2);
+}
+
+static inline float8
+bound_box_low(float8 val1, float8 val2)
+{
+ if (unlikely(isnan(val1) || isnan(val2)))
+ return -get_float8_infinity();
+ return float8_min(val1, val2);
+}
+
/*
* Smallest bounding box that includes both of the given boxes
*/
@@ -4419,10 +4445,10 @@ boxes_bound_box(PG_FUNCTION_ARGS)
container = palloc_object(BOX);
- container->high.x = float8_max(box1->high.x, box2->high.x);
- container->low.x = float8_min(box1->low.x, box2->low.x);
- container->high.y = float8_max(box1->high.y, box2->high.y);
- container->low.y = float8_min(box1->low.y, box2->low.y);
+ container->high.x = bound_box_high(box1->high.x, box2->high.x);
+ container->low.x = bound_box_low(box1->low.x, box2->low.x);
+ container->high.y = bound_box_high(box1->high.y, box2->high.y);
+ container->low.y = bound_box_low(box1->low.y, box2->low.y);
PG_RETURN_BOX_P(container);
}
--
2.37.1 (Apple Git-137.1)
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows
2026-09-19 17:37 BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows PG Bug reporting form <noreply@postgresql.org>
2026-09-21 23:49 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
@ 2026-09-22 12:12 ` Kirill Reshke <reshkekirill@gmail.com>
2026-09-22 12:37 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Andrey Borodin <x4mmm@yandex-team.ru>
0 siblings, 1 reply; 11+ messages in thread
From: Kirill Reshke @ 2026-09-22 12:12 UTC (permalink / raw)
To: shihao zhong <zhong950419@gmail.com>; +Cc: kehan5800@gmail.com; pgsql-bugs@lists.postgresql.org
Hi!
I think your fix is basically correct and comes in sync with previous
fix of this kind [0]
Your 0001 makes the BRIN symptom go away, including on the back branches.
A few observations and an alternative (your 0001 adjusted) I have been
working on (PFA)
On Tue, 22 Sept 2026 at 04:50, shihao zhong <zhong950419@gmail.com> wrote:
>
> Hi,
>
> I can reproduce this on master, and your analysis is right. bound_box()
> lets the NaN into the range summary, and then every box operator says
> the range cannot match.
>
> You suggested adding a mergeable support function, like inet has. That
> works, but it needs a new pg_amproc row, so it cannot go to the back
> branches. The same code is in 14 and later.
Yes, but for HEAD it's OK and will be an idiomatic way to fix. So, we
can make different patches for HEAD and back branches, where the
back-branched version will miss pg_amproc fix (and misbehave on index
scan, but looks like we can live with that). Example:
CREATE TABLE b (v box);
INSERT INTO b SELECT box '(0,0),(1,1)' FROM generate_series(1, 1000);
INSERT INTO b VALUES (box '(NaN,NaN),(0,0)');
CREATE INDEX bi ON b USING brin (v);
SET enable_seqscan = off;
SELECT count(*) FROM b WHERE v && box '(-2,-2),(2,2)'; -- seq: 1000
SELECT count(*) FROM b WHERE v @> point '(0.5,0.5)'; -- seq: 1000
SELECT count(*) FROM b WHERE v ~= box '(0,0),(1,1)'; -- seq: 1000
SELECT count(*) FROM b WHERE v ~= box '(NaN,NaN),(0,0)'; -- seq:
1 (your v1 will return 0)
> The attached 0001 fixes bound_box() instead. When an input coordinate is
> NaN, the result is infinite on that side. The summary then matches any
> query on that axis and the recheck does the rest. Only the NaN axis
> becomes lossy, the other axis still prunes.
>
> Existing summaries that hold a NaN stay broken until REINDEX.
>
> 0002 adds tests and is optional.
>
> Thanks,
> Shihao
>
>
As the reporter already measured, GiST has the same defect, so 0002
fixes that too.
The GiST issue is in fallbackSplit(), gist_box_picksplit etc build the
groups union keys by copying the first entry
as-is `*leftBox = *box` and then growing the copy, so a NaN box placed
first in its group produces a NaN key. To fix this, there is a small
NaN-aware copy helper function. Also, adjustBoxnow uses two NaN-aware
helper functions, so they replace NaN with inf values too. This is
what makes the NaN keys not propagate to the root on the insert path.
Note that this efficiently means we will insert different tuples in
index (without NaNs but with Inf). See gist_page_items output for
reference.
GiST index failing test case is the same:
db2=#
CREATE TABLE b (v box);
INSERT INTO b SELECT box '(0,0),(1,1)' FROM generate_series(1, 1000);
INSERT INTO b VALUES (box '(NaN,NaN),(0,0)'); -- one row
CREATE TABLE
INSERT 0 1000
INSERT 0 1
db2=# CREATE INDEX bi ON b USING gist (v);
CREATE INDEX
db2=# select count(*) from b where v && box(point(1,1), point(4,4));
count
-------
835
(1 row)
db2=# drop index bi;
DROP INDEX
db2=# select count(*) from b where v && box(point(1,1), point(4,4));
count
-------
1000
(1 row)
before/after 0002 with pageinspect.
```
db2=# SELECT * FROM gist_page_items(get_raw_page('gist_nan_tbl_index',
0), 'gist_nan_tbl_index');
itemoffset | ctid | itemlen | dead | keys
------------+------------+---------+------+-----------------------------
1 | (1,65535) | 40 | f | (b)=("(82,82),(0,0)")
2 | (2,65535) | 40 | f | (b)=("(165,165),(83,83)")
3 | (3,65535) | 40 | f | (b)=("(NaN,NaN),(0,0)")
4 | (4,65535) | 40 | f | (b)=("(331,331),(249,249)")
5 | (5,65535) | 40 | f | (b)=("(414,414),(332,332)")
6 | (6,65535) | 40 | f | (b)=("(497,497),(415,415)")
7 | (7,65535) | 40 | f | (b)=("(580,580),(498,498)")
8 | (8,65535) | 40 | f | (b)=("(663,663),(581,581)")
9 | (9,65535) | 40 | f | (b)=("(746,746),(664,664)")
10 | (10,65535) | 40 | f | (b)=("(829,829),(747,747)")
11 | (11,65535) | 40 | f | (b)=("(912,912),(830,830)")
12 | (12,65535) | 40 | f | (b)=("(999,999),(913,913)")
```
```
reshke=# SELECT * FROM
gist_page_items(get_raw_page('gist_nan_tbl_index', 0),
'gist_nan_tbl_index');
itemoffset | ctid | itemlen | dead | keys
------------+------------+---------+------+-----------------------------------
1 | (1,65535) | 40 | f | (b)=("(Infinity,Infinity),(0,0)")
2 | (2,65535) | 40 | f | (b)=("(165,165),(83,83)")
3 | (3,65535) | 40 | f | (b)=("(248,248),(166,166)")
4 | (4,65535) | 40 | f | (b)=("(331,331),(249,249)")
5 | (5,65535) | 40 | f | (b)=("(414,414),(332,332)")
6 | (6,65535) | 40 | f | (b)=("(497,497),(415,415)")
7 | (7,65535) | 40 | f | (b)=("(580,580),(498,498)")
8 | (8,65535) | 40 | f | (b)=("(663,663),(581,581)")
9 | (9,65535) | 40 | f | (b)=("(746,746),(664,664)")
10 | (10,65535) | 40 | f | (b)=("(829,829),(747,747)")
11 | (11,65535) | 40 | f | (b)=("(912,912),(830,830)")
12 | (12,65535) | 40 | f | (b)=("(999,999),(913,913)")
(12 rows)
```
GiST indexes obviously need to be REINDEX-ed after that.
[0] https://github.com/postgres/postgres/commit/1acf7572554
--
Best regards,
Kirill Reshke
Attachments:
[application/octet-stream] v2-0001-Fix-NaN-handling-in-BRIN-box_inclusion_ops-and-Gi.patch (14.4K, ../../CALdSSPgBZXc0Arj-VwFWsuGF19yTkzRtkq0JzR+feaYZukUTTw@mail.gmail.com/2-v2-0001-Fix-NaN-handling-in-BRIN-box_inclusion_ops-and-Gi.patch)
download | inline diff:
From 6562a30ab43d4124642dc8b256d32fe39d14c467 Mon Sep 17 00:00:00 2001
From: reshke <reshke@double.cloud>
Date: Mon, 21 Sep 2026 14:29:36 +0300
Subject: [PATCH v2] Fix NaN handling in BRIN box_inclusion_ops and GiST ops
A box with a NaN coordinate fooled BRIN summaries and GiST union
keys, masking completely unlrelated rows. Previous issue of this kind was
fixed in 1acf757, adopt the same fix here.
To fix, add helper functions to float.h and use them in box etc ops,
replacing NaN box bounds with infinite bounds.
Also add support function to box inclusion ops, to mark box with NaN
bounds unmergable.
In GiST, fix search keys to correctly support bounding box with NaN
values.
Reported-by: BUG #19705
Discussion: https://postgr.es/m/19705-548fda77321e062d@postgresql.org
---
src/backend/access/gist/gistproc.c | 40 +++++++++++++++++---------
src/backend/utils/adt/geo_ops.c | 29 ++++++++++++++++---
src/include/catalog/pg_amproc.dat | 2 ++
src/include/catalog/pg_proc.dat | 3 ++
src/include/utils/float.h | 22 ++++++++++++++
src/test/regress/expected/brin.out | 34 ++++++++++++++++++++++
src/test/regress/expected/geometry.out | 20 +++++++++++++
src/test/regress/expected/gist.out | 31 ++++++++++++++++++++
src/test/regress/sql/brin.sql | 15 ++++++++++
src/test/regress/sql/geometry.sql | 6 ++++
src/test/regress/sql/gist.sql | 17 +++++++++++
11 files changed, 201 insertions(+), 18 deletions(-)
diff --git a/src/backend/access/gist/gistproc.c b/src/backend/access/gist/gistproc.c
index f1044f49d6c..2538eb50b07 100644
--- a/src/backend/access/gist/gistproc.c
+++ b/src/backend/access/gist/gistproc.c
@@ -141,19 +141,31 @@ gist_box_consistent(PG_FUNCTION_ARGS)
}
/*
- * Increase BOX b to include addon.
+ * Increase BOX b to include addon. A NaN in either box grows b to the
+ * corresponding infinity.
*/
static void
adjustBox(BOX *b, const BOX *addon)
{
- if (float8_lt(b->high.x, addon->high.x))
- b->high.x = addon->high.x;
- if (float8_gt(b->low.x, addon->low.x))
- b->low.x = addon->low.x;
- if (float8_lt(b->high.y, addon->high.y))
- b->high.y = addon->high.y;
- if (float8_gt(b->low.y, addon->low.y))
- b->low.y = addon->low.y;
+ b->high.x = float8_bound_max(b->high.x, addon->high.x);
+ b->low.x = float8_bound_min(b->low.x, addon->low.x);
+ b->high.y = float8_bound_max(b->high.y, addon->high.y);
+ b->low.y = float8_bound_min(b->low.y, addon->low.y);
+}
+
+/* Copy a BOX, mapping NaNs to infinities. */
+static void
+snapBox(BOX *b, const BOX *box)
+{
+ *b = *box;
+ if (isnan(b->high.x))
+ b->high.x = get_float8_infinity();
+ if (isnan(b->high.y))
+ b->high.y = get_float8_infinity();
+ if (isnan(b->low.x))
+ b->low.x = -get_float8_infinity();
+ if (isnan(b->low.y))
+ b->low.y = -get_float8_infinity();
}
/*
@@ -239,7 +251,7 @@ fallbackSplit(GistEntryVector *entryvec, GIST_SPLITVEC *v)
if (unionL == NULL)
{
unionL = palloc_object(BOX);
- *unionL = *cur;
+ snapBox(unionL, cur);
}
else
adjustBox(unionL, cur);
@@ -252,7 +264,7 @@ fallbackSplit(GistEntryVector *entryvec, GIST_SPLITVEC *v)
if (unionR == NULL)
{
unionR = palloc_object(BOX);
- *unionR = *cur;
+ snapBox(unionR, cur);
}
else
adjustBox(unionR, cur);
@@ -526,7 +538,7 @@ gist_box_picksplit(PG_FUNCTION_ARGS)
{
box = DatumGetBoxP(entryvec->vector[i].key);
if (i == FirstOffsetNumber)
- context.boundingBox = *box;
+ snapBox(&context.boundingBox, box);
else
adjustBox(&context.boundingBox, box);
}
@@ -715,7 +727,7 @@ gist_box_picksplit(PG_FUNCTION_ARGS)
if (v->spl_nleft > 0) \
adjustBox(leftBox, box); \
else \
- *leftBox = *(box); \
+ snapBox(leftBox, box); \
v->spl_left[v->spl_nleft++] = off; \
} while(0)
@@ -724,7 +736,7 @@ gist_box_picksplit(PG_FUNCTION_ARGS)
if (v->spl_nright > 0) \
adjustBox(rightBox, box); \
else \
- *rightBox = *(box); \
+ snapBox(rightBox, box); \
v->spl_right[v->spl_nright++] = off; \
} while(0)
diff --git a/src/backend/utils/adt/geo_ops.c b/src/backend/utils/adt/geo_ops.c
index 73324b91fe5..6a2ad14cb3f 100644
--- a/src/backend/utils/adt/geo_ops.c
+++ b/src/backend/utils/adt/geo_ops.c
@@ -4419,14 +4419,35 @@ boxes_bound_box(PG_FUNCTION_ARGS)
container = palloc_object(BOX);
- container->high.x = float8_max(box1->high.x, box2->high.x);
- container->low.x = float8_min(box1->low.x, box2->low.x);
- container->high.y = float8_max(box1->high.y, box2->high.y);
- container->low.y = float8_min(box1->low.y, box2->low.y);
+ /* NaNs become infinite bounds, never NaNs. */
+ container->high.x = float8_bound_max(box1->high.x, box2->high.x);
+ container->low.x = float8_bound_min(box1->low.x, box2->low.x);
+ container->high.y = float8_bound_max(box1->high.y, box2->high.y);
+ container->low.y = float8_bound_min(box1->low.y, box2->low.y);
PG_RETURN_BOX_P(container);
}
+/*
+ * Boxes with NaN coordinates cannot be merged, since no operator can match a NaN summary.
+ * Used in BRIN box_inclusion_ops PROCNUM_MERGEABLE support.
+ */
+extern Datum box_mergeable(PG_FUNCTION_ARGS);
+Datum
+box_mergeable(PG_FUNCTION_ARGS)
+{
+ BOX *box1 = PG_GETARG_BOX_P(0),
+ *box2 = PG_GETARG_BOX_P(1);
+
+ if (isnan(box1->high.x) || isnan(box1->high.y) ||
+ isnan(box1->low.x) || isnan(box1->low.y) ||
+ isnan(box2->high.x) || isnan(box2->high.y) ||
+ isnan(box2->low.x) || isnan(box2->low.y))
+ PG_RETURN_BOOL(false);
+
+ PG_RETURN_BOOL(true);
+}
+
/***********************************************************************
**
diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat
index 4a1efdbc899..db24e42e795 100644
--- a/src/include/catalog/pg_amproc.dat
+++ b/src/include/catalog/pg_amproc.dat
@@ -2033,6 +2033,8 @@
amproc => 'brin_inclusion_union' },
{ amprocfamily => 'brin/box_inclusion_ops', amproclefttype => 'box',
amprocrighttype => 'box', amprocnum => '11', amproc => 'bound_box' },
+{ amprocfamily => 'brin/box_inclusion_ops', amproclefttype => 'box',
+ amprocrighttype => 'box', amprocnum => '12', amproc => 'box_mergeable' },
{ amprocfamily => 'brin/box_inclusion_ops', amproclefttype => 'box',
amprocrighttype => 'box', amprocnum => '13', amproc => 'box_contain' },
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f46427258e3..bbbce58a962 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -2140,6 +2140,9 @@
{ oid => '4067', descr => 'bounding box of two boxes',
proname => 'bound_box', prorettype => 'box', proargtypes => 'box box',
prosrc => 'boxes_bound_box' },
+{ oid => '8400', descr => 'can two boxes be merged into a single summary',
+ proname => 'box_mergeable', prorettype => 'bool', proargtypes => 'box box',
+ prosrc => 'box_mergeable' },
{ oid => '981', descr => 'box diagonal',
proname => 'diagonal', prorettype => 'lseg', proargtypes => 'box',
prosrc => 'box_diagonal' },
diff --git a/src/include/utils/float.h b/src/include/utils/float.h
index ffa743d6273..843c4ecdef5 100644
--- a/src/include/utils/float.h
+++ b/src/include/utils/float.h
@@ -336,4 +336,26 @@ float8_max(const float8 val1, const float8 val2)
return float8_gt(val1, val2) ? val1 : val2;
}
+/*
+ * float8_bound_max/min: like float8_max/float8_min, but a NaN is mapped
+ * to the corresponding infinity. Used for geometric bounds, where a NaN
+ * would make every comparison with the bound false and thus hide the
+ * values it was meant to cover.
+ */
+static inline float8
+float8_bound_max(const float8 val1, const float8 val2)
+{
+ if (isnan(val1) || isnan(val2))
+ return get_float8_infinity();
+ return float8_max(val1, val2);
+}
+
+static inline float8
+float8_bound_min(const float8 val1, const float8 val2)
+{
+ if (isnan(val1) || isnan(val2))
+ return -get_float8_infinity();
+ return float8_min(val1, val2);
+}
+
#endif /* FLOAT_H */
diff --git a/src/test/regress/expected/brin.out b/src/test/regress/expected/brin.out
index e1db2280cf9..a73f7b7e82d 100644
--- a/src/test/regress/expected/brin.out
+++ b/src/test/regress/expected/brin.out
@@ -589,3 +589,37 @@ CREATE INDEX brin_insert_optimization_idx ON brin_insert_optimization USING brin
UPDATE brin_insert_optimization SET a = a;
REINDEX INDEX CONCURRENTLY brin_insert_optimization_idx;
DROP TABLE brin_insert_optimization;
+-- a box with a NaN coordinate must not hide the other rows of its page
+-- range (bug #19705)
+CREATE TABLE brin_box_nan (v box);
+INSERT INTO brin_box_nan SELECT box '(0,0),(1,1)' FROM generate_series(1, 100);
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+CREATE INDEX brin_box_nan_idx ON brin_box_nan USING brin (v);
+SET enable_seqscan = off;
+SELECT count(*) FROM brin_box_nan WHERE v && box '(-2,-2),(2,2)';
+ count
+-------
+ 100
+(1 row)
+
+SELECT count(*) FROM brin_box_nan WHERE v @> point '(0.5,0.5)';
+ count
+-------
+ 100
+(1 row)
+
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(0,0),(1,1)';
+ count
+-------
+ 100
+(1 row)
+
+-- the NaN row is found too: unmergeable ranges are always scanned
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(NaN,NaN),(0,0)';
+ count
+-------
+ 1
+(1 row)
+
+RESET enable_seqscan;
+DROP TABLE brin_box_nan;
diff --git a/src/test/regress/expected/geometry.out b/src/test/regress/expected/geometry.out
index 1d168b21cbc..1372d34a668 100644
--- a/src/test/regress/expected/geometry.out
+++ b/src/test/regress/expected/geometry.out
@@ -5321,3 +5321,23 @@ SELECT * FROM pg_input_error_info('(1,2),-1', 'circle');
invalid input syntax for type circle: "(1,2),-1" | | | 22P02
(1 row)
+-- NaN becomes an infinite bound (bug #19705)
+SELECT bound_box(box '(0,0),(1,1)', box '(NaN,3),(0,2)');
+ bound_box
+--------------------
+ (Infinity,3),(0,0)
+(1 row)
+
+-- box_mergeable() rejects NaN coordinates (BRIN box_inclusion_ops)
+SELECT box_mergeable(box '(0,0),(1,1)', box '(NaN,3),(0,2)');
+ box_mergeable
+---------------
+ f
+(1 row)
+
+SELECT box_mergeable(box '(0,0),(1,1)', box '(2,3),(0,2)');
+ box_mergeable
+---------------
+ t
+(1 row)
+
diff --git a/src/test/regress/expected/gist.out b/src/test/regress/expected/gist.out
index ac79f94aa80..82197b4dcef 100644
--- a/src/test/regress/expected/gist.out
+++ b/src/test/regress/expected/gist.out
@@ -463,3 +463,34 @@ create index gist_tbl_box_index on gist_tbl using gist (b);
insert into gist_tbl
select box(point(0.05*i, 0.05*i)) from generate_series(0,10) as i;
drop table gist_tbl;
+-- a box with a NaN coordinate must not poison the union keys (bug #19705)
+create table gist_nan_tbl (b box);
+insert into gist_nan_tbl select box '(1,1),(2,2)' from generate_series(1, 1000);
+insert into gist_nan_tbl values (box '(NaN,NaN),(0,0)');
+create index gist_nan_tbl_index on gist_nan_tbl using gist (b);
+-- also insert rows after the build, to exercise the insertion path
+insert into gist_nan_tbl select box '(3,3),(4,4)' from generate_series(1, 1000);
+insert into gist_nan_tbl values (box '(NaN,NaN),(0,0)');
+set enable_seqscan = off;
+set enable_bitmapscan = off;
+select count(*) from gist_nan_tbl where b && box(point(0,0), point(5,5));
+ count
+-------
+ 2000
+(1 row)
+
+select count(*) from gist_nan_tbl where b <@ box(point(0,0), point(5,5));
+ count
+-------
+ 2000
+(1 row)
+
+select count(*) from gist_nan_tbl where b @> point '(1.5,1.5)';
+ count
+-------
+ 1000
+(1 row)
+
+reset enable_seqscan;
+reset enable_bitmapscan;
+drop table gist_nan_tbl;
diff --git a/src/test/regress/sql/brin.sql b/src/test/regress/sql/brin.sql
index 7ea97f47c8d..3484869d676 100644
--- a/src/test/regress/sql/brin.sql
+++ b/src/test/regress/sql/brin.sql
@@ -534,3 +534,18 @@ CREATE INDEX brin_insert_optimization_idx ON brin_insert_optimization USING brin
UPDATE brin_insert_optimization SET a = a;
REINDEX INDEX CONCURRENTLY brin_insert_optimization_idx;
DROP TABLE brin_insert_optimization;
+
+-- a box with a NaN coordinate must not hide the other rows of its page
+-- range (bug #19705)
+CREATE TABLE brin_box_nan (v box);
+INSERT INTO brin_box_nan SELECT box '(0,0),(1,1)' FROM generate_series(1, 100);
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+CREATE INDEX brin_box_nan_idx ON brin_box_nan USING brin (v);
+SET enable_seqscan = off;
+SELECT count(*) FROM brin_box_nan WHERE v && box '(-2,-2),(2,2)';
+SELECT count(*) FROM brin_box_nan WHERE v @> point '(0.5,0.5)';
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(0,0),(1,1)';
+-- the NaN row is found too: unmergeable ranges are always scanned
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(NaN,NaN),(0,0)';
+RESET enable_seqscan;
+DROP TABLE brin_box_nan;
diff --git a/src/test/regress/sql/geometry.sql b/src/test/regress/sql/geometry.sql
index c3ea368da5e..6f45999310c 100644
--- a/src/test/regress/sql/geometry.sql
+++ b/src/test/regress/sql/geometry.sql
@@ -529,3 +529,9 @@ SELECT pg_input_is_valid('(1', 'circle');
SELECT * FROM pg_input_error_info('1,', 'circle');
SELECT pg_input_is_valid('(1,2),-1', 'circle');
SELECT * FROM pg_input_error_info('(1,2),-1', 'circle');
+
+-- NaN becomes an infinite bound (bug #19705)
+SELECT bound_box(box '(0,0),(1,1)', box '(NaN,3),(0,2)');
+-- box_mergeable() rejects NaN coordinates (BRIN box_inclusion_ops)
+SELECT box_mergeable(box '(0,0),(1,1)', box '(NaN,3),(0,2)');
+SELECT box_mergeable(box '(0,0),(1,1)', box '(2,3),(0,2)');
diff --git a/src/test/regress/sql/gist.sql b/src/test/regress/sql/gist.sql
index 57dcc082450..5e637ce11a4 100644
--- a/src/test/regress/sql/gist.sql
+++ b/src/test/regress/sql/gist.sql
@@ -236,3 +236,20 @@ create index gist_tbl_box_index on gist_tbl using gist (b);
insert into gist_tbl
select box(point(0.05*i, 0.05*i)) from generate_series(0,10) as i;
drop table gist_tbl;
+
+-- a box with a NaN coordinate must not poison the union keys (bug #19705)
+create table gist_nan_tbl (b box);
+insert into gist_nan_tbl select box '(1,1),(2,2)' from generate_series(1, 1000);
+insert into gist_nan_tbl values (box '(NaN,NaN),(0,0)');
+create index gist_nan_tbl_index on gist_nan_tbl using gist (b);
+-- also insert rows after the build, to exercise the insertion path
+insert into gist_nan_tbl select box '(3,3),(4,4)' from generate_series(1, 1000);
+insert into gist_nan_tbl values (box '(NaN,NaN),(0,0)');
+set enable_seqscan = off;
+set enable_bitmapscan = off;
+select count(*) from gist_nan_tbl where b && box(point(0,0), point(5,5));
+select count(*) from gist_nan_tbl where b <@ box(point(0,0), point(5,5));
+select count(*) from gist_nan_tbl where b @> point '(1.5,1.5)';
+reset enable_seqscan;
+reset enable_bitmapscan;
+drop table gist_nan_tbl;
--
2.43.0
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows
2026-09-19 17:37 BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows PG Bug reporting form <noreply@postgresql.org>
2026-09-21 23:49 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
2026-09-22 12:12 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Kirill Reshke <reshkekirill@gmail.com>
@ 2026-09-22 12:37 ` Andrey Borodin <x4mmm@yandex-team.ru>
2026-09-23 03:43 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
0 siblings, 1 reply; 11+ messages in thread
From: Andrey Borodin @ 2026-09-22 12:37 UTC (permalink / raw)
To: Kirill Reshke <reshkekirill@gmail.com>; +Cc: shihao zhong <zhong950419@gmail.com>; kehan5800@gmail.com; pgsql-bugs@lists.postgresql.org
On 22 Sep 2026, Kirill Reshke wrote:
> Yes, but for HEAD it's OK and will be an idiomatic way to fix.
I found two false negatives with v2, on fresh indexes.
1. A BRIN range containing just one non-NULL NaN box never calls
box_mergeable(). brin_inclusion_add_value() copies the first value and
returns at "if (new)", leaving INCLUSION_UNMERGEABLE false:
CREATE TABLE b (v box);
INSERT INTO b VALUES ('(NaN,NaN),(0,0)');
CREATE INDEX bi ON b USING brin (v);
SET enable_seqscan = off;
SELECT * FROM b WHERE v ~= box '(NaN,NaN),(0,0)';
This returns no rows, versus one with a seq scan. Adding a finite box
to the range makes the NaN row findable. We need to cover the initial
value too, not just merges.
2. GiST still loses the NaN row for the same equality predicate. With
1000 finite boxes and one NaN box, I get one row via seq scan and none
via GiST. rtree_internal_consistent() implements RTSameStrategyNumber
using box_contain(), which rejects a NaN query even when the stored
bound is infinite.
Thank you!
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows
2026-09-19 17:37 BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows PG Bug reporting form <noreply@postgresql.org>
2026-09-21 23:49 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
2026-09-22 12:12 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Kirill Reshke <reshkekirill@gmail.com>
2026-09-22 12:37 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Andrey Borodin <x4mmm@yandex-team.ru>
@ 2026-09-23 03:43 ` shihao zhong <zhong950419@gmail.com>
2026-09-23 17:51 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Kirill Reshke <reshkekirill@gmail.com>
0 siblings, 1 reply; 11+ messages in thread
From: shihao zhong @ 2026-09-23 03:43 UTC (permalink / raw)
To: Andrey Borodin <x4mmm@yandex-team.ru>; +Cc: Kirill Reshke <reshkekirill@gmail.com>; kehan5800@gmail.com; pgsql-bugs@lists.postgresql.org
Hi Andrey,
> I found two false negatives with v2, on fresh indexes.
Thanks, I reproduced both. v3 attached. 0001 is Kirill's v2 unchanged,
0002 fixes your cases, with tests that fail without it.
1. BRIN now asks the mergeable function about the first value too, by
merging it with itself. No change for inet.
2. GiST internal pages no longer prune ~= when the query has a NaN.
There is nothing safe to compare it with.
0002 also stops gist_box_union() from copying a NaN first entry.
Thanks,
Shihao
Attachments:
[application/octet-stream] v3-0001-Fix-NaN-handling-in-BRIN-box_inclusion_ops-and-Gi.patch (14.5K, ../../CAGRkXqSFL35o-wR2jgmXbeVaRKxADEsLNOcTQFShHCUddZB-bw@mail.gmail.com/3-v3-0001-Fix-NaN-handling-in-BRIN-box_inclusion_ops-and-Gi.patch)
download | inline diff:
From bf6c59f4322aef22bfe583c3c3909da4a767b4df Mon Sep 17 00:00:00 2001
From: reshke <reshke@double.cloud>
Date: Mon, 21 Sep 2026 14:29:36 +0300
Subject: [PATCH v3 1/2] Fix NaN handling in BRIN box_inclusion_ops and GiST
ops
A box with a NaN coordinate fooled BRIN summaries and GiST union
keys, masking completely unlrelated rows. Previous issue of this kind was
fixed in 1acf757, adopt the same fix here.
To fix, add helper functions to float.h and use them in box etc ops,
replacing NaN box bounds with infinite bounds.
Also add support function to box inclusion ops, to mark box with NaN
bounds unmergable.
In GiST, fix search keys to correctly support bounding box with NaN
values.
Author: Kirill Reshke <reshkekirill@gmail.com>
Author: Shihao Zhong <zhong950419@gmail.com>
Reported-by: BUG #19705
Discussion: https://postgr.es/m/19705-548fda77321e062d@postgresql.org
---
src/backend/access/gist/gistproc.c | 40 +++++++++++++++++---------
src/backend/utils/adt/geo_ops.c | 29 ++++++++++++++++---
src/include/catalog/pg_amproc.dat | 2 ++
src/include/catalog/pg_proc.dat | 3 ++
src/include/utils/float.h | 22 ++++++++++++++
src/test/regress/expected/brin.out | 34 ++++++++++++++++++++++
src/test/regress/expected/geometry.out | 20 +++++++++++++
src/test/regress/expected/gist.out | 31 ++++++++++++++++++++
src/test/regress/sql/brin.sql | 15 ++++++++++
src/test/regress/sql/geometry.sql | 6 ++++
src/test/regress/sql/gist.sql | 17 +++++++++++
11 files changed, 201 insertions(+), 18 deletions(-)
diff --git a/src/backend/access/gist/gistproc.c b/src/backend/access/gist/gistproc.c
index f1044f49d6c..2538eb50b07 100644
--- a/src/backend/access/gist/gistproc.c
+++ b/src/backend/access/gist/gistproc.c
@@ -141,19 +141,31 @@ gist_box_consistent(PG_FUNCTION_ARGS)
}
/*
- * Increase BOX b to include addon.
+ * Increase BOX b to include addon. A NaN in either box grows b to the
+ * corresponding infinity.
*/
static void
adjustBox(BOX *b, const BOX *addon)
{
- if (float8_lt(b->high.x, addon->high.x))
- b->high.x = addon->high.x;
- if (float8_gt(b->low.x, addon->low.x))
- b->low.x = addon->low.x;
- if (float8_lt(b->high.y, addon->high.y))
- b->high.y = addon->high.y;
- if (float8_gt(b->low.y, addon->low.y))
- b->low.y = addon->low.y;
+ b->high.x = float8_bound_max(b->high.x, addon->high.x);
+ b->low.x = float8_bound_min(b->low.x, addon->low.x);
+ b->high.y = float8_bound_max(b->high.y, addon->high.y);
+ b->low.y = float8_bound_min(b->low.y, addon->low.y);
+}
+
+/* Copy a BOX, mapping NaNs to infinities. */
+static void
+snapBox(BOX *b, const BOX *box)
+{
+ *b = *box;
+ if (isnan(b->high.x))
+ b->high.x = get_float8_infinity();
+ if (isnan(b->high.y))
+ b->high.y = get_float8_infinity();
+ if (isnan(b->low.x))
+ b->low.x = -get_float8_infinity();
+ if (isnan(b->low.y))
+ b->low.y = -get_float8_infinity();
}
/*
@@ -239,7 +251,7 @@ fallbackSplit(GistEntryVector *entryvec, GIST_SPLITVEC *v)
if (unionL == NULL)
{
unionL = palloc_object(BOX);
- *unionL = *cur;
+ snapBox(unionL, cur);
}
else
adjustBox(unionL, cur);
@@ -252,7 +264,7 @@ fallbackSplit(GistEntryVector *entryvec, GIST_SPLITVEC *v)
if (unionR == NULL)
{
unionR = palloc_object(BOX);
- *unionR = *cur;
+ snapBox(unionR, cur);
}
else
adjustBox(unionR, cur);
@@ -526,7 +538,7 @@ gist_box_picksplit(PG_FUNCTION_ARGS)
{
box = DatumGetBoxP(entryvec->vector[i].key);
if (i == FirstOffsetNumber)
- context.boundingBox = *box;
+ snapBox(&context.boundingBox, box);
else
adjustBox(&context.boundingBox, box);
}
@@ -715,7 +727,7 @@ gist_box_picksplit(PG_FUNCTION_ARGS)
if (v->spl_nleft > 0) \
adjustBox(leftBox, box); \
else \
- *leftBox = *(box); \
+ snapBox(leftBox, box); \
v->spl_left[v->spl_nleft++] = off; \
} while(0)
@@ -724,7 +736,7 @@ gist_box_picksplit(PG_FUNCTION_ARGS)
if (v->spl_nright > 0) \
adjustBox(rightBox, box); \
else \
- *rightBox = *(box); \
+ snapBox(rightBox, box); \
v->spl_right[v->spl_nright++] = off; \
} while(0)
diff --git a/src/backend/utils/adt/geo_ops.c b/src/backend/utils/adt/geo_ops.c
index 73324b91fe5..6a2ad14cb3f 100644
--- a/src/backend/utils/adt/geo_ops.c
+++ b/src/backend/utils/adt/geo_ops.c
@@ -4419,14 +4419,35 @@ boxes_bound_box(PG_FUNCTION_ARGS)
container = palloc_object(BOX);
- container->high.x = float8_max(box1->high.x, box2->high.x);
- container->low.x = float8_min(box1->low.x, box2->low.x);
- container->high.y = float8_max(box1->high.y, box2->high.y);
- container->low.y = float8_min(box1->low.y, box2->low.y);
+ /* NaNs become infinite bounds, never NaNs. */
+ container->high.x = float8_bound_max(box1->high.x, box2->high.x);
+ container->low.x = float8_bound_min(box1->low.x, box2->low.x);
+ container->high.y = float8_bound_max(box1->high.y, box2->high.y);
+ container->low.y = float8_bound_min(box1->low.y, box2->low.y);
PG_RETURN_BOX_P(container);
}
+/*
+ * Boxes with NaN coordinates cannot be merged, since no operator can match a NaN summary.
+ * Used in BRIN box_inclusion_ops PROCNUM_MERGEABLE support.
+ */
+extern Datum box_mergeable(PG_FUNCTION_ARGS);
+Datum
+box_mergeable(PG_FUNCTION_ARGS)
+{
+ BOX *box1 = PG_GETARG_BOX_P(0),
+ *box2 = PG_GETARG_BOX_P(1);
+
+ if (isnan(box1->high.x) || isnan(box1->high.y) ||
+ isnan(box1->low.x) || isnan(box1->low.y) ||
+ isnan(box2->high.x) || isnan(box2->high.y) ||
+ isnan(box2->low.x) || isnan(box2->low.y))
+ PG_RETURN_BOOL(false);
+
+ PG_RETURN_BOOL(true);
+}
+
/***********************************************************************
**
diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat
index 4a1efdbc899..db24e42e795 100644
--- a/src/include/catalog/pg_amproc.dat
+++ b/src/include/catalog/pg_amproc.dat
@@ -2033,6 +2033,8 @@
amproc => 'brin_inclusion_union' },
{ amprocfamily => 'brin/box_inclusion_ops', amproclefttype => 'box',
amprocrighttype => 'box', amprocnum => '11', amproc => 'bound_box' },
+{ amprocfamily => 'brin/box_inclusion_ops', amproclefttype => 'box',
+ amprocrighttype => 'box', amprocnum => '12', amproc => 'box_mergeable' },
{ amprocfamily => 'brin/box_inclusion_ops', amproclefttype => 'box',
amprocrighttype => 'box', amprocnum => '13', amproc => 'box_contain' },
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f46427258e3..bbbce58a962 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -2140,6 +2140,9 @@
{ oid => '4067', descr => 'bounding box of two boxes',
proname => 'bound_box', prorettype => 'box', proargtypes => 'box box',
prosrc => 'boxes_bound_box' },
+{ oid => '8400', descr => 'can two boxes be merged into a single summary',
+ proname => 'box_mergeable', prorettype => 'bool', proargtypes => 'box box',
+ prosrc => 'box_mergeable' },
{ oid => '981', descr => 'box diagonal',
proname => 'diagonal', prorettype => 'lseg', proargtypes => 'box',
prosrc => 'box_diagonal' },
diff --git a/src/include/utils/float.h b/src/include/utils/float.h
index ffa743d6273..843c4ecdef5 100644
--- a/src/include/utils/float.h
+++ b/src/include/utils/float.h
@@ -336,4 +336,26 @@ float8_max(const float8 val1, const float8 val2)
return float8_gt(val1, val2) ? val1 : val2;
}
+/*
+ * float8_bound_max/min: like float8_max/float8_min, but a NaN is mapped
+ * to the corresponding infinity. Used for geometric bounds, where a NaN
+ * would make every comparison with the bound false and thus hide the
+ * values it was meant to cover.
+ */
+static inline float8
+float8_bound_max(const float8 val1, const float8 val2)
+{
+ if (isnan(val1) || isnan(val2))
+ return get_float8_infinity();
+ return float8_max(val1, val2);
+}
+
+static inline float8
+float8_bound_min(const float8 val1, const float8 val2)
+{
+ if (isnan(val1) || isnan(val2))
+ return -get_float8_infinity();
+ return float8_min(val1, val2);
+}
+
#endif /* FLOAT_H */
diff --git a/src/test/regress/expected/brin.out b/src/test/regress/expected/brin.out
index e1db2280cf9..a73f7b7e82d 100644
--- a/src/test/regress/expected/brin.out
+++ b/src/test/regress/expected/brin.out
@@ -589,3 +589,37 @@ CREATE INDEX brin_insert_optimization_idx ON brin_insert_optimization USING brin
UPDATE brin_insert_optimization SET a = a;
REINDEX INDEX CONCURRENTLY brin_insert_optimization_idx;
DROP TABLE brin_insert_optimization;
+-- a box with a NaN coordinate must not hide the other rows of its page
+-- range (bug #19705)
+CREATE TABLE brin_box_nan (v box);
+INSERT INTO brin_box_nan SELECT box '(0,0),(1,1)' FROM generate_series(1, 100);
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+CREATE INDEX brin_box_nan_idx ON brin_box_nan USING brin (v);
+SET enable_seqscan = off;
+SELECT count(*) FROM brin_box_nan WHERE v && box '(-2,-2),(2,2)';
+ count
+-------
+ 100
+(1 row)
+
+SELECT count(*) FROM brin_box_nan WHERE v @> point '(0.5,0.5)';
+ count
+-------
+ 100
+(1 row)
+
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(0,0),(1,1)';
+ count
+-------
+ 100
+(1 row)
+
+-- the NaN row is found too: unmergeable ranges are always scanned
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(NaN,NaN),(0,0)';
+ count
+-------
+ 1
+(1 row)
+
+RESET enable_seqscan;
+DROP TABLE brin_box_nan;
diff --git a/src/test/regress/expected/geometry.out b/src/test/regress/expected/geometry.out
index 1d168b21cbc..1372d34a668 100644
--- a/src/test/regress/expected/geometry.out
+++ b/src/test/regress/expected/geometry.out
@@ -5321,3 +5321,23 @@ SELECT * FROM pg_input_error_info('(1,2),-1', 'circle');
invalid input syntax for type circle: "(1,2),-1" | | | 22P02
(1 row)
+-- NaN becomes an infinite bound (bug #19705)
+SELECT bound_box(box '(0,0),(1,1)', box '(NaN,3),(0,2)');
+ bound_box
+--------------------
+ (Infinity,3),(0,0)
+(1 row)
+
+-- box_mergeable() rejects NaN coordinates (BRIN box_inclusion_ops)
+SELECT box_mergeable(box '(0,0),(1,1)', box '(NaN,3),(0,2)');
+ box_mergeable
+---------------
+ f
+(1 row)
+
+SELECT box_mergeable(box '(0,0),(1,1)', box '(2,3),(0,2)');
+ box_mergeable
+---------------
+ t
+(1 row)
+
diff --git a/src/test/regress/expected/gist.out b/src/test/regress/expected/gist.out
index ac79f94aa80..82197b4dcef 100644
--- a/src/test/regress/expected/gist.out
+++ b/src/test/regress/expected/gist.out
@@ -463,3 +463,34 @@ create index gist_tbl_box_index on gist_tbl using gist (b);
insert into gist_tbl
select box(point(0.05*i, 0.05*i)) from generate_series(0,10) as i;
drop table gist_tbl;
+-- a box with a NaN coordinate must not poison the union keys (bug #19705)
+create table gist_nan_tbl (b box);
+insert into gist_nan_tbl select box '(1,1),(2,2)' from generate_series(1, 1000);
+insert into gist_nan_tbl values (box '(NaN,NaN),(0,0)');
+create index gist_nan_tbl_index on gist_nan_tbl using gist (b);
+-- also insert rows after the build, to exercise the insertion path
+insert into gist_nan_tbl select box '(3,3),(4,4)' from generate_series(1, 1000);
+insert into gist_nan_tbl values (box '(NaN,NaN),(0,0)');
+set enable_seqscan = off;
+set enable_bitmapscan = off;
+select count(*) from gist_nan_tbl where b && box(point(0,0), point(5,5));
+ count
+-------
+ 2000
+(1 row)
+
+select count(*) from gist_nan_tbl where b <@ box(point(0,0), point(5,5));
+ count
+-------
+ 2000
+(1 row)
+
+select count(*) from gist_nan_tbl where b @> point '(1.5,1.5)';
+ count
+-------
+ 1000
+(1 row)
+
+reset enable_seqscan;
+reset enable_bitmapscan;
+drop table gist_nan_tbl;
diff --git a/src/test/regress/sql/brin.sql b/src/test/regress/sql/brin.sql
index 7ea97f47c8d..3484869d676 100644
--- a/src/test/regress/sql/brin.sql
+++ b/src/test/regress/sql/brin.sql
@@ -534,3 +534,18 @@ CREATE INDEX brin_insert_optimization_idx ON brin_insert_optimization USING brin
UPDATE brin_insert_optimization SET a = a;
REINDEX INDEX CONCURRENTLY brin_insert_optimization_idx;
DROP TABLE brin_insert_optimization;
+
+-- a box with a NaN coordinate must not hide the other rows of its page
+-- range (bug #19705)
+CREATE TABLE brin_box_nan (v box);
+INSERT INTO brin_box_nan SELECT box '(0,0),(1,1)' FROM generate_series(1, 100);
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+CREATE INDEX brin_box_nan_idx ON brin_box_nan USING brin (v);
+SET enable_seqscan = off;
+SELECT count(*) FROM brin_box_nan WHERE v && box '(-2,-2),(2,2)';
+SELECT count(*) FROM brin_box_nan WHERE v @> point '(0.5,0.5)';
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(0,0),(1,1)';
+-- the NaN row is found too: unmergeable ranges are always scanned
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(NaN,NaN),(0,0)';
+RESET enable_seqscan;
+DROP TABLE brin_box_nan;
diff --git a/src/test/regress/sql/geometry.sql b/src/test/regress/sql/geometry.sql
index c3ea368da5e..6f45999310c 100644
--- a/src/test/regress/sql/geometry.sql
+++ b/src/test/regress/sql/geometry.sql
@@ -529,3 +529,9 @@ SELECT pg_input_is_valid('(1', 'circle');
SELECT * FROM pg_input_error_info('1,', 'circle');
SELECT pg_input_is_valid('(1,2),-1', 'circle');
SELECT * FROM pg_input_error_info('(1,2),-1', 'circle');
+
+-- NaN becomes an infinite bound (bug #19705)
+SELECT bound_box(box '(0,0),(1,1)', box '(NaN,3),(0,2)');
+-- box_mergeable() rejects NaN coordinates (BRIN box_inclusion_ops)
+SELECT box_mergeable(box '(0,0),(1,1)', box '(NaN,3),(0,2)');
+SELECT box_mergeable(box '(0,0),(1,1)', box '(2,3),(0,2)');
diff --git a/src/test/regress/sql/gist.sql b/src/test/regress/sql/gist.sql
index 57dcc082450..5e637ce11a4 100644
--- a/src/test/regress/sql/gist.sql
+++ b/src/test/regress/sql/gist.sql
@@ -236,3 +236,20 @@ create index gist_tbl_box_index on gist_tbl using gist (b);
insert into gist_tbl
select box(point(0.05*i, 0.05*i)) from generate_series(0,10) as i;
drop table gist_tbl;
+
+-- a box with a NaN coordinate must not poison the union keys (bug #19705)
+create table gist_nan_tbl (b box);
+insert into gist_nan_tbl select box '(1,1),(2,2)' from generate_series(1, 1000);
+insert into gist_nan_tbl values (box '(NaN,NaN),(0,0)');
+create index gist_nan_tbl_index on gist_nan_tbl using gist (b);
+-- also insert rows after the build, to exercise the insertion path
+insert into gist_nan_tbl select box '(3,3),(4,4)' from generate_series(1, 1000);
+insert into gist_nan_tbl values (box '(NaN,NaN),(0,0)');
+set enable_seqscan = off;
+set enable_bitmapscan = off;
+select count(*) from gist_nan_tbl where b && box(point(0,0), point(5,5));
+select count(*) from gist_nan_tbl where b <@ box(point(0,0), point(5,5));
+select count(*) from gist_nan_tbl where b @> point '(1.5,1.5)';
+reset enable_seqscan;
+reset enable_bitmapscan;
+drop table gist_nan_tbl;
--
2.37.1 (Apple Git-137.1)
[application/octet-stream] v3-0002-Fix-NaN-false-negatives-in-BRIN-and-GiST-box-inde.patch (5.8K, ../../CAGRkXqSFL35o-wR2jgmXbeVaRKxADEsLNOcTQFShHCUddZB-bw@mail.gmail.com/4-v3-0002-Fix-NaN-false-negatives-in-BRIN-and-GiST-box-inde.patch)
download | inline diff:
From f6a9b0efee91e3ba6d3052efccaccb08d4c80ae8 Mon Sep 17 00:00:00 2001
From: Shihao <zhong950419@gmail.com>
Date: Tue, 22 Sep 2026 23:09:22 -0400
Subject: [PATCH v3 2/2] Fix NaN false negatives in BRIN and GiST box indexes
BRIN never checked the first value of a range for mergeability, so a
range holding only a NaN box matched nothing. GiST internal pages
checked ~= with box_contain(), which never matches a NaN query.
Also map NaN to infinity for the first entry in gist_box_union().
Reported-by: Andrey Borodin <x4mmm@yandex-team.ru>
Discussion: https://postgr.es/m/19705-548fda77321e062d@postgresql.org
---
src/backend/access/brin/brin_inclusion.c | 8 ++++++++
src/backend/access/gist/gistproc.c | 11 ++++++++++-
src/backend/utils/adt/geo_ops.c | 4 ++--
src/test/regress/expected/brin.out | 9 +++++++++
src/test/regress/expected/gist.out | 6 ++++++
src/test/regress/sql/brin.sql | 4 ++++
src/test/regress/sql/gist.sql | 1 +
7 files changed, 40 insertions(+), 3 deletions(-)
diff --git a/src/backend/access/brin/brin_inclusion.c b/src/backend/access/brin/brin_inclusion.c
index 5a2058d9aad..b772617cf59 100644
--- a/src/backend/access/brin/brin_inclusion.c
+++ b/src/backend/access/brin/brin_inclusion.c
@@ -191,8 +191,16 @@ brin_inclusion_add_value(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(false);
}
+ /* A value may not be mergeable even with itself */
if (new)
+ {
+ finfo = inclusion_get_procinfo(bdesc, attno, PROCNUM_MERGEABLE, true);
+ if (finfo != NULL &&
+ !DatumGetBool(FunctionCall2Coll(finfo, colloid, newval, newval)))
+ column->bv_values[INCLUSION_UNMERGEABLE] = BoolGetDatum(true);
+
PG_RETURN_BOOL(true);
+ }
/* Check if the new value is already contained. */
finfo = inclusion_get_procinfo(bdesc, attno, PROCNUM_CONTAINS, true);
diff --git a/src/backend/access/gist/gistproc.c b/src/backend/access/gist/gistproc.c
index 2538eb50b07..98c10018959 100644
--- a/src/backend/access/gist/gistproc.c
+++ b/src/backend/access/gist/gistproc.c
@@ -186,7 +186,7 @@ gist_box_union(PG_FUNCTION_ARGS)
numranges = entryvec->n;
pageunion = palloc_object(BOX);
cur = DatumGetBoxP(entryvec->vector[0].key);
- memcpy(pageunion, cur, sizeof(BOX));
+ snapBox(pageunion, cur);
for (i = 1; i < numranges; i++)
{
@@ -999,6 +999,15 @@ rtree_internal_consistent(BOX *key, BOX *query, StrategyNumber strategy)
PointerGetDatum(query)));
break;
case RTSameStrategyNumber:
+ /* box_same() matches NaN, box_contain() never does */
+ if (isnan(query->high.x) || isnan(query->high.y) ||
+ isnan(query->low.x) || isnan(query->low.y))
+ retval = true;
+ else
+ retval = DatumGetBool(DirectFunctionCall2(box_contain,
+ PointerGetDatum(key),
+ PointerGetDatum(query)));
+ break;
case RTContainsStrategyNumber:
retval = DatumGetBool(DirectFunctionCall2(box_contain,
PointerGetDatum(key),
diff --git a/src/backend/utils/adt/geo_ops.c b/src/backend/utils/adt/geo_ops.c
index 6a2ad14cb3f..d38149267a3 100644
--- a/src/backend/utils/adt/geo_ops.c
+++ b/src/backend/utils/adt/geo_ops.c
@@ -4436,8 +4436,8 @@ extern Datum box_mergeable(PG_FUNCTION_ARGS);
Datum
box_mergeable(PG_FUNCTION_ARGS)
{
- BOX *box1 = PG_GETARG_BOX_P(0),
- *box2 = PG_GETARG_BOX_P(1);
+ BOX *box1 = PG_GETARG_BOX_P(0),
+ *box2 = PG_GETARG_BOX_P(1);
if (isnan(box1->high.x) || isnan(box1->high.y) ||
isnan(box1->low.x) || isnan(box1->low.y) ||
diff --git a/src/test/regress/expected/brin.out b/src/test/regress/expected/brin.out
index a73f7b7e82d..6b91c00ca15 100644
--- a/src/test/regress/expected/brin.out
+++ b/src/test/regress/expected/brin.out
@@ -621,5 +621,14 @@ SELECT count(*) FROM brin_box_nan WHERE v ~= box '(NaN,NaN),(0,0)';
1
(1 row)
+TRUNCATE brin_box_nan;
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+REINDEX INDEX brin_box_nan_idx;
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(NaN,NaN),(0,0)';
+ count
+-------
+ 1
+(1 row)
+
RESET enable_seqscan;
DROP TABLE brin_box_nan;
diff --git a/src/test/regress/expected/gist.out b/src/test/regress/expected/gist.out
index 82197b4dcef..16cb5d1574d 100644
--- a/src/test/regress/expected/gist.out
+++ b/src/test/regress/expected/gist.out
@@ -491,6 +491,12 @@ select count(*) from gist_nan_tbl where b @> point '(1.5,1.5)';
1000
(1 row)
+select count(*) from gist_nan_tbl where b ~= box '(NaN,NaN),(0,0)';
+ count
+-------
+ 2
+(1 row)
+
reset enable_seqscan;
reset enable_bitmapscan;
drop table gist_nan_tbl;
diff --git a/src/test/regress/sql/brin.sql b/src/test/regress/sql/brin.sql
index 3484869d676..94b8c2755dc 100644
--- a/src/test/regress/sql/brin.sql
+++ b/src/test/regress/sql/brin.sql
@@ -547,5 +547,9 @@ SELECT count(*) FROM brin_box_nan WHERE v @> point '(0.5,0.5)';
SELECT count(*) FROM brin_box_nan WHERE v ~= box '(0,0),(1,1)';
-- the NaN row is found too: unmergeable ranges are always scanned
SELECT count(*) FROM brin_box_nan WHERE v ~= box '(NaN,NaN),(0,0)';
+TRUNCATE brin_box_nan;
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+REINDEX INDEX brin_box_nan_idx;
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(NaN,NaN),(0,0)';
RESET enable_seqscan;
DROP TABLE brin_box_nan;
diff --git a/src/test/regress/sql/gist.sql b/src/test/regress/sql/gist.sql
index 5e637ce11a4..cab9e06cfb5 100644
--- a/src/test/regress/sql/gist.sql
+++ b/src/test/regress/sql/gist.sql
@@ -250,6 +250,7 @@ set enable_bitmapscan = off;
select count(*) from gist_nan_tbl where b && box(point(0,0), point(5,5));
select count(*) from gist_nan_tbl where b <@ box(point(0,0), point(5,5));
select count(*) from gist_nan_tbl where b @> point '(1.5,1.5)';
+select count(*) from gist_nan_tbl where b ~= box '(NaN,NaN),(0,0)';
reset enable_seqscan;
reset enable_bitmapscan;
drop table gist_nan_tbl;
--
2.37.1 (Apple Git-137.1)
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows
2026-09-19 17:37 BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows PG Bug reporting form <noreply@postgresql.org>
2026-09-21 23:49 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
2026-09-22 12:12 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Kirill Reshke <reshkekirill@gmail.com>
2026-09-22 12:37 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Andrey Borodin <x4mmm@yandex-team.ru>
2026-09-23 03:43 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
@ 2026-09-23 17:51 ` Kirill Reshke <reshkekirill@gmail.com>
2026-09-24 02:43 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
0 siblings, 1 reply; 11+ messages in thread
From: Kirill Reshke @ 2026-09-23 17:51 UTC (permalink / raw)
To: shihao zhong <zhong950419@gmail.com>; +Cc: Andrey Borodin <x4mmm@yandex-team.ru>; kehan5800@gmail.com; pgsql-bugs@lists.postgresql.org
On Wed, 23 Sept 2026 at 08:43, shihao zhong <zhong950419@gmail.com> wrote:
>
> Hi Andrey,
> > I found two false negatives with v2, on fresh indexes.
>
> Thanks, I reproduced both. v3 attached. 0001 is Kirill's v2 unchanged,
> 0002 fixes your cases, with tests that fail without it.
>
> 1. BRIN now asks the mergeable function about the first value too, by
> merging it with itself. No change for inet.
>
> 2. GiST internal pages no longer prune ~= when the query has a NaN.
> There is nothing safe to compare it with.
>
> 0002 also stops gist_box_union() from copying a NaN first entry.
>
> Thanks,
> Shihao
>
Cool, thanks for v3.
I think that addition to brin_inclusion_add_value is correct but looks
like we need the same guard for brin_inclusion_consistent?
Also, maybe it is worth making two patches here, one for BRIN and
other for GiST for sake of simplicity.
I also think that we can avoid rebuilding indexes after fix here, only
lose selectivity on the poisoned subtrees. Looks like an internal key
carrying a NaN should match every strategy. So, we can place new
tuples with NaN -> inf substitution, while making tree search recurse
in whole subtree in case of old (NaN) value.
Does it sound?
Also, we can make the master BRIN patch to not touch boxes_bound_box()
at al. Only INCLUSION_UNMERGEABLE is sufficient.
--
Best regards,
Kirill Reshke
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows
2026-09-19 17:37 BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows PG Bug reporting form <noreply@postgresql.org>
2026-09-21 23:49 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
2026-09-22 12:12 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Kirill Reshke <reshkekirill@gmail.com>
2026-09-22 12:37 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Andrey Borodin <x4mmm@yandex-team.ru>
2026-09-23 03:43 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
2026-09-23 17:51 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Kirill Reshke <reshkekirill@gmail.com>
@ 2026-09-24 02:43 ` shihao zhong <zhong950419@gmail.com>
2026-09-24 06:22 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Kirill Reshke <reshkekirill@gmail.com>
0 siblings, 1 reply; 11+ messages in thread
From: shihao zhong @ 2026-09-24 02:43 UTC (permalink / raw)
To: Kirill Reshke <reshkekirill@gmail.com>; +Cc: Andrey Borodin <x4mmm@yandex-team.ru>; kehan5800@gmail.com; pgsql-bugs@lists.postgresql.org
Hi Kirill,
Agreed on all four. v4 attached, split into BRIN and GiST.
0001 is BRIN, master only. bound_box() is untouched. The consistent
function also scans a range whose union is not mergeable with itself,
so old NaN summaries work without REINDEX. Needs a catversion bump.
0002 is GiST, back to 14. Old internal keys with a NaN match every
search and get KNN distance zero. Without the distance part, KNN on old
indexes returned rows out of order. 0002 also fixes point ~= with a NaN
query, which lost rows even on a fresh index. The leaf check used
FPeq(), but point_eq() compares exactly when there is a NaN.
The .nocfbot file is BRIN for the back branches, against REL_18. It
needs no catalog change. Consistent scans a range whose union does not
contain itself, using the existing contains support function. A NaN box
fails that, and any sane union passes. Nothing on disk changes, so old
indexes work without REINDEX after a minor upgrade. The code applies to
14 and later, the test hunk needs a small rebase on 14 to 16. This check
would also work on master, if we want one fix everywhere.
I checked 0001 and 0002 with indexes built by unpatched master, then
pg_upgraded, no REINDEX. The back branch patch got the same check on
REL_18, with a minor upgrade.
Thanks,
Shihao
Attachments:
[application/octet-stream] v4-REL_18-0001-Fix-BRIN-box_inclusion_ops-hiding-rows-next-to-a-.patch.nocfbot (5.6K, ../../CAGRkXqQ7q0U3jmitwOjPY2BNBkxWsnyahgoYyMs4H=hwVtPFgw@mail.gmail.com/3-v4-REL_18-0001-Fix-BRIN-box_inclusion_ops-hiding-rows-next-to-a-.patch.nocfbot)
download
[application/octet-stream] v4-0002-Fix-GiST-box-indexes-hiding-rows-next-to-a-NaN-bo.patch (13.8K, ../../CAGRkXqQ7q0U3jmitwOjPY2BNBkxWsnyahgoYyMs4H=hwVtPFgw@mail.gmail.com/4-v4-0002-Fix-GiST-box-indexes-hiding-rows-next-to-a-NaN-bo.patch)
download | inline diff:
From 6b6c0c7d77c7bca944087140a8a8002cc55b1d2e Mon Sep 17 00:00:00 2001
From: Shihao <zhong950419@gmail.com>
Date: Wed, 23 Sep 2026 21:23:52 -0400
Subject: [PATCH v4 2/2] Fix GiST box indexes hiding rows next to a NaN box
GiST union keys kept NaN coordinates, and no box comparison is true
against a NaN bound. So an internal key covering a NaN box hid its
whole subtree from searches, and KNN scans could return rows from it
too late.
Map NaNs to infinities when building union keys. Internal keys that
still have a NaN, from indexes built before this fix, now match every
search and get distance zero. So those indexes return correct results
without a REINDEX, only less selectively. Also stop internal pages
from pruning ~= when the query has a NaN, since box_same() matches it.
For points, ~= also missed NaN queries at the leaf level, because it
compared with FPeq() while point_eq() compares exactly when there is a
NaN. Make the index agree with point_eq().
Author: Kirill Reshke <reshkekirill@gmail.com>
Author: Shihao Zhong <zhong950419@gmail.com>
Reported-by: Ke <kehan5800@gmail.com>
Reported-by: Andrey Borodin <x4mmm@yandex-team.ru>
Discussion: https://postgr.es/m/19705-548fda77321e062d@postgresql.org
Backpatch-through: 14
---
src/backend/access/gist/gistproc.c | 139 ++++++++++++++++++++++++-----
src/test/regress/expected/gist.out | 73 +++++++++++++++
src/test/regress/sql/gist.sql | 35 ++++++++
3 files changed, 225 insertions(+), 22 deletions(-)
diff --git a/src/backend/access/gist/gistproc.c b/src/backend/access/gist/gistproc.c
index f1044f49d6c..0f405d4450f 100644
--- a/src/backend/access/gist/gistproc.c
+++ b/src/backend/access/gist/gistproc.c
@@ -43,6 +43,45 @@ static bool gist_bbox_zorder_abbrev_abort(int memtupcount, SortSupport ssup);
/* Minimum accepted ratio of split */
#define LIMIT_RATIO 0.3
+/*
+ * Does the box have a NaN coordinate?
+ *
+ * No box comparison is true against a NaN, so a union key with one would
+ * hide everything below it. Union keys map NaNs to infinities instead (see
+ * adjustBox), but indexes built before that was done can still have NaN
+ * internal keys, and searches must descend into those.
+ */
+static inline bool
+box_has_nan(const BOX *box)
+{
+ return isnan(box->high.x) || isnan(box->high.y) ||
+ isnan(box->low.x) || isnan(box->low.y);
+}
+
+/* Is the entry an internal key with a NaN coordinate? */
+static inline bool
+nan_internal_key(const GISTENTRY *entry)
+{
+ return !GIST_LEAF(entry) && box_has_nan(DatumGetBoxP(entry->key));
+}
+
+/* float8_max() and float8_min(), but a NaN gives the matching infinity */
+static inline float8
+bound_max(float8 val1, float8 val2)
+{
+ if (isnan(val1) || isnan(val2))
+ return get_float8_infinity();
+ return float8_max(val1, val2);
+}
+
+static inline float8
+bound_min(float8 val1, float8 val2)
+{
+ if (isnan(val1) || isnan(val2))
+ return -get_float8_infinity();
+ return float8_min(val1, val2);
+}
+
/**************************************************
* Box ops
@@ -126,6 +165,10 @@ gist_box_consistent(PG_FUNCTION_ARGS)
if (DatumGetBoxP(entry->key) == NULL || query == NULL)
PG_RETURN_BOOL(false);
+ /* A NaN internal key tells us nothing, see box_has_nan() */
+ if (nan_internal_key(entry))
+ PG_RETURN_BOOL(true);
+
/*
* if entry is not leaf, use rtree_internal_consistent, else use
* gist_box_leaf_consistent
@@ -141,19 +184,31 @@ gist_box_consistent(PG_FUNCTION_ARGS)
}
/*
- * Increase BOX b to include addon.
+ * Increase BOX b to include addon. A NaN in either box grows b to the
+ * matching infinity.
*/
static void
adjustBox(BOX *b, const BOX *addon)
{
- if (float8_lt(b->high.x, addon->high.x))
- b->high.x = addon->high.x;
- if (float8_gt(b->low.x, addon->low.x))
- b->low.x = addon->low.x;
- if (float8_lt(b->high.y, addon->high.y))
- b->high.y = addon->high.y;
- if (float8_gt(b->low.y, addon->low.y))
- b->low.y = addon->low.y;
+ b->high.x = bound_max(b->high.x, addon->high.x);
+ b->low.x = bound_min(b->low.x, addon->low.x);
+ b->high.y = bound_max(b->high.y, addon->high.y);
+ b->low.y = bound_min(b->low.y, addon->low.y);
+}
+
+/* Copy a BOX, mapping NaNs to infinities. */
+static void
+snapBox(BOX *b, const BOX *box)
+{
+ *b = *box;
+ if (isnan(b->high.x))
+ b->high.x = get_float8_infinity();
+ if (isnan(b->high.y))
+ b->high.y = get_float8_infinity();
+ if (isnan(b->low.x))
+ b->low.x = -get_float8_infinity();
+ if (isnan(b->low.y))
+ b->low.y = -get_float8_infinity();
}
/*
@@ -174,7 +229,7 @@ gist_box_union(PG_FUNCTION_ARGS)
numranges = entryvec->n;
pageunion = palloc_object(BOX);
cur = DatumGetBoxP(entryvec->vector[0].key);
- memcpy(pageunion, cur, sizeof(BOX));
+ snapBox(pageunion, cur);
for (i = 1; i < numranges; i++)
{
@@ -239,7 +294,7 @@ fallbackSplit(GistEntryVector *entryvec, GIST_SPLITVEC *v)
if (unionL == NULL)
{
unionL = palloc_object(BOX);
- *unionL = *cur;
+ snapBox(unionL, cur);
}
else
adjustBox(unionL, cur);
@@ -252,7 +307,7 @@ fallbackSplit(GistEntryVector *entryvec, GIST_SPLITVEC *v)
if (unionR == NULL)
{
unionR = palloc_object(BOX);
- *unionR = *cur;
+ snapBox(unionR, cur);
}
else
adjustBox(unionR, cur);
@@ -526,7 +581,7 @@ gist_box_picksplit(PG_FUNCTION_ARGS)
{
box = DatumGetBoxP(entryvec->vector[i].key);
if (i == FirstOffsetNumber)
- context.boundingBox = *box;
+ snapBox(&context.boundingBox, box);
else
adjustBox(&context.boundingBox, box);
}
@@ -715,7 +770,7 @@ gist_box_picksplit(PG_FUNCTION_ARGS)
if (v->spl_nleft > 0) \
adjustBox(leftBox, box); \
else \
- *leftBox = *(box); \
+ snapBox(leftBox, box); \
v->spl_left[v->spl_nleft++] = off; \
} while(0)
@@ -724,7 +779,7 @@ gist_box_picksplit(PG_FUNCTION_ARGS)
if (v->spl_nright > 0) \
adjustBox(rightBox, box); \
else \
- *rightBox = *(box); \
+ snapBox(rightBox, box); \
v->spl_right[v->spl_nright++] = off; \
} while(0)
@@ -959,6 +1014,13 @@ rtree_internal_consistent(BOX *key, BOX *query, StrategyNumber strategy)
{
bool retval;
+ /*
+ * A NaN query can still match with ~=, since box_same() treats NaNs as
+ * equal, but box_contain() never matches it.
+ */
+ if (strategy == RTSameStrategyNumber && box_has_nan(query))
+ return true;
+
switch (strategy)
{
case RTLeftStrategyNumber:
@@ -1077,6 +1139,10 @@ gist_poly_consistent(PG_FUNCTION_ARGS)
if (DatumGetBoxP(entry->key) == NULL || query == NULL)
PG_RETURN_BOOL(false);
+ /* A NaN internal key tells us nothing, see box_has_nan() */
+ if (nan_internal_key(entry))
+ PG_RETURN_BOOL(true);
+
/*
* Since the operators require recheck anyway, we can just use
* rtree_internal_consistent even at leaf nodes. (This works in part
@@ -1147,6 +1213,10 @@ gist_circle_consistent(PG_FUNCTION_ARGS)
if (DatumGetBoxP(entry->key) == NULL || query == NULL)
PG_RETURN_BOOL(false);
+ /* A NaN internal key tells us nothing, see box_has_nan() */
+ if (nan_internal_key(entry))
+ PG_RETURN_BOOL(true);
+
/*
* Since the operators require recheck anyway, we can just use
* rtree_internal_consistent even at leaf nodes. (This works in part
@@ -1307,7 +1377,20 @@ gist_point_consistent_internal(StrategyNumber strategy,
result = FPlt(key->low.y, query->y);
break;
case RTSameStrategyNumber:
- if (isLeaf)
+
+ /*
+ * point_eq() compares exactly when there is a NaN, so a NaN query
+ * matches points with the same NaN coordinates. Internal keys
+ * cannot tell us where those are, so search everything below.
+ */
+ if (isnan(query->x) || isnan(query->y))
+ {
+ /* key.high must equal key.low, so we can disregard it */
+ result = !isLeaf ||
+ (float8_eq(key->low.x, query->x) &&
+ float8_eq(key->low.y, query->y));
+ }
+ else if (isLeaf)
{
/* key.high must equal key.low, so we can disregard it */
result = (FPeq(key->low.x, query->x) &&
@@ -1345,6 +1428,10 @@ gist_point_consistent(PG_FUNCTION_ARGS)
bool result;
StrategyNumber strategyGroup;
+ /* A NaN internal key tells us nothing, see box_has_nan() */
+ if (nan_internal_key(entry))
+ PG_RETURN_BOOL(true);
+
/*
* We have to remap these strategy numbers to get this klugy
* classification logic to work.
@@ -1465,9 +1552,13 @@ gist_point_distance(PG_FUNCTION_ARGS)
switch (strategyGroup)
{
case PointStrategyNumberGroup:
- distance = computeDistance(GIST_LEAF(entry),
- DatumGetBoxP(entry->key),
- PG_GETARG_POINT_P(1));
+ /* A NaN internal key tells us nothing, see box_has_nan() */
+ if (nan_internal_key(entry))
+ distance = 0.0;
+ else
+ distance = computeDistance(GIST_LEAF(entry),
+ DatumGetBoxP(entry->key),
+ PG_GETARG_POINT_P(1));
break;
default:
elog(ERROR, "unrecognized strategy number: %d", strategy);
@@ -1487,9 +1578,13 @@ gist_bbox_distance(GISTENTRY *entry, Datum query, StrategyNumber strategy)
switch (strategyGroup)
{
case PointStrategyNumberGroup:
- distance = computeDistance(false,
- DatumGetBoxP(entry->key),
- DatumGetPointP(query));
+ /* A NaN internal key tells us nothing, see box_has_nan() */
+ if (nan_internal_key(entry))
+ distance = 0.0;
+ else
+ distance = computeDistance(false,
+ DatumGetBoxP(entry->key),
+ DatumGetPointP(query));
break;
default:
elog(ERROR, "unrecognized strategy number: %d", strategy);
diff --git a/src/test/regress/expected/gist.out b/src/test/regress/expected/gist.out
index ac79f94aa80..174f8b4251f 100644
--- a/src/test/regress/expected/gist.out
+++ b/src/test/regress/expected/gist.out
@@ -463,3 +463,76 @@ create index gist_tbl_box_index on gist_tbl using gist (b);
insert into gist_tbl
select box(point(0.05*i, 0.05*i)) from generate_series(0,10) as i;
drop table gist_tbl;
+-- a box with a NaN coordinate must not poison the union keys (bug #19705)
+create table gist_nan_tbl (b box);
+insert into gist_nan_tbl select box '(1,1),(2,2)' from generate_series(1, 1000);
+insert into gist_nan_tbl values (box '(NaN,NaN),(0,0)');
+create index gist_nan_tbl_index on gist_nan_tbl using gist (b);
+-- also insert rows after the build, to exercise the insertion path
+insert into gist_nan_tbl select box '(3,3),(4,4)' from generate_series(1, 1000);
+insert into gist_nan_tbl values (box '(NaN,NaN),(0,0)');
+set enable_seqscan = off;
+set enable_bitmapscan = off;
+select count(*) from gist_nan_tbl where b && box(point(0,0), point(5,5));
+ count
+-------
+ 2000
+(1 row)
+
+select count(*) from gist_nan_tbl where b <@ box(point(0,0), point(5,5));
+ count
+-------
+ 2000
+(1 row)
+
+select count(*) from gist_nan_tbl where b @> point '(1.5,1.5)';
+ count
+-------
+ 1000
+(1 row)
+
+select count(*) from gist_nan_tbl where b ~= box '(NaN,NaN),(0,0)';
+ count
+-------
+ 2
+(1 row)
+
+reset enable_seqscan;
+reset enable_bitmapscan;
+drop table gist_nan_tbl;
+-- point ~= matches NaN coordinates exactly, the index must agree
+create table gist_nan_point_tbl (p point);
+insert into gist_nan_point_tbl
+ select point(i % 100, i / 100) from generate_series(0, 9999) i;
+insert into gist_nan_point_tbl
+ values ('(NaN,NaN)'), ('(NaN,3)'), ('(3,NaN)'), ('(NaN,NaN)');
+create index gist_nan_point_tbl_index on gist_nan_point_tbl using gist (p);
+set enable_seqscan = off;
+set enable_bitmapscan = off;
+select count(*) from gist_nan_point_tbl where p ~= point '(NaN,NaN)';
+ count
+-------
+ 2
+(1 row)
+
+select count(*) from gist_nan_point_tbl where p ~= point '(NaN,3)';
+ count
+-------
+ 1
+(1 row)
+
+select count(*) from gist_nan_point_tbl where p ~= point '(3,NaN)';
+ count
+-------
+ 1
+(1 row)
+
+select count(*) from gist_nan_point_tbl where p ~= point '(3,3)';
+ count
+-------
+ 1
+(1 row)
+
+reset enable_seqscan;
+reset enable_bitmapscan;
+drop table gist_nan_point_tbl;
diff --git a/src/test/regress/sql/gist.sql b/src/test/regress/sql/gist.sql
index 57dcc082450..654318fb4db 100644
--- a/src/test/regress/sql/gist.sql
+++ b/src/test/regress/sql/gist.sql
@@ -236,3 +236,38 @@ create index gist_tbl_box_index on gist_tbl using gist (b);
insert into gist_tbl
select box(point(0.05*i, 0.05*i)) from generate_series(0,10) as i;
drop table gist_tbl;
+
+-- a box with a NaN coordinate must not poison the union keys (bug #19705)
+create table gist_nan_tbl (b box);
+insert into gist_nan_tbl select box '(1,1),(2,2)' from generate_series(1, 1000);
+insert into gist_nan_tbl values (box '(NaN,NaN),(0,0)');
+create index gist_nan_tbl_index on gist_nan_tbl using gist (b);
+-- also insert rows after the build, to exercise the insertion path
+insert into gist_nan_tbl select box '(3,3),(4,4)' from generate_series(1, 1000);
+insert into gist_nan_tbl values (box '(NaN,NaN),(0,0)');
+set enable_seqscan = off;
+set enable_bitmapscan = off;
+select count(*) from gist_nan_tbl where b && box(point(0,0), point(5,5));
+select count(*) from gist_nan_tbl where b <@ box(point(0,0), point(5,5));
+select count(*) from gist_nan_tbl where b @> point '(1.5,1.5)';
+select count(*) from gist_nan_tbl where b ~= box '(NaN,NaN),(0,0)';
+reset enable_seqscan;
+reset enable_bitmapscan;
+drop table gist_nan_tbl;
+
+-- point ~= matches NaN coordinates exactly, the index must agree
+create table gist_nan_point_tbl (p point);
+insert into gist_nan_point_tbl
+ select point(i % 100, i / 100) from generate_series(0, 9999) i;
+insert into gist_nan_point_tbl
+ values ('(NaN,NaN)'), ('(NaN,3)'), ('(3,NaN)'), ('(NaN,NaN)');
+create index gist_nan_point_tbl_index on gist_nan_point_tbl using gist (p);
+set enable_seqscan = off;
+set enable_bitmapscan = off;
+select count(*) from gist_nan_point_tbl where p ~= point '(NaN,NaN)';
+select count(*) from gist_nan_point_tbl where p ~= point '(NaN,3)';
+select count(*) from gist_nan_point_tbl where p ~= point '(3,NaN)';
+select count(*) from gist_nan_point_tbl where p ~= point '(3,3)';
+reset enable_seqscan;
+reset enable_bitmapscan;
+drop table gist_nan_point_tbl;
--
2.37.1 (Apple Git-137.1)
[application/octet-stream] v4-0001-Fix-BRIN-box_inclusion_ops-hiding-rows-next-to-a-.patch (13.0K, ../../CAGRkXqQ7q0U3jmitwOjPY2BNBkxWsnyahgoYyMs4H=hwVtPFgw@mail.gmail.com/5-v4-0001-Fix-BRIN-box_inclusion_ops-hiding-rows-next-to-a-.patch)
download | inline diff:
From e00c0b4414e0bfb1d8405e67de510b549fc8deed Mon Sep 17 00:00:00 2001
From: Shihao <zhong950419@gmail.com>
Date: Wed, 23 Sep 2026 21:23:52 -0400
Subject: [PATCH v4 1/2] Fix BRIN box_inclusion_ops hiding rows next to a NaN
box
bound_box() lets a NaN coordinate into the range summary, and no box
operator matches a NaN bound, so BRIN skipped the whole range.
Add a mergeable support function, box_mergeable(), that rejects boxes
with a NaN coordinate. A range holding one is then marked unmergeable
and always scanned. BRIN also checks the first value of a range now,
which it used to copy into the summary unchecked. At scan time, a
summary that is not mergeable even with itself is treated as
unmergeable. So indexes built before this fix return correct results
without a REINDEX.
This needs a catalog change, so it is for master only.
Author: Kirill Reshke <reshkekirill@gmail.com>
Author: Shihao Zhong <zhong950419@gmail.com>
Reported-by: Ke <kehan5800@gmail.com>
Reported-by: Andrey Borodin <x4mmm@yandex-team.ru>
Discussion: https://postgr.es/m/19705-548fda77321e062d@postgresql.org
---
doc/src/sgml/brin.sgml | 6 +-
src/backend/access/brin/brin_inclusion.c | 22 +++++++
src/backend/utils/adt/geo_ops.c | 19 ++++++
src/include/catalog/pg_amproc.dat | 2 +
src/include/catalog/pg_proc.dat | 3 +
src/test/regress/expected/brin.out | 73 ++++++++++++++++++++++++
src/test/regress/expected/geometry.out | 13 +++++
src/test/regress/sql/brin.sql | 45 +++++++++++++++
src/test/regress/sql/geometry.sql | 4 ++
9 files changed, 185 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/brin.sgml b/doc/src/sgml/brin.sgml
index 64fb520db7e..2b3e41056b1 100644
--- a/doc/src/sgml/brin.sgml
+++ b/doc/src/sgml/brin.sgml
@@ -1187,8 +1187,10 @@ typedef struct BrinOpcInfo
<para>
Support function numbers 12 and 14 are provided to support
irregularities of built-in data types. Function number 12
- is used to support network addresses from different families which
- are not mergeable. Function number 14 is used to support
+ is used to support network addresses from different families, and
+ boxes with NaN coordinates, which are not mergeable. A value that is
+ not mergeable even with itself makes its block range always match.
+ Function number 14 is used to support
empty ranges. Function number 13 is an optional but
recommended one, which allows the new value to be checked before
it is passed to the union function. As the BRIN framework can shortcut
diff --git a/src/backend/access/brin/brin_inclusion.c b/src/backend/access/brin/brin_inclusion.c
index 5a2058d9aad..4dddc45f03d 100644
--- a/src/backend/access/brin/brin_inclusion.c
+++ b/src/backend/access/brin/brin_inclusion.c
@@ -191,8 +191,19 @@ brin_inclusion_add_value(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(false);
}
+ /*
+ * A new union is not checked for mergeability below, so check the value
+ * against itself. A box with a NaN coordinate fails this.
+ */
if (new)
+ {
+ finfo = inclusion_get_procinfo(bdesc, attno, PROCNUM_MERGEABLE, true);
+ if (finfo != NULL &&
+ !DatumGetBool(FunctionCall2Coll(finfo, colloid, newval, newval)))
+ column->bv_values[INCLUSION_UNMERGEABLE] = BoolGetDatum(true);
+
PG_RETURN_BOOL(true);
+ }
/* Check if the new value is already contained. */
finfo = inclusion_get_procinfo(bdesc, attno, PROCNUM_CONTAINS, true);
@@ -274,6 +285,17 @@ brin_inclusion_consistent(PG_FUNCTION_ARGS)
subtype = key->sk_subtype;
query = key->sk_argument;
unionval = column->bv_values[INCLUSION_UNION];
+
+ /*
+ * Likewise if the union is not mergeable even with itself. An index
+ * built before the opclass had a mergeable function can hold such a union
+ * without the flag, like a box union with NaN bounds.
+ */
+ finfo = inclusion_get_procinfo(bdesc, attno, PROCNUM_MERGEABLE, true);
+ if (finfo != NULL &&
+ !DatumGetBool(FunctionCall2Coll(finfo, colloid, unionval, unionval)))
+ PG_RETURN_BOOL(true);
+
switch (key->sk_strategy)
{
/*
diff --git a/src/backend/utils/adt/geo_ops.c b/src/backend/utils/adt/geo_ops.c
index 73324b91fe5..66bb58fb7d6 100644
--- a/src/backend/utils/adt/geo_ops.c
+++ b/src/backend/utils/adt/geo_ops.c
@@ -4427,6 +4427,25 @@ boxes_bound_box(PG_FUNCTION_ARGS)
PG_RETURN_BOX_P(container);
}
+/*
+ * Can the two boxes be merged into one bounding box?
+ *
+ * Not if either has a NaN coordinate: the bounding box would have NaN
+ * bounds too, and no box operator matches those. This is the mergeable
+ * support function of BRIN box_inclusion_ops.
+ */
+Datum
+box_mergeable(PG_FUNCTION_ARGS)
+{
+ BOX *box1 = PG_GETARG_BOX_P(0),
+ *box2 = PG_GETARG_BOX_P(1);
+
+ PG_RETURN_BOOL(!(isnan(box1->high.x) || isnan(box1->high.y) ||
+ isnan(box1->low.x) || isnan(box1->low.y) ||
+ isnan(box2->high.x) || isnan(box2->high.y) ||
+ isnan(box2->low.x) || isnan(box2->low.y)));
+}
+
/***********************************************************************
**
diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat
index 4a1efdbc899..db24e42e795 100644
--- a/src/include/catalog/pg_amproc.dat
+++ b/src/include/catalog/pg_amproc.dat
@@ -2033,6 +2033,8 @@
amproc => 'brin_inclusion_union' },
{ amprocfamily => 'brin/box_inclusion_ops', amproclefttype => 'box',
amprocrighttype => 'box', amprocnum => '11', amproc => 'bound_box' },
+{ amprocfamily => 'brin/box_inclusion_ops', amproclefttype => 'box',
+ amprocrighttype => 'box', amprocnum => '12', amproc => 'box_mergeable' },
{ amprocfamily => 'brin/box_inclusion_ops', amproclefttype => 'box',
amprocrighttype => 'box', amprocnum => '13', amproc => 'box_contain' },
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f46427258e3..bbbce58a962 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -2140,6 +2140,9 @@
{ oid => '4067', descr => 'bounding box of two boxes',
proname => 'bound_box', prorettype => 'box', proargtypes => 'box box',
prosrc => 'boxes_bound_box' },
+{ oid => '8400', descr => 'can two boxes be merged into a single summary',
+ proname => 'box_mergeable', prorettype => 'bool', proargtypes => 'box box',
+ prosrc => 'box_mergeable' },
{ oid => '981', descr => 'box diagonal',
proname => 'diagonal', prorettype => 'lseg', proargtypes => 'box',
prosrc => 'box_diagonal' },
diff --git a/src/test/regress/expected/brin.out b/src/test/regress/expected/brin.out
index e1db2280cf9..445efddd8f6 100644
--- a/src/test/regress/expected/brin.out
+++ b/src/test/regress/expected/brin.out
@@ -589,3 +589,76 @@ CREATE INDEX brin_insert_optimization_idx ON brin_insert_optimization USING brin
UPDATE brin_insert_optimization SET a = a;
REINDEX INDEX CONCURRENTLY brin_insert_optimization_idx;
DROP TABLE brin_insert_optimization;
+-- a box with a NaN coordinate must not hide the other rows of its page
+-- range (bug #19705)
+CREATE TABLE brin_box_nan (v box);
+INSERT INTO brin_box_nan SELECT box '(0,0),(1,1)' FROM generate_series(1, 100);
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+CREATE INDEX brin_box_nan_idx ON brin_box_nan USING brin (v);
+SET enable_seqscan = off;
+SELECT count(*) FROM brin_box_nan WHERE v && box '(-2,-2),(2,2)';
+ count
+-------
+ 100
+(1 row)
+
+SELECT count(*) FROM brin_box_nan WHERE v @> point '(0.5,0.5)';
+ count
+-------
+ 100
+(1 row)
+
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(0,0),(1,1)';
+ count
+-------
+ 100
+(1 row)
+
+-- the NaN row is found too: unmergeable ranges are always scanned
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(NaN,NaN),(0,0)';
+ count
+-------
+ 1
+(1 row)
+
+-- also when it is the only value in its range
+TRUNCATE brin_box_nan;
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+REINDEX INDEX brin_box_nan_idx;
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(NaN,NaN),(0,0)';
+ count
+-------
+ 1
+(1 row)
+
+RESET enable_seqscan;
+DROP TABLE brin_box_nan;
+-- An index built before box_inclusion_ops had a mergeable function can
+-- hold a NaN union. Mimic one with an opclass that gets the function
+-- only after the build.
+CREATE OPERATOR FAMILY brin_box_nan_ops USING brin;
+CREATE OPERATOR CLASS brin_box_nan_ops FOR TYPE box USING brin
+ FAMILY brin_box_nan_ops AS
+ OPERATOR 3 &&,
+ FUNCTION 1 brin_inclusion_opcinfo(internal),
+ FUNCTION 2 brin_inclusion_add_value(internal, internal, internal, internal),
+ FUNCTION 3 brin_inclusion_consistent(internal, internal, internal),
+ FUNCTION 4 brin_inclusion_union(internal, internal, internal),
+ FUNCTION 11 bound_box(box, box);
+CREATE TABLE brin_box_nan (v box);
+INSERT INTO brin_box_nan SELECT box '(0,0),(1,1)' FROM generate_series(1, 100);
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+CREATE INDEX brin_box_nan_idx ON brin_box_nan USING brin (v brin_box_nan_ops);
+ALTER OPERATOR FAMILY brin_box_nan_ops USING brin
+ ADD FUNCTION 12 (box, box) box_mergeable(box, box);
+\c -
+SET enable_seqscan = off;
+SELECT count(*) FROM brin_box_nan WHERE v && box '(-2,-2),(2,2)';
+ count
+-------
+ 100
+(1 row)
+
+RESET enable_seqscan;
+DROP TABLE brin_box_nan;
+DROP OPERATOR FAMILY brin_box_nan_ops USING brin;
diff --git a/src/test/regress/expected/geometry.out b/src/test/regress/expected/geometry.out
index 1d168b21cbc..9d03c4718a9 100644
--- a/src/test/regress/expected/geometry.out
+++ b/src/test/regress/expected/geometry.out
@@ -5321,3 +5321,16 @@ SELECT * FROM pg_input_error_info('(1,2),-1', 'circle');
invalid input syntax for type circle: "(1,2),-1" | | | 22P02
(1 row)
+-- box_mergeable() rejects NaN coordinates (BRIN box_inclusion_ops)
+SELECT box_mergeable(box '(0,0),(1,1)', box '(NaN,3),(0,2)');
+ box_mergeable
+---------------
+ f
+(1 row)
+
+SELECT box_mergeable(box '(0,0),(1,1)', box '(2,3),(0,2)');
+ box_mergeable
+---------------
+ t
+(1 row)
+
diff --git a/src/test/regress/sql/brin.sql b/src/test/regress/sql/brin.sql
index 7ea97f47c8d..33bfef7a5e6 100644
--- a/src/test/regress/sql/brin.sql
+++ b/src/test/regress/sql/brin.sql
@@ -534,3 +534,48 @@ CREATE INDEX brin_insert_optimization_idx ON brin_insert_optimization USING brin
UPDATE brin_insert_optimization SET a = a;
REINDEX INDEX CONCURRENTLY brin_insert_optimization_idx;
DROP TABLE brin_insert_optimization;
+
+-- a box with a NaN coordinate must not hide the other rows of its page
+-- range (bug #19705)
+CREATE TABLE brin_box_nan (v box);
+INSERT INTO brin_box_nan SELECT box '(0,0),(1,1)' FROM generate_series(1, 100);
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+CREATE INDEX brin_box_nan_idx ON brin_box_nan USING brin (v);
+SET enable_seqscan = off;
+SELECT count(*) FROM brin_box_nan WHERE v && box '(-2,-2),(2,2)';
+SELECT count(*) FROM brin_box_nan WHERE v @> point '(0.5,0.5)';
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(0,0),(1,1)';
+-- the NaN row is found too: unmergeable ranges are always scanned
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(NaN,NaN),(0,0)';
+-- also when it is the only value in its range
+TRUNCATE brin_box_nan;
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+REINDEX INDEX brin_box_nan_idx;
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(NaN,NaN),(0,0)';
+RESET enable_seqscan;
+DROP TABLE brin_box_nan;
+
+-- An index built before box_inclusion_ops had a mergeable function can
+-- hold a NaN union. Mimic one with an opclass that gets the function
+-- only after the build.
+CREATE OPERATOR FAMILY brin_box_nan_ops USING brin;
+CREATE OPERATOR CLASS brin_box_nan_ops FOR TYPE box USING brin
+ FAMILY brin_box_nan_ops AS
+ OPERATOR 3 &&,
+ FUNCTION 1 brin_inclusion_opcinfo(internal),
+ FUNCTION 2 brin_inclusion_add_value(internal, internal, internal, internal),
+ FUNCTION 3 brin_inclusion_consistent(internal, internal, internal),
+ FUNCTION 4 brin_inclusion_union(internal, internal, internal),
+ FUNCTION 11 bound_box(box, box);
+CREATE TABLE brin_box_nan (v box);
+INSERT INTO brin_box_nan SELECT box '(0,0),(1,1)' FROM generate_series(1, 100);
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+CREATE INDEX brin_box_nan_idx ON brin_box_nan USING brin (v brin_box_nan_ops);
+ALTER OPERATOR FAMILY brin_box_nan_ops USING brin
+ ADD FUNCTION 12 (box, box) box_mergeable(box, box);
+\c -
+SET enable_seqscan = off;
+SELECT count(*) FROM brin_box_nan WHERE v && box '(-2,-2),(2,2)';
+RESET enable_seqscan;
+DROP TABLE brin_box_nan;
+DROP OPERATOR FAMILY brin_box_nan_ops USING brin;
diff --git a/src/test/regress/sql/geometry.sql b/src/test/regress/sql/geometry.sql
index c3ea368da5e..994797c4d24 100644
--- a/src/test/regress/sql/geometry.sql
+++ b/src/test/regress/sql/geometry.sql
@@ -529,3 +529,7 @@ SELECT pg_input_is_valid('(1', 'circle');
SELECT * FROM pg_input_error_info('1,', 'circle');
SELECT pg_input_is_valid('(1,2),-1', 'circle');
SELECT * FROM pg_input_error_info('(1,2),-1', 'circle');
+
+-- box_mergeable() rejects NaN coordinates (BRIN box_inclusion_ops)
+SELECT box_mergeable(box '(0,0),(1,1)', box '(NaN,3),(0,2)');
+SELECT box_mergeable(box '(0,0),(1,1)', box '(2,3),(0,2)');
--
2.37.1 (Apple Git-137.1)
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows
2026-09-19 17:37 BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows PG Bug reporting form <noreply@postgresql.org>
2026-09-21 23:49 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
2026-09-22 12:12 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Kirill Reshke <reshkekirill@gmail.com>
2026-09-22 12:37 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Andrey Borodin <x4mmm@yandex-team.ru>
2026-09-23 03:43 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
2026-09-23 17:51 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Kirill Reshke <reshkekirill@gmail.com>
2026-09-24 02:43 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
@ 2026-09-24 06:22 ` Kirill Reshke <reshkekirill@gmail.com>
2026-09-25 04:29 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
0 siblings, 1 reply; 11+ messages in thread
From: Kirill Reshke @ 2026-09-24 06:22 UTC (permalink / raw)
To: shihao zhong <zhong950419@gmail.com>; +Cc: Andrey Borodin <x4mmm@yandex-team.ru>; kehan5800@gmail.com; pgsql-bugs@lists.postgresql.org
On Thu, 24 Sept 2026 at 07:44, shihao zhong <zhong950419@gmail.com> wrote:
>
> Hi Kirill,
>
> Agreed on all four. v4 attached, split into BRIN and GiST.
>
> 0001 is BRIN, master only. bound_box() is untouched. The consistent
> function also scans a range whose union is not mergeable with itself,
> so old NaN summaries work without REINDEX. Needs a catversion bump.
>
> 0002 is GiST, back to 14. Old internal keys with a NaN match every
> search and get KNN distance zero. Without the distance part, KNN on old
> indexes returned rows out of order. 0002 also fixes point ~= with a NaN
> query, which lost rows even on a fresh index. The leaf check used
> FPeq(), but point_eq() compares exactly when there is a NaN.
>
> The .nocfbot file is BRIN for the back branches, against REL_18. It
> needs no catalog change. Consistent scans a range whose union does not
> contain itself, using the existing contains support function. A NaN box
> fails that, and any sane union passes. Nothing on disk changes, so old
> indexes work without REINDEX after a minor upgrade. The code applies to
> 14 and later, the test hunk needs a small rebase on 14 to 16. This check
> would also work on master, if we want one fix everywhere.
>
> I checked 0001 and 0002 with indexes built by unpatched master, then
> pg_upgraded, no REINDEX. The back branch patch got the same check on
> REL_18, with a minor upgrade.
>
> Thanks,
> Shihao
Hi! v4 good.
I have tested point fix for ~= and it survives an index built by the
old master binary. The REL_18 backport looks good to me.
I see your changes to KNN search:
```
* classification logic to work.
@@ -1465,9 +1552,13 @@ gist_point_distance(PG_FUNCTION_ARGS)
switch (strategyGroup)
{
case PointStrategyNumberGroup:
- distance = computeDistance(GIST_LEAF(entry),
- DatumGetBoxP(entry->key),
- PG_GETARG_POINT_P(1));
+ /* A NaN internal key tells us nothing, see box_has_nan() */
+ if (nan_internal_key(entry))
+ distance = 0.0;
+ else
+ distance = computeDistance(GIST_LEAF(entry),
+ DatumGetBoxP(entry->key),
+ PG_GETARG_POINT_P(1));
break;
default:
```
But I cannot reproduce any wrong results with unpatched binary for
KNN. Something that I tried:
CREATE TABLE knn (p point);
INSERT INTO knn SELECT point(0.001*i, 0.001*i)
FROM generate_series(1,1000) i;
INSERT INTO knn VALUES ('(NaN,NaN)');
INSERT INTO knn SELECT point(50+0.001*i, 50+0.001*i)
FROM generate_series(1,1000) i;
CREATE INDEX knn_idx ON knn USING gist (p);
SET enable_seqscan = off;
SELECT count(*) FROM (SELECT 1 FROM knn
ORDER BY p <-> point '(0,0)' LIMIT 2200) s; -- 2001
SELECT count(*) FROM (SELECT p <-> point '(0,0)' d FROM knn
ORDER BY p <-> point '(0,0)' LIMIT 2200) s
WHERE d::text = 'NaN'; -- 1
and this works without v4 (and with).
Another case (from regress tests):
```
reshke=# CREATE TABLE POINT_TBL(f1 point);
INSERT INTO POINT_TBL(f1) VALUES
('(0.0,0.0)'),
('(-10.0,0.0)'),
('(-3.0,4.0)'),
('(5.1, 34.5)'),
('(-5.0,-12.0)'),
('(1e-300,-1e-300)'), -- To underflow
('(1e+300,Inf)'), -- To overflow
('(Inf,1e+300)'), -- Transposed
(' ( Nan , NaN ) '),
('10.0,10.0');
-- We intentionally don't vacuum point_tbl here; geometry depends on that
reshke=#
SELECT * FROM point_tbl WHERE f1 <@ polygon
'(0,0),(0,100),(100,100),(50,50),(100,0),(0,0)';
f1
------------------
(1e-300,-1e-300)
(0,0)
(10,10)
(5.1,34.5)
(4 rows)
reshke=# set enable_seqscan to true;
SET
reshke=#
SELECT * FROM point_tbl WHERE f1 <@ polygon
'(0,0),(0,100),(100,100),(50,50),(100,0),(0,0)';
f1
------------------
(0,0)
(5.1,34.5)
(1e-300,-1e-300)
(NaN,NaN)
(10,10)
(5 rows)
```
So, v4 keeps point <@ polygon and point <@ circle losing NaN points
that the heap returns. I added simple NaN check to
CircleStrategyNumberGroup/PolygonStrategyNumberGroup cases.
So, attaching v4 as v5 (without changes) and v5-0003 for fix the latter issue
--
Best regards,
Kirill Reshke
Attachments:
[application/octet-stream] v5-0002-Fix-GiST-box-indexes-hiding-rows-next-to-a-NaN-bo.patch (13.8K, ../../CALdSSPhh0w=CkW3xNuRGLu=hCJjcwCcdHDddBt5C2RRHbML8Hg@mail.gmail.com/2-v5-0002-Fix-GiST-box-indexes-hiding-rows-next-to-a-NaN-bo.patch)
download | inline diff:
From 09e12b211a9de3bab76aa1513f122bed0de628cb Mon Sep 17 00:00:00 2001
From: Shihao <zhong950419@gmail.com>
Date: Wed, 23 Sep 2026 21:23:52 -0400
Subject: [PATCH v5 2/3] Fix GiST box indexes hiding rows next to a NaN box
GiST union keys kept NaN coordinates, and no box comparison is true
against a NaN bound. So an internal key covering a NaN box hid its
whole subtree from searches, and KNN scans could return rows from it
too late.
Map NaNs to infinities when building union keys. Internal keys that
still have a NaN, from indexes built before this fix, now match every
search and get distance zero. So those indexes return correct results
without a REINDEX, only less selectively. Also stop internal pages
from pruning ~= when the query has a NaN, since box_same() matches it.
For points, ~= also missed NaN queries at the leaf level, because it
compared with FPeq() while point_eq() compares exactly when there is a
NaN. Make the index agree with point_eq().
Author: Kirill Reshke <reshkekirill@gmail.com>
Author: Shihao Zhong <zhong950419@gmail.com>
Reported-by: Ke <kehan5800@gmail.com>
Reported-by: Andrey Borodin <x4mmm@yandex-team.ru>
Discussion: https://postgr.es/m/19705-548fda77321e062d@postgresql.org
Backpatch-through: 14
---
src/backend/access/gist/gistproc.c | 139 ++++++++++++++++++++++++-----
src/test/regress/expected/gist.out | 73 +++++++++++++++
src/test/regress/sql/gist.sql | 35 ++++++++
3 files changed, 225 insertions(+), 22 deletions(-)
diff --git a/src/backend/access/gist/gistproc.c b/src/backend/access/gist/gistproc.c
index f1044f49d6c..0f405d4450f 100644
--- a/src/backend/access/gist/gistproc.c
+++ b/src/backend/access/gist/gistproc.c
@@ -43,6 +43,45 @@ static bool gist_bbox_zorder_abbrev_abort(int memtupcount, SortSupport ssup);
/* Minimum accepted ratio of split */
#define LIMIT_RATIO 0.3
+/*
+ * Does the box have a NaN coordinate?
+ *
+ * No box comparison is true against a NaN, so a union key with one would
+ * hide everything below it. Union keys map NaNs to infinities instead (see
+ * adjustBox), but indexes built before that was done can still have NaN
+ * internal keys, and searches must descend into those.
+ */
+static inline bool
+box_has_nan(const BOX *box)
+{
+ return isnan(box->high.x) || isnan(box->high.y) ||
+ isnan(box->low.x) || isnan(box->low.y);
+}
+
+/* Is the entry an internal key with a NaN coordinate? */
+static inline bool
+nan_internal_key(const GISTENTRY *entry)
+{
+ return !GIST_LEAF(entry) && box_has_nan(DatumGetBoxP(entry->key));
+}
+
+/* float8_max() and float8_min(), but a NaN gives the matching infinity */
+static inline float8
+bound_max(float8 val1, float8 val2)
+{
+ if (isnan(val1) || isnan(val2))
+ return get_float8_infinity();
+ return float8_max(val1, val2);
+}
+
+static inline float8
+bound_min(float8 val1, float8 val2)
+{
+ if (isnan(val1) || isnan(val2))
+ return -get_float8_infinity();
+ return float8_min(val1, val2);
+}
+
/**************************************************
* Box ops
@@ -126,6 +165,10 @@ gist_box_consistent(PG_FUNCTION_ARGS)
if (DatumGetBoxP(entry->key) == NULL || query == NULL)
PG_RETURN_BOOL(false);
+ /* A NaN internal key tells us nothing, see box_has_nan() */
+ if (nan_internal_key(entry))
+ PG_RETURN_BOOL(true);
+
/*
* if entry is not leaf, use rtree_internal_consistent, else use
* gist_box_leaf_consistent
@@ -141,19 +184,31 @@ gist_box_consistent(PG_FUNCTION_ARGS)
}
/*
- * Increase BOX b to include addon.
+ * Increase BOX b to include addon. A NaN in either box grows b to the
+ * matching infinity.
*/
static void
adjustBox(BOX *b, const BOX *addon)
{
- if (float8_lt(b->high.x, addon->high.x))
- b->high.x = addon->high.x;
- if (float8_gt(b->low.x, addon->low.x))
- b->low.x = addon->low.x;
- if (float8_lt(b->high.y, addon->high.y))
- b->high.y = addon->high.y;
- if (float8_gt(b->low.y, addon->low.y))
- b->low.y = addon->low.y;
+ b->high.x = bound_max(b->high.x, addon->high.x);
+ b->low.x = bound_min(b->low.x, addon->low.x);
+ b->high.y = bound_max(b->high.y, addon->high.y);
+ b->low.y = bound_min(b->low.y, addon->low.y);
+}
+
+/* Copy a BOX, mapping NaNs to infinities. */
+static void
+snapBox(BOX *b, const BOX *box)
+{
+ *b = *box;
+ if (isnan(b->high.x))
+ b->high.x = get_float8_infinity();
+ if (isnan(b->high.y))
+ b->high.y = get_float8_infinity();
+ if (isnan(b->low.x))
+ b->low.x = -get_float8_infinity();
+ if (isnan(b->low.y))
+ b->low.y = -get_float8_infinity();
}
/*
@@ -174,7 +229,7 @@ gist_box_union(PG_FUNCTION_ARGS)
numranges = entryvec->n;
pageunion = palloc_object(BOX);
cur = DatumGetBoxP(entryvec->vector[0].key);
- memcpy(pageunion, cur, sizeof(BOX));
+ snapBox(pageunion, cur);
for (i = 1; i < numranges; i++)
{
@@ -239,7 +294,7 @@ fallbackSplit(GistEntryVector *entryvec, GIST_SPLITVEC *v)
if (unionL == NULL)
{
unionL = palloc_object(BOX);
- *unionL = *cur;
+ snapBox(unionL, cur);
}
else
adjustBox(unionL, cur);
@@ -252,7 +307,7 @@ fallbackSplit(GistEntryVector *entryvec, GIST_SPLITVEC *v)
if (unionR == NULL)
{
unionR = palloc_object(BOX);
- *unionR = *cur;
+ snapBox(unionR, cur);
}
else
adjustBox(unionR, cur);
@@ -526,7 +581,7 @@ gist_box_picksplit(PG_FUNCTION_ARGS)
{
box = DatumGetBoxP(entryvec->vector[i].key);
if (i == FirstOffsetNumber)
- context.boundingBox = *box;
+ snapBox(&context.boundingBox, box);
else
adjustBox(&context.boundingBox, box);
}
@@ -715,7 +770,7 @@ gist_box_picksplit(PG_FUNCTION_ARGS)
if (v->spl_nleft > 0) \
adjustBox(leftBox, box); \
else \
- *leftBox = *(box); \
+ snapBox(leftBox, box); \
v->spl_left[v->spl_nleft++] = off; \
} while(0)
@@ -724,7 +779,7 @@ gist_box_picksplit(PG_FUNCTION_ARGS)
if (v->spl_nright > 0) \
adjustBox(rightBox, box); \
else \
- *rightBox = *(box); \
+ snapBox(rightBox, box); \
v->spl_right[v->spl_nright++] = off; \
} while(0)
@@ -959,6 +1014,13 @@ rtree_internal_consistent(BOX *key, BOX *query, StrategyNumber strategy)
{
bool retval;
+ /*
+ * A NaN query can still match with ~=, since box_same() treats NaNs as
+ * equal, but box_contain() never matches it.
+ */
+ if (strategy == RTSameStrategyNumber && box_has_nan(query))
+ return true;
+
switch (strategy)
{
case RTLeftStrategyNumber:
@@ -1077,6 +1139,10 @@ gist_poly_consistent(PG_FUNCTION_ARGS)
if (DatumGetBoxP(entry->key) == NULL || query == NULL)
PG_RETURN_BOOL(false);
+ /* A NaN internal key tells us nothing, see box_has_nan() */
+ if (nan_internal_key(entry))
+ PG_RETURN_BOOL(true);
+
/*
* Since the operators require recheck anyway, we can just use
* rtree_internal_consistent even at leaf nodes. (This works in part
@@ -1147,6 +1213,10 @@ gist_circle_consistent(PG_FUNCTION_ARGS)
if (DatumGetBoxP(entry->key) == NULL || query == NULL)
PG_RETURN_BOOL(false);
+ /* A NaN internal key tells us nothing, see box_has_nan() */
+ if (nan_internal_key(entry))
+ PG_RETURN_BOOL(true);
+
/*
* Since the operators require recheck anyway, we can just use
* rtree_internal_consistent even at leaf nodes. (This works in part
@@ -1307,7 +1377,20 @@ gist_point_consistent_internal(StrategyNumber strategy,
result = FPlt(key->low.y, query->y);
break;
case RTSameStrategyNumber:
- if (isLeaf)
+
+ /*
+ * point_eq() compares exactly when there is a NaN, so a NaN query
+ * matches points with the same NaN coordinates. Internal keys
+ * cannot tell us where those are, so search everything below.
+ */
+ if (isnan(query->x) || isnan(query->y))
+ {
+ /* key.high must equal key.low, so we can disregard it */
+ result = !isLeaf ||
+ (float8_eq(key->low.x, query->x) &&
+ float8_eq(key->low.y, query->y));
+ }
+ else if (isLeaf)
{
/* key.high must equal key.low, so we can disregard it */
result = (FPeq(key->low.x, query->x) &&
@@ -1345,6 +1428,10 @@ gist_point_consistent(PG_FUNCTION_ARGS)
bool result;
StrategyNumber strategyGroup;
+ /* A NaN internal key tells us nothing, see box_has_nan() */
+ if (nan_internal_key(entry))
+ PG_RETURN_BOOL(true);
+
/*
* We have to remap these strategy numbers to get this klugy
* classification logic to work.
@@ -1465,9 +1552,13 @@ gist_point_distance(PG_FUNCTION_ARGS)
switch (strategyGroup)
{
case PointStrategyNumberGroup:
- distance = computeDistance(GIST_LEAF(entry),
- DatumGetBoxP(entry->key),
- PG_GETARG_POINT_P(1));
+ /* A NaN internal key tells us nothing, see box_has_nan() */
+ if (nan_internal_key(entry))
+ distance = 0.0;
+ else
+ distance = computeDistance(GIST_LEAF(entry),
+ DatumGetBoxP(entry->key),
+ PG_GETARG_POINT_P(1));
break;
default:
elog(ERROR, "unrecognized strategy number: %d", strategy);
@@ -1487,9 +1578,13 @@ gist_bbox_distance(GISTENTRY *entry, Datum query, StrategyNumber strategy)
switch (strategyGroup)
{
case PointStrategyNumberGroup:
- distance = computeDistance(false,
- DatumGetBoxP(entry->key),
- DatumGetPointP(query));
+ /* A NaN internal key tells us nothing, see box_has_nan() */
+ if (nan_internal_key(entry))
+ distance = 0.0;
+ else
+ distance = computeDistance(false,
+ DatumGetBoxP(entry->key),
+ DatumGetPointP(query));
break;
default:
elog(ERROR, "unrecognized strategy number: %d", strategy);
diff --git a/src/test/regress/expected/gist.out b/src/test/regress/expected/gist.out
index ac79f94aa80..174f8b4251f 100644
--- a/src/test/regress/expected/gist.out
+++ b/src/test/regress/expected/gist.out
@@ -463,3 +463,76 @@ create index gist_tbl_box_index on gist_tbl using gist (b);
insert into gist_tbl
select box(point(0.05*i, 0.05*i)) from generate_series(0,10) as i;
drop table gist_tbl;
+-- a box with a NaN coordinate must not poison the union keys (bug #19705)
+create table gist_nan_tbl (b box);
+insert into gist_nan_tbl select box '(1,1),(2,2)' from generate_series(1, 1000);
+insert into gist_nan_tbl values (box '(NaN,NaN),(0,0)');
+create index gist_nan_tbl_index on gist_nan_tbl using gist (b);
+-- also insert rows after the build, to exercise the insertion path
+insert into gist_nan_tbl select box '(3,3),(4,4)' from generate_series(1, 1000);
+insert into gist_nan_tbl values (box '(NaN,NaN),(0,0)');
+set enable_seqscan = off;
+set enable_bitmapscan = off;
+select count(*) from gist_nan_tbl where b && box(point(0,0), point(5,5));
+ count
+-------
+ 2000
+(1 row)
+
+select count(*) from gist_nan_tbl where b <@ box(point(0,0), point(5,5));
+ count
+-------
+ 2000
+(1 row)
+
+select count(*) from gist_nan_tbl where b @> point '(1.5,1.5)';
+ count
+-------
+ 1000
+(1 row)
+
+select count(*) from gist_nan_tbl where b ~= box '(NaN,NaN),(0,0)';
+ count
+-------
+ 2
+(1 row)
+
+reset enable_seqscan;
+reset enable_bitmapscan;
+drop table gist_nan_tbl;
+-- point ~= matches NaN coordinates exactly, the index must agree
+create table gist_nan_point_tbl (p point);
+insert into gist_nan_point_tbl
+ select point(i % 100, i / 100) from generate_series(0, 9999) i;
+insert into gist_nan_point_tbl
+ values ('(NaN,NaN)'), ('(NaN,3)'), ('(3,NaN)'), ('(NaN,NaN)');
+create index gist_nan_point_tbl_index on gist_nan_point_tbl using gist (p);
+set enable_seqscan = off;
+set enable_bitmapscan = off;
+select count(*) from gist_nan_point_tbl where p ~= point '(NaN,NaN)';
+ count
+-------
+ 2
+(1 row)
+
+select count(*) from gist_nan_point_tbl where p ~= point '(NaN,3)';
+ count
+-------
+ 1
+(1 row)
+
+select count(*) from gist_nan_point_tbl where p ~= point '(3,NaN)';
+ count
+-------
+ 1
+(1 row)
+
+select count(*) from gist_nan_point_tbl where p ~= point '(3,3)';
+ count
+-------
+ 1
+(1 row)
+
+reset enable_seqscan;
+reset enable_bitmapscan;
+drop table gist_nan_point_tbl;
diff --git a/src/test/regress/sql/gist.sql b/src/test/regress/sql/gist.sql
index 57dcc082450..654318fb4db 100644
--- a/src/test/regress/sql/gist.sql
+++ b/src/test/regress/sql/gist.sql
@@ -236,3 +236,38 @@ create index gist_tbl_box_index on gist_tbl using gist (b);
insert into gist_tbl
select box(point(0.05*i, 0.05*i)) from generate_series(0,10) as i;
drop table gist_tbl;
+
+-- a box with a NaN coordinate must not poison the union keys (bug #19705)
+create table gist_nan_tbl (b box);
+insert into gist_nan_tbl select box '(1,1),(2,2)' from generate_series(1, 1000);
+insert into gist_nan_tbl values (box '(NaN,NaN),(0,0)');
+create index gist_nan_tbl_index on gist_nan_tbl using gist (b);
+-- also insert rows after the build, to exercise the insertion path
+insert into gist_nan_tbl select box '(3,3),(4,4)' from generate_series(1, 1000);
+insert into gist_nan_tbl values (box '(NaN,NaN),(0,0)');
+set enable_seqscan = off;
+set enable_bitmapscan = off;
+select count(*) from gist_nan_tbl where b && box(point(0,0), point(5,5));
+select count(*) from gist_nan_tbl where b <@ box(point(0,0), point(5,5));
+select count(*) from gist_nan_tbl where b @> point '(1.5,1.5)';
+select count(*) from gist_nan_tbl where b ~= box '(NaN,NaN),(0,0)';
+reset enable_seqscan;
+reset enable_bitmapscan;
+drop table gist_nan_tbl;
+
+-- point ~= matches NaN coordinates exactly, the index must agree
+create table gist_nan_point_tbl (p point);
+insert into gist_nan_point_tbl
+ select point(i % 100, i / 100) from generate_series(0, 9999) i;
+insert into gist_nan_point_tbl
+ values ('(NaN,NaN)'), ('(NaN,3)'), ('(3,NaN)'), ('(NaN,NaN)');
+create index gist_nan_point_tbl_index on gist_nan_point_tbl using gist (p);
+set enable_seqscan = off;
+set enable_bitmapscan = off;
+select count(*) from gist_nan_point_tbl where p ~= point '(NaN,NaN)';
+select count(*) from gist_nan_point_tbl where p ~= point '(NaN,3)';
+select count(*) from gist_nan_point_tbl where p ~= point '(3,NaN)';
+select count(*) from gist_nan_point_tbl where p ~= point '(3,3)';
+reset enable_seqscan;
+reset enable_bitmapscan;
+drop table gist_nan_point_tbl;
--
2.43.0
[application/octet-stream] v5-REL_18-0001-Fix-BRIN-box_inclusion_ops-hiding-rows-next-to-a-.patch.nocfbot (5.6K, ../../CALdSSPhh0w=CkW3xNuRGLu=hCJjcwCcdHDddBt5C2RRHbML8Hg@mail.gmail.com/3-v5-REL_18-0001-Fix-BRIN-box_inclusion_ops-hiding-rows-next-to-a-.patch.nocfbot)
download
[application/octet-stream] v5-0001-Fix-BRIN-box_inclusion_ops-hiding-rows-next-to-a-.patch (12.9K, ../../CALdSSPhh0w=CkW3xNuRGLu=hCJjcwCcdHDddBt5C2RRHbML8Hg@mail.gmail.com/4-v5-0001-Fix-BRIN-box_inclusion_ops-hiding-rows-next-to-a-.patch)
download | inline diff:
From 02bcf66049be38124da4027da736690e2a709dd8 Mon Sep 17 00:00:00 2001
From: Shihao <zhong950419@gmail.com>
Date: Wed, 23 Sep 2026 21:23:52 -0400
Subject: [PATCH v5 1/3] Fix BRIN box_inclusion_ops hiding rows next to a NaN
box
bound_box() lets a NaN coordinate into the range summary, and no box
operator matches a NaN bound, so BRIN skipped the whole range.
Add a mergeable support function, box_mergeable(), that rejects boxes
with a NaN coordinate. A range holding one is then marked unmergeable
and always scanned. BRIN also checks the first value of a range now,
which it used to copy into the summary unchecked. At scan time, a
summary that is not mergeable even with itself is treated as
unmergeable. So indexes built before this fix return correct results
without a REINDEX.
This needs a catalog change, so it is for master only.
Author: Kirill Reshke <reshkekirill@gmail.com>
Author: Shihao Zhong <zhong950419@gmail.com>
Reported-by: Ke <kehan5800@gmail.com>
Reported-by: Andrey Borodin <x4mmm@yandex-team.ru>
Discussion: https://postgr.es/m/19705-548fda77321e062d@postgresql.org
---
doc/src/sgml/brin.sgml | 6 +-
src/backend/access/brin/brin_inclusion.c | 22 +++++++
src/backend/utils/adt/geo_ops.c | 19 ++++++
src/include/catalog/pg_amproc.dat | 2 +
src/include/catalog/pg_proc.dat | 3 +
src/test/regress/expected/brin.out | 73 ++++++++++++++++++++++++
src/test/regress/expected/geometry.out | 13 +++++
src/test/regress/sql/brin.sql | 45 +++++++++++++++
src/test/regress/sql/geometry.sql | 4 ++
9 files changed, 185 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/brin.sgml b/doc/src/sgml/brin.sgml
index 64fb520db7e..2b3e41056b1 100644
--- a/doc/src/sgml/brin.sgml
+++ b/doc/src/sgml/brin.sgml
@@ -1187,8 +1187,10 @@ typedef struct BrinOpcInfo
<para>
Support function numbers 12 and 14 are provided to support
irregularities of built-in data types. Function number 12
- is used to support network addresses from different families which
- are not mergeable. Function number 14 is used to support
+ is used to support network addresses from different families, and
+ boxes with NaN coordinates, which are not mergeable. A value that is
+ not mergeable even with itself makes its block range always match.
+ Function number 14 is used to support
empty ranges. Function number 13 is an optional but
recommended one, which allows the new value to be checked before
it is passed to the union function. As the BRIN framework can shortcut
diff --git a/src/backend/access/brin/brin_inclusion.c b/src/backend/access/brin/brin_inclusion.c
index 5a2058d9aad..4dddc45f03d 100644
--- a/src/backend/access/brin/brin_inclusion.c
+++ b/src/backend/access/brin/brin_inclusion.c
@@ -191,8 +191,19 @@ brin_inclusion_add_value(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(false);
}
+ /*
+ * A new union is not checked for mergeability below, so check the value
+ * against itself. A box with a NaN coordinate fails this.
+ */
if (new)
+ {
+ finfo = inclusion_get_procinfo(bdesc, attno, PROCNUM_MERGEABLE, true);
+ if (finfo != NULL &&
+ !DatumGetBool(FunctionCall2Coll(finfo, colloid, newval, newval)))
+ column->bv_values[INCLUSION_UNMERGEABLE] = BoolGetDatum(true);
+
PG_RETURN_BOOL(true);
+ }
/* Check if the new value is already contained. */
finfo = inclusion_get_procinfo(bdesc, attno, PROCNUM_CONTAINS, true);
@@ -274,6 +285,17 @@ brin_inclusion_consistent(PG_FUNCTION_ARGS)
subtype = key->sk_subtype;
query = key->sk_argument;
unionval = column->bv_values[INCLUSION_UNION];
+
+ /*
+ * Likewise if the union is not mergeable even with itself. An index
+ * built before the opclass had a mergeable function can hold such a union
+ * without the flag, like a box union with NaN bounds.
+ */
+ finfo = inclusion_get_procinfo(bdesc, attno, PROCNUM_MERGEABLE, true);
+ if (finfo != NULL &&
+ !DatumGetBool(FunctionCall2Coll(finfo, colloid, unionval, unionval)))
+ PG_RETURN_BOOL(true);
+
switch (key->sk_strategy)
{
/*
diff --git a/src/backend/utils/adt/geo_ops.c b/src/backend/utils/adt/geo_ops.c
index 73324b91fe5..66bb58fb7d6 100644
--- a/src/backend/utils/adt/geo_ops.c
+++ b/src/backend/utils/adt/geo_ops.c
@@ -4427,6 +4427,25 @@ boxes_bound_box(PG_FUNCTION_ARGS)
PG_RETURN_BOX_P(container);
}
+/*
+ * Can the two boxes be merged into one bounding box?
+ *
+ * Not if either has a NaN coordinate: the bounding box would have NaN
+ * bounds too, and no box operator matches those. This is the mergeable
+ * support function of BRIN box_inclusion_ops.
+ */
+Datum
+box_mergeable(PG_FUNCTION_ARGS)
+{
+ BOX *box1 = PG_GETARG_BOX_P(0),
+ *box2 = PG_GETARG_BOX_P(1);
+
+ PG_RETURN_BOOL(!(isnan(box1->high.x) || isnan(box1->high.y) ||
+ isnan(box1->low.x) || isnan(box1->low.y) ||
+ isnan(box2->high.x) || isnan(box2->high.y) ||
+ isnan(box2->low.x) || isnan(box2->low.y)));
+}
+
/***********************************************************************
**
diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat
index 4a1efdbc899..db24e42e795 100644
--- a/src/include/catalog/pg_amproc.dat
+++ b/src/include/catalog/pg_amproc.dat
@@ -2033,6 +2033,8 @@
amproc => 'brin_inclusion_union' },
{ amprocfamily => 'brin/box_inclusion_ops', amproclefttype => 'box',
amprocrighttype => 'box', amprocnum => '11', amproc => 'bound_box' },
+{ amprocfamily => 'brin/box_inclusion_ops', amproclefttype => 'box',
+ amprocrighttype => 'box', amprocnum => '12', amproc => 'box_mergeable' },
{ amprocfamily => 'brin/box_inclusion_ops', amproclefttype => 'box',
amprocrighttype => 'box', amprocnum => '13', amproc => 'box_contain' },
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f46427258e3..bbbce58a962 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -2140,6 +2140,9 @@
{ oid => '4067', descr => 'bounding box of two boxes',
proname => 'bound_box', prorettype => 'box', proargtypes => 'box box',
prosrc => 'boxes_bound_box' },
+{ oid => '8400', descr => 'can two boxes be merged into a single summary',
+ proname => 'box_mergeable', prorettype => 'bool', proargtypes => 'box box',
+ prosrc => 'box_mergeable' },
{ oid => '981', descr => 'box diagonal',
proname => 'diagonal', prorettype => 'lseg', proargtypes => 'box',
prosrc => 'box_diagonal' },
diff --git a/src/test/regress/expected/brin.out b/src/test/regress/expected/brin.out
index e1db2280cf9..445efddd8f6 100644
--- a/src/test/regress/expected/brin.out
+++ b/src/test/regress/expected/brin.out
@@ -589,3 +589,76 @@ CREATE INDEX brin_insert_optimization_idx ON brin_insert_optimization USING brin
UPDATE brin_insert_optimization SET a = a;
REINDEX INDEX CONCURRENTLY brin_insert_optimization_idx;
DROP TABLE brin_insert_optimization;
+-- a box with a NaN coordinate must not hide the other rows of its page
+-- range (bug #19705)
+CREATE TABLE brin_box_nan (v box);
+INSERT INTO brin_box_nan SELECT box '(0,0),(1,1)' FROM generate_series(1, 100);
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+CREATE INDEX brin_box_nan_idx ON brin_box_nan USING brin (v);
+SET enable_seqscan = off;
+SELECT count(*) FROM brin_box_nan WHERE v && box '(-2,-2),(2,2)';
+ count
+-------
+ 100
+(1 row)
+
+SELECT count(*) FROM brin_box_nan WHERE v @> point '(0.5,0.5)';
+ count
+-------
+ 100
+(1 row)
+
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(0,0),(1,1)';
+ count
+-------
+ 100
+(1 row)
+
+-- the NaN row is found too: unmergeable ranges are always scanned
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(NaN,NaN),(0,0)';
+ count
+-------
+ 1
+(1 row)
+
+-- also when it is the only value in its range
+TRUNCATE brin_box_nan;
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+REINDEX INDEX brin_box_nan_idx;
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(NaN,NaN),(0,0)';
+ count
+-------
+ 1
+(1 row)
+
+RESET enable_seqscan;
+DROP TABLE brin_box_nan;
+-- An index built before box_inclusion_ops had a mergeable function can
+-- hold a NaN union. Mimic one with an opclass that gets the function
+-- only after the build.
+CREATE OPERATOR FAMILY brin_box_nan_ops USING brin;
+CREATE OPERATOR CLASS brin_box_nan_ops FOR TYPE box USING brin
+ FAMILY brin_box_nan_ops AS
+ OPERATOR 3 &&,
+ FUNCTION 1 brin_inclusion_opcinfo(internal),
+ FUNCTION 2 brin_inclusion_add_value(internal, internal, internal, internal),
+ FUNCTION 3 brin_inclusion_consistent(internal, internal, internal),
+ FUNCTION 4 brin_inclusion_union(internal, internal, internal),
+ FUNCTION 11 bound_box(box, box);
+CREATE TABLE brin_box_nan (v box);
+INSERT INTO brin_box_nan SELECT box '(0,0),(1,1)' FROM generate_series(1, 100);
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+CREATE INDEX brin_box_nan_idx ON brin_box_nan USING brin (v brin_box_nan_ops);
+ALTER OPERATOR FAMILY brin_box_nan_ops USING brin
+ ADD FUNCTION 12 (box, box) box_mergeable(box, box);
+\c -
+SET enable_seqscan = off;
+SELECT count(*) FROM brin_box_nan WHERE v && box '(-2,-2),(2,2)';
+ count
+-------
+ 100
+(1 row)
+
+RESET enable_seqscan;
+DROP TABLE brin_box_nan;
+DROP OPERATOR FAMILY brin_box_nan_ops USING brin;
diff --git a/src/test/regress/expected/geometry.out b/src/test/regress/expected/geometry.out
index 1d168b21cbc..9d03c4718a9 100644
--- a/src/test/regress/expected/geometry.out
+++ b/src/test/regress/expected/geometry.out
@@ -5321,3 +5321,16 @@ SELECT * FROM pg_input_error_info('(1,2),-1', 'circle');
invalid input syntax for type circle: "(1,2),-1" | | | 22P02
(1 row)
+-- box_mergeable() rejects NaN coordinates (BRIN box_inclusion_ops)
+SELECT box_mergeable(box '(0,0),(1,1)', box '(NaN,3),(0,2)');
+ box_mergeable
+---------------
+ f
+(1 row)
+
+SELECT box_mergeable(box '(0,0),(1,1)', box '(2,3),(0,2)');
+ box_mergeable
+---------------
+ t
+(1 row)
+
diff --git a/src/test/regress/sql/brin.sql b/src/test/regress/sql/brin.sql
index 7ea97f47c8d..33bfef7a5e6 100644
--- a/src/test/regress/sql/brin.sql
+++ b/src/test/regress/sql/brin.sql
@@ -534,3 +534,48 @@ CREATE INDEX brin_insert_optimization_idx ON brin_insert_optimization USING brin
UPDATE brin_insert_optimization SET a = a;
REINDEX INDEX CONCURRENTLY brin_insert_optimization_idx;
DROP TABLE brin_insert_optimization;
+
+-- a box with a NaN coordinate must not hide the other rows of its page
+-- range (bug #19705)
+CREATE TABLE brin_box_nan (v box);
+INSERT INTO brin_box_nan SELECT box '(0,0),(1,1)' FROM generate_series(1, 100);
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+CREATE INDEX brin_box_nan_idx ON brin_box_nan USING brin (v);
+SET enable_seqscan = off;
+SELECT count(*) FROM brin_box_nan WHERE v && box '(-2,-2),(2,2)';
+SELECT count(*) FROM brin_box_nan WHERE v @> point '(0.5,0.5)';
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(0,0),(1,1)';
+-- the NaN row is found too: unmergeable ranges are always scanned
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(NaN,NaN),(0,0)';
+-- also when it is the only value in its range
+TRUNCATE brin_box_nan;
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+REINDEX INDEX brin_box_nan_idx;
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(NaN,NaN),(0,0)';
+RESET enable_seqscan;
+DROP TABLE brin_box_nan;
+
+-- An index built before box_inclusion_ops had a mergeable function can
+-- hold a NaN union. Mimic one with an opclass that gets the function
+-- only after the build.
+CREATE OPERATOR FAMILY brin_box_nan_ops USING brin;
+CREATE OPERATOR CLASS brin_box_nan_ops FOR TYPE box USING brin
+ FAMILY brin_box_nan_ops AS
+ OPERATOR 3 &&,
+ FUNCTION 1 brin_inclusion_opcinfo(internal),
+ FUNCTION 2 brin_inclusion_add_value(internal, internal, internal, internal),
+ FUNCTION 3 brin_inclusion_consistent(internal, internal, internal),
+ FUNCTION 4 brin_inclusion_union(internal, internal, internal),
+ FUNCTION 11 bound_box(box, box);
+CREATE TABLE brin_box_nan (v box);
+INSERT INTO brin_box_nan SELECT box '(0,0),(1,1)' FROM generate_series(1, 100);
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+CREATE INDEX brin_box_nan_idx ON brin_box_nan USING brin (v brin_box_nan_ops);
+ALTER OPERATOR FAMILY brin_box_nan_ops USING brin
+ ADD FUNCTION 12 (box, box) box_mergeable(box, box);
+\c -
+SET enable_seqscan = off;
+SELECT count(*) FROM brin_box_nan WHERE v && box '(-2,-2),(2,2)';
+RESET enable_seqscan;
+DROP TABLE brin_box_nan;
+DROP OPERATOR FAMILY brin_box_nan_ops USING brin;
diff --git a/src/test/regress/sql/geometry.sql b/src/test/regress/sql/geometry.sql
index c3ea368da5e..994797c4d24 100644
--- a/src/test/regress/sql/geometry.sql
+++ b/src/test/regress/sql/geometry.sql
@@ -529,3 +529,7 @@ SELECT pg_input_is_valid('(1', 'circle');
SELECT * FROM pg_input_error_info('1,', 'circle');
SELECT pg_input_is_valid('(1,2),-1', 'circle');
SELECT * FROM pg_input_error_info('(1,2),-1', 'circle');
+
+-- box_mergeable() rejects NaN coordinates (BRIN box_inclusion_ops)
+SELECT box_mergeable(box '(0,0),(1,1)', box '(NaN,3),(0,2)');
+SELECT box_mergeable(box '(0,0),(1,1)', box '(2,3),(0,2)');
--
2.43.0
[application/octet-stream] v5-0001-Check-for-NaN-point-in-GiST-polygon-and-circle-se.patch (6.2K, ../../CALdSSPhh0w=CkW3xNuRGLu=hCJjcwCcdHDddBt5C2RRHbML8Hg@mail.gmail.com/5-v5-0001-Check-for-NaN-point-in-GiST-polygon-and-circle-se.patch)
download | inline diff:
From f88fbf5c894e88da85efc11683732b543b022c62 Mon Sep 17 00:00:00 2001
From: reshke <reshke@double.cloud>
Date: Thu, 24 Sep 2026 08:10:07 +0300
Subject: [PATCH v5] Check for NaN point in GiST polygon and circle searches.
The polygon/circle strategy groups of GiST consistent function check
leaf points with its bounding box and fast-filter check.
Points with NaN coordinate always fails, so such points were not returned by index
serach while poly_contain_pt/circle_contain_pt would actaully match them.
This was actaully the case in regression test - "SELECT count(*) FROM point_tbl WHERE f1
<@ polygon ..." in create_index.sql has answered 5 all along while the
index run answered 4.
Also asserts has been updated with NaN check.
Per BUG 19705
Discussion: https://postgr.es/m/19705-548fda77321e062d@postgresql.org
---
src/backend/access/gist/gistproc.c | 42 ++++++++++++++--------
src/test/regress/expected/create_index.out | 2 +-
src/test/regress/expected/gist.out | 14 ++++++++
src/test/regress/sql/gist.sql | 4 +++
4 files changed, 47 insertions(+), 15 deletions(-)
diff --git a/src/backend/access/gist/gistproc.c b/src/backend/access/gist/gistproc.c
index 0f405d4450f..f0c598e1bda 100644
--- a/src/backend/access/gist/gistproc.c
+++ b/src/backend/access/gist/gistproc.c
@@ -1482,11 +1482,16 @@ gist_point_consistent(PG_FUNCTION_ARGS)
{
POLYGON *query = PG_GETARG_POLYGON_P(1);
- result = DatumGetBool(DirectFunctionCall5(gist_poly_consistent,
- PointerGetDatum(entry),
- PolygonPGetDatum(query),
- Int16GetDatum(RTOverlapStrategyNumber),
- 0, PointerGetDatum(recheck)));
+ /*
+ * A NaN point fails the bounding-box prefilter, though the
+ * exact operator below may still match it, as on the heap.
+ */
+ result = box_has_nan(DatumGetBoxP(entry->key)) ||
+ DatumGetBool(DirectFunctionCall5(gist_poly_consistent,
+ PointerGetDatum(entry),
+ PolygonPGetDatum(query),
+ Int16GetDatum(RTOverlapStrategyNumber),
+ 0, PointerGetDatum(recheck)));
if (GIST_LEAF(entry) && result)
{
@@ -1496,8 +1501,10 @@ gist_point_consistent(PG_FUNCTION_ARGS)
*/
BOX *box = DatumGetBoxP(entry->key);
- Assert(box->high.x == box->low.x
- && box->high.y == box->low.y);
+ Assert((box->high.x == box->low.x ||
+ (isnan(box->high.x) && isnan(box->low.x))) &&
+ (box->high.y == box->low.y ||
+ (isnan(box->high.y) && isnan(box->low.y))));
result = DatumGetBool(DirectFunctionCall2(poly_contain_pt,
PolygonPGetDatum(query),
PointPGetDatum(&box->high)));
@@ -1509,11 +1516,16 @@ gist_point_consistent(PG_FUNCTION_ARGS)
{
CIRCLE *query = PG_GETARG_CIRCLE_P(1);
- result = DatumGetBool(DirectFunctionCall5(gist_circle_consistent,
- PointerGetDatum(entry),
- CirclePGetDatum(query),
- Int16GetDatum(RTOverlapStrategyNumber),
- 0, PointerGetDatum(recheck)));
+ /*
+ * A NaN point fails the bounding-box prefilter, though the
+ * exact operator below may still match it, as on the heap.
+ */
+ result = box_has_nan(DatumGetBoxP(entry->key)) ||
+ DatumGetBool(DirectFunctionCall5(gist_circle_consistent,
+ PointerGetDatum(entry),
+ CirclePGetDatum(query),
+ Int16GetDatum(RTOverlapStrategyNumber),
+ 0, PointerGetDatum(recheck)));
if (GIST_LEAF(entry) && result)
{
@@ -1523,8 +1535,10 @@ gist_point_consistent(PG_FUNCTION_ARGS)
*/
BOX *box = DatumGetBoxP(entry->key);
- Assert(box->high.x == box->low.x
- && box->high.y == box->low.y);
+ Assert((box->high.x == box->low.x ||
+ (isnan(box->high.x) && isnan(box->low.x))) &&
+ (box->high.y == box->low.y ||
+ (isnan(box->high.y) && isnan(box->low.y))));
result = DatumGetBool(DirectFunctionCall2(circle_contain_pt,
CirclePGetDatum(query),
PointPGetDatum(&box->high)));
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 7b2640f0e04..272b679bbac 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -380,7 +380,7 @@ SELECT count(*) FROM point_tbl WHERE f1 <@ polygon '(0,0),(0,100),(100,100),(50,
SELECT count(*) FROM point_tbl WHERE f1 <@ polygon '(0,0),(0,100),(100,100),(50,50),(100,0),(0,0)';
count
-------
- 4
+ 5
(1 row)
EXPLAIN (COSTS OFF)
diff --git a/src/test/regress/expected/gist.out b/src/test/regress/expected/gist.out
index 174f8b4251f..e7f1a27f866 100644
--- a/src/test/regress/expected/gist.out
+++ b/src/test/regress/expected/gist.out
@@ -533,6 +533,20 @@ select count(*) from gist_nan_point_tbl where p ~= point '(3,3)';
1
(1 row)
+-- a NaN point fails the bounding-box prefilter of the polygon and circle
+-- contains strategies, so the exact operator decides, as on the heap
+select count(*) from gist_nan_point_tbl where p <@ polygon '(0,0),(0,100),(100,100),(50,50),(100,0),(0,0)';
+ count
+-------
+ 7602
+(1 row)
+
+select count(*) from gist_nan_point_tbl where p <@ circle '<(50,50),50>';
+ count
+-------
+ 7843
+(1 row)
+
reset enable_seqscan;
reset enable_bitmapscan;
drop table gist_nan_point_tbl;
diff --git a/src/test/regress/sql/gist.sql b/src/test/regress/sql/gist.sql
index 654318fb4db..b9df6490ca5 100644
--- a/src/test/regress/sql/gist.sql
+++ b/src/test/regress/sql/gist.sql
@@ -268,6 +268,10 @@ select count(*) from gist_nan_point_tbl where p ~= point '(NaN,NaN)';
select count(*) from gist_nan_point_tbl where p ~= point '(NaN,3)';
select count(*) from gist_nan_point_tbl where p ~= point '(3,NaN)';
select count(*) from gist_nan_point_tbl where p ~= point '(3,3)';
+-- a NaN point fails the bounding-box prefilter of the polygon and circle
+-- contains strategies, so the exact operator decides, as on the heap
+select count(*) from gist_nan_point_tbl where p <@ polygon '(0,0),(0,100),(100,100),(50,50),(100,0),(0,0)';
+select count(*) from gist_nan_point_tbl where p <@ circle '<(50,50),50>';
reset enable_seqscan;
reset enable_bitmapscan;
drop table gist_nan_point_tbl;
--
2.43.0
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows
2026-09-19 17:37 BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows PG Bug reporting form <noreply@postgresql.org>
2026-09-21 23:49 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
2026-09-22 12:12 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Kirill Reshke <reshkekirill@gmail.com>
2026-09-22 12:37 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Andrey Borodin <x4mmm@yandex-team.ru>
2026-09-23 03:43 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
2026-09-23 17:51 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Kirill Reshke <reshkekirill@gmail.com>
2026-09-24 02:43 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
2026-09-24 06:22 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Kirill Reshke <reshkekirill@gmail.com>
@ 2026-09-25 04:29 ` shihao zhong <zhong950419@gmail.com>
2026-09-26 00:09 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Manu <manuelreyesbravo@gmail.com>
0 siblings, 1 reply; 11+ messages in thread
From: shihao zhong @ 2026-09-25 04:29 UTC (permalink / raw)
To: Kirill Reshke <reshkekirill@gmail.com>; +Cc: Andrey Borodin <x4mmm@yandex-team.ru>; kehan5800@gmail.com; pgsql-bugs@lists.postgresql.org
Hi Kirill,
> But I cannot reproduce any wrong results with unpatched binary for
> KNN.
It does fail on master. Your points lie on one line and the query is at
the origin, so the low corner is always the closest point of a box. Try
a grid:
CREATE TABLE knn (p point);
INSERT INTO knn SELECT point(x, y)
FROM generate_series(0, 99) x, generate_series(0, 99) y;
INSERT INTO knn VALUES ('(NaN,NaN)');
CREATE INDEX ON knn USING gist (p);
SET enable_seqscan = off;
SELECT p FROM knn ORDER BY p <-> point '(99,99)' LIMIT 1;
Master returns (92,99), v5 returns (99,99). With a NaN high corner,
computeDistance() falls to the vertex case and returns the distance to
the low corner. That is too big for a query point inside the box. The
new union code alone fixes fresh indexes. Without the distance hunk, an
index built by master still gives the same wrong answers.
> So, v4 keeps point <@ polygon and point <@ circle losing NaN points
> that the heap returns.
0003 looks right to me, it just needs pgindent. Your circle test passes
without the circle hunk though, its failure on master comes from 0002.
A NaN point only matches a circle when it also has an infinity and the
radius is infinite.
The root of the polygon case is point_inside(), which puts (NaN,NaN)
inside every polygon, even '((0,0))'. Changing that changes query
results, so I'd leave it for a separate master only patch.
Thanks,
Shihao
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows
2026-09-19 17:37 BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows PG Bug reporting form <noreply@postgresql.org>
2026-09-21 23:49 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
2026-09-22 12:12 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Kirill Reshke <reshkekirill@gmail.com>
2026-09-22 12:37 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Andrey Borodin <x4mmm@yandex-team.ru>
2026-09-23 03:43 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
2026-09-23 17:51 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Kirill Reshke <reshkekirill@gmail.com>
2026-09-24 02:43 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
2026-09-24 06:22 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Kirill Reshke <reshkekirill@gmail.com>
2026-09-25 04:29 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
@ 2026-09-26 00:09 ` Manu <manuelreyesbravo@gmail.com>
2026-09-28 07:01 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
0 siblings, 1 reply; 11+ messages in thread
From: Manu @ 2026-09-26 00:09 UTC (permalink / raw)
To: shihao zhong <zhong950419@gmail.com>; +Cc: Kirill Reshke <reshkekirill@gmail.com>; Andrey Borodin <x4mmm@yandex-team.ru>; kehan5800@gmail.com; pgsql-bugs@lists.postgresql.org
Hi,
Since the cases keep turning up one by one, I ran a differential check
of v5 (0001, 0002 and the NaN point check for polygon and circle) on
master and REL_18_STABLE. For every operator pg_amop lists for the
BRIN, GiST and SP-GiST box, point, polygon and circle opclasses, with
NaN rows first, last, alone and scattered, it compares a seq scan with
the index: count(*) for searches, the first 50 distances for ordering.
6769 checks per build, built without assertions.
BRIN: 79 mismatches without the patches, none with them, on master and
on REL_18. None either with indexes built by unpatched REL_18 and
searched after the minor upgrade.
GiST on master: 376 mismatches without the patches, 91 with them. Most of what
is left is ordering: the index scan fails with "inconsistent point
values" (with assertions, Assert(box->low.y <= box->high.y) fails in
computeDistance()), with and without v5, on both branches:
CREATE TABLE b (v box);
INSERT INTO b VALUES ('(1,NaN),(0,0)');
INSERT INTO b SELECT box(point(x, y), point(x + 1, y + 1))
FROM generate_series(0, 44) x, generate_series(0, 44) y;
CREATE INDEX ON b USING gist (v);
SET enable_seqscan = off;
SELECT v <-> point '(0.5,0.5)' FROM b
ORDER BY v <-> point '(0.5,0.5)' LIMIT 3;
The seq scan returns 0, 0.5, 0.5. Circle and polygon fail the same
way with a finite query point, and all four types with a NaN in the
query point. Which LIMIT reaches the NaN entry depends on where it
lands in the tree, so v5 changes which queries fail but not that they
do. Besides that, box ordering returns NaN distances before finite
ones, and polygon ordering also gets "index returned tuples in wrong
order", with finite query points as well. The rest is point <@
polygon with a NaN in the polygon, which you already set aside.
v5-0002 does not apply to REL_18 as is: gist_box_picksplit() uses
palloc_object(), which 18 does not have. With palloc(sizeof(BOX))
there it applies, and REL_18 gives the same counts as master.
Outside this thread: SP-GiST box_ops and poly_ops show similar ordering
and ~= mismatches, unchanged by v5. quad_point_ops fails CREATE
INDEX with NaN points with "getQuadrant: impossible case", the error of
bug #19597, although that report reaches it through rounding, not NaN.
I have not checked whether the patch there covers NaN too.
The counts, the full list and the scripts are attached.
Regards,
Manu
Differential check of the v5 series for bug #19705 (Kirill's v5-0001 BRIN, v5-0002 GiST,
and the NaN point check for GiST polygon/circle searches), all built without assertions.
master 87762cbaa4f; REL_18_STABLE e1b2a86d6e7 with v5-REL_18-0001 + v5-0002 + the NaN point
patch (code only; v5-0002 needs the rebase below).
For each index (one per table), NaN value and position of the NaN rows, every operator the
opclass lists in pg_amop is run with a seq scan and with the index (EXPLAIN checked that the
index was used in every case). Search operators compare count(*), ordering operators the
first 50 distances. A mismatch is any difference, including an ERROR on one side only.
===== m0nc (index | mismatches | checks)
checks 6769, mismatches 493, index not used 0
brin box_inclusion_ops|79|1092
gist box_ops|87|1092
gist circle_ops|58|804
gist point_ops|162|864
gist poly_ops|69|440
spgist box_ops|10|1092
spgist kd_point_ops|0|756
spgist poly_ops|28|440
spgist quad_point_ops|0|189
===== m5nc (index | mismatches | checks)
checks 6769, mismatches 129, index not used 0
brin box_inclusion_ops|0|1092
gist box_ops|20|1092
gist circle_ops|14|804
gist point_ops|18|864
gist poly_ops|39|440
spgist box_ops|10|1092
spgist kd_point_ops|0|756
spgist poly_ops|28|440
spgist quad_point_ops|0|189
===== r0nc (index | mismatches | checks)
checks 6769, mismatches 498, index not used 0
brin box_inclusion_ops|79|1092
gist box_ops|86|1092
gist circle_ops|62|804
gist point_ops|162|864
gist poly_ops|71|440
spgist box_ops|10|1092
spgist kd_point_ops|0|756
spgist poly_ops|28|440
spgist quad_point_ops|0|189
===== r5nc (index | mismatches | checks)
checks 6769, mismatches 129, index not used 0
brin box_inclusion_ops|0|1092
gist box_ops|20|1092
gist circle_ops|14|804
gist point_ops|18|864
gist poly_ops|39|440
spgist box_ops|10|1092
spgist kd_point_ops|0|756
spgist poly_ops|28|440
spgist quad_point_ops|0|189
===== r0nc-to-r5nc (index | mismatches | checks)
checks 6769, mismatches 126, index not used 0
brin box_inclusion_ops|0|1092
gist box_ops|20|1092
gist circle_ops|14|804
gist point_ops|15|864
gist poly_ops|39|440
spgist box_ops|10|1092
spgist kd_point_ops|0|756
spgist poly_ops|28|440
spgist quad_point_ops|0|189
(r0nc-to-r5nc: indexes built by unpatched REL_18, searched by patched REL_18, no REINDEX)
===== the 129 remaining mismatches with v5 on master
gist box_ops | nan (1,NaN),(0,0) head | order <->(box,point) | (0.5,0.5) seq=0,0.5,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898, idx=ERROR: inconsistent point values
gist box_ops | nan (1,NaN),(0,0) head | order <->(box,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist box_ops | nan (1,NaN),(0,0) only | order <->(box,point) | (0.5,0.5) seq=0.5 idx=ERROR: inconsistent point values
gist box_ops | nan (1,NaN),(0,0) only | order <->(box,point) | (1,NaN) seq=NaN idx=ERROR: inconsistent point values
gist box_ops | nan (1,NaN),(0,0) sparse | order <->(box,point) | (0.5,0.5) seq=0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.707106 idx=ERROR: inconsistent point values
gist box_ops | nan (1,NaN),(0,0) sparse | order <->(box,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist box_ops | nan (1,NaN),(0,0) tail | order <->(box,point) | (0.5,0.5) seq=0,0.5,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898, idx=ERROR: inconsistent point values
gist box_ops | nan (1,NaN),(0,0) tail | order <->(box,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist box_ops | nan (NaN,NaN),(0,0) head | order <->(box,point) | (0.5,0.5) seq=0,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,1.58 idx=0,0.5,0.5,0.7071067811865476,NaN,1.5,1.5,1.5811388300841898,
gist box_ops | nan (NaN,NaN),(0,0) head | order <->(box,point) | (-1,-1) seq=1.4142135623730951,2.23606797749979,2.23606797749979,2.82842 idx=NaN,1.4142135623730951,2.23606797749979,2.23606797749979,2.8
gist box_ops | nan (NaN,NaN),(0,0) head | order <->(box,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist box_ops | nan (NaN,NaN),(0,0) sparse | order <->(box,point) | (0.5,0.5) seq=0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,1.5811 idx=0.5,0.5,0.7071067811865476,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,N
gist box_ops | nan (NaN,NaN),(0,0) sparse | order <->(box,point) | (-1,-1) seq=2.23606797749979,2.23606797749979,2.8284271247461903,3.16227 idx=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,2.23606797749979
gist box_ops | nan (NaN,NaN),(0,0) sparse | order <->(box,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist box_ops | nan (NaN,NaN),(0,0) tail | order <->(box,point) | (0.5,0.5) seq=0,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,1.58 idx=0,0.5,0.5,0.7071067811865476,NaN,1.5,1.5,1.5811388300841898,
gist box_ops | nan (NaN,NaN),(0,0) tail | order <->(box,point) | (-1,-1) seq=1.4142135623730951,2.23606797749979,2.23606797749979,2.82842 idx=NaN,1.4142135623730951,2.23606797749979,2.23606797749979,2.8
gist box_ops | nan (NaN,NaN),(0,0) tail | order <->(box,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist box_ops | nan (NaN,NaN),(NaN,NaN) head | order <->(box,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist box_ops | nan (NaN,NaN),(NaN,NaN) sparse | order <->(box,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist box_ops | nan (NaN,NaN),(NaN,NaN) tail | order <->(box,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist circle_ops | nan <(0,0),NaN> head | order <->(circle,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist circle_ops | nan <(0,0),NaN> sparse | order <->(circle,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist circle_ops | nan <(0,0),NaN> tail | order <->(circle,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist circle_ops | nan <(1,NaN),1> head | order <->(circle,point) | (0.5,0.5) seq=0,0,0,0,0.5811388300841898,0.5811388300841898,0.581138830084 idx=ERROR: inconsistent point values
gist circle_ops | nan <(1,NaN),1> head | order <->(circle,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist circle_ops | nan <(1,NaN),1> only | order <->(circle,point) | (0.5,0.5) seq=NaN idx=ERROR: inconsistent point values
gist circle_ops | nan <(1,NaN),1> only | order <->(circle,point) | (1,NaN) seq=NaN idx=ERROR: inconsistent point values
gist circle_ops | nan <(1,NaN),1> sparse | order <->(circle,point) | (0.5,0.5) seq=0,0,0,0.5811388300841898,0.5811388300841898,0.58113883008418 idx=ERROR: inconsistent point values
gist circle_ops | nan <(1,NaN),1> sparse | order <->(circle,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist circle_ops | nan <(1,NaN),1> tail | order <->(circle,point) | (0.5,0.5) seq=0,0,0,0,0.5811388300841898,0.5811388300841898,0.581138830084 idx=ERROR: inconsistent point values
gist circle_ops | nan <(1,NaN),1> tail | order <->(circle,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist circle_ops | nan <(NaN,NaN),1> head | order <->(circle,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist circle_ops | nan <(NaN,NaN),1> sparse | order <->(circle,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist circle_ops | nan <(NaN,NaN),1> tail | order <->(circle,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist point_ops | nan (1,NaN) head | order <->(point,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist point_ops | nan (1,NaN) sparse | order <->(point,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist point_ops | nan (1,NaN) tail | order <->(point,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist point_ops | nan (NaN,1) head | order <->(point,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist point_ops | nan (NaN,1) sparse | order <->(point,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist point_ops | nan (NaN,1) tail | order <->(point,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist point_ops | nan (NaN,NaN) head | order <->(point,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist point_ops | nan (NaN,NaN) sparse | order <->(point,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist point_ops | nan (NaN,NaN) tail | order <->(point,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist point_ops | nan (1,NaN) head | search <@(point,polygon) | ((NaN,NaN),(0,0),(1,1)) seq=2026 idx=0
gist point_ops | nan (1,NaN) sparse | search <@(point,polygon) | ((NaN,NaN),(0,0),(1,1)) seq=2025 idx=0
gist point_ops | nan (1,NaN) tail | search <@(point,polygon) | ((NaN,NaN),(0,0),(1,1)) seq=2026 idx=0
gist point_ops | nan (NaN,1) head | search <@(point,polygon) | ((NaN,NaN),(0,0),(1,1)) seq=2026 idx=0
gist point_ops | nan (NaN,1) sparse | search <@(point,polygon) | ((NaN,NaN),(0,0),(1,1)) seq=2025 idx=0
gist point_ops | nan (NaN,1) tail | search <@(point,polygon) | ((NaN,NaN),(0,0),(1,1)) seq=2026 idx=0
gist point_ops | nan (NaN,NaN) head | search <@(point,polygon) | ((NaN,NaN),(0,0),(1,1)) seq=2026 idx=0
gist point_ops | nan (NaN,NaN) sparse | search <@(point,polygon) | ((NaN,NaN),(0,0),(1,1)) seq=2025 idx=0
gist point_ops | nan (NaN,NaN) tail | search <@(point,polygon) | ((NaN,NaN),(0,0),(1,1)) seq=2026 idx=0
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) head | order <->(polygon,point) | (0.5,0.5) seq=0,0,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,1. idx=ERROR: inconsistent point values
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) head | order <->(polygon,point) | (1,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: inconsistent point values
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) head | order <->(polygon,point) | (NaN,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) only | order <->(polygon,point) | (0.5,0.5) seq=0 idx=ERROR: inconsistent point values
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) only | order <->(polygon,point) | (10,10) seq=12.727922061357855 idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) only | order <->(polygon,point) | (1,NaN) seq=0 idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) only | order <->(polygon,point) | (44,44) seq=60.81118318204309 idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) only | order <->(polygon,point) | (99,99) seq=138.59292911256333 idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) only | order <->(polygon,point) | (NaN,NaN) seq=0 idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) sparse | order <->(polygon,point) | (0.5,0.5) seq=0,0,0,0,0,0,0,0,0,0,0,0.5,0.5,0.7071067811865476,1.5,1.5,1.5 idx=ERROR: inconsistent point values
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) sparse | order <->(polygon,point) | (1,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: inconsistent point values
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) sparse | order <->(polygon,point) | (NaN,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) tail | order <->(polygon,point) | (0.5,0.5) seq=0,0,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,1. idx=ERROR: inconsistent point values
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) tail | order <->(polygon,point) | (1,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: inconsistent point values
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) tail | order <->(polygon,point) | (NaN,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) head | order <->(polygon,point) | (0.5,0.5) seq=0,0,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,1. idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) head | order <->(polygon,point) | (10,10) seq=0,0,0,0,0,1,1,1,1,1,1,1,1,1.4142135623730951,1.4142135623730 idx=0,0,0,0,1,1,1,1,1,1,1,1,1.4142135623730951,1.414213562373095
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) head | order <->(polygon,point) | (1,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: inconsistent point values
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) head | order <->(polygon,point) | (44,44) seq=0,0,0,0,0,1,1,1,1,1.4142135623730951,2,2,2,2,2.2360679774997 idx=0,0,0,0,1,1,1,1,1.4142135623730951,2,2,2,2,2.23606797749979,
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) head | order <->(polygon,point) | (99,99) seq=0,76.36753236814714,77.07788269017254,77.07788269017254,77.7 idx=76.36753236814714,77.07788269017254,77.07788269017254,77.781
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) head | order <->(polygon,point) | (NaN,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) only | order <->(polygon,point) | (0.5,0.5) seq=0 idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) only | order <->(polygon,point) | (10,10) seq=0 idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) only | order <->(polygon,point) | (1,NaN) seq=0 idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) only | order <->(polygon,point) | (44,44) seq=0 idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) only | order <->(polygon,point) | (99,99) seq=0 idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) only | order <->(polygon,point) | (NaN,NaN) seq=0 idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) sparse | order <->(polygon,point) | (0.5,0.5) seq=0,0,0,0,0,0,0,0,0,0,0,0.5,0.5,0.7071067811865476,1.5,1.5,1.5 idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) sparse | order <->(polygon,point) | (10,10) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1.414213562373 idx=0,0,0,0,1,1,1,1,1,1,1,1,1.4142135623730951,1.414213562373095
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) sparse | order <->(polygon,point) | (1,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: inconsistent point values
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) sparse | order <->(polygon,point) | (44,44) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1.4142135623730951,2,2 idx=0,0,0,0,1,1,1,1,1.4142135623730951,2,2,2,2,2.23606797749979,
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) sparse | order <->(polygon,point) | (99,99) seq=0,0,0,0,0,0,0,0,0,0,0,76.36753236814714,77.07788269017254,77 idx=76.36753236814714,77.07788269017254,77.07788269017254,77.781
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) sparse | order <->(polygon,point) | (NaN,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) tail | order <->(polygon,point) | (0.5,0.5) seq=0,0,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,1. idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) tail | order <->(polygon,point) | (10,10) seq=0,0,0,0,0,1,1,1,1,1,1,1,1,1.4142135623730951,1.4142135623730 idx=0,0,0,0,1,1,1,1,1,1,1,1,1.4142135623730951,1.414213562373095
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) tail | order <->(polygon,point) | (1,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: inconsistent point values
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) tail | order <->(polygon,point) | (44,44) seq=0,0,0,0,0,1,1,1,1,1.4142135623730951,2,2,2,2,2.2360679774997 idx=0,0,0,0,1,1,1,1,1.4142135623730951,2,2,2,2,2.23606797749979,
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) tail | order <->(polygon,point) | (99,99) seq=0,76.36753236814714,77.07788269017254,77.07788269017254,77.7 idx=76.36753236814714,77.07788269017254,77.07788269017254,77.781
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) tail | order <->(polygon,point) | (NaN,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist box_ops | nan (NaN,NaN),(0,0) head | search ~=(box,box) | (NaN,NaN),(0,0) seq=1 idx=0
spgist box_ops | nan (NaN,NaN),(0,0) sparse | search ~=(box,box) | (NaN,NaN),(0,0) seq=11 idx=0
spgist box_ops | nan (NaN,NaN),(0,0) tail | search ~=(box,box) | (NaN,NaN),(0,0) seq=1 idx=0
spgist box_ops | nan (1,NaN),(0,0) tail | order <->(box,point) | (0.5,0.5) seq=0,0.5,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898, idx=0.5,0,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,
spgist box_ops | nan (NaN,NaN),(0,0) head | order <->(box,point) | (0.5,0.5) seq=0,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,1.58 idx=0,NaN,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,
spgist box_ops | nan (NaN,NaN),(0,0) head | order <->(box,point) | (-1,-1) seq=1.4142135623730951,2.23606797749979,2.23606797749979,2.82842 idx=1.4142135623730951,NaN,2.23606797749979,2.23606797749979,2.8
spgist box_ops | nan (NaN,NaN),(0,0) sparse | order <->(box,point) | (0.5,0.5) seq=0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,1.5811 idx=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,0.5,0.5,0.707106
spgist box_ops | nan (NaN,NaN),(0,0) sparse | order <->(box,point) | (-1,-1) seq=2.23606797749979,2.23606797749979,2.8284271247461903,3.16227 idx=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,2.23606797749979
spgist box_ops | nan (NaN,NaN),(0,0) tail | order <->(box,point) | (0.5,0.5) seq=0,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,1.58 idx=NaN,0,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,
spgist box_ops | nan (NaN,NaN),(0,0) tail | order <->(box,point) | (-1,-1) seq=1.4142135623730951,2.23606797749979,2.23606797749979,2.82842 idx=NaN,1.4142135623730951,2.23606797749979,2.23606797749979,2.8
spgist poly_ops | nan ((0,0),(NaN,1),(1,1)) head | order <->(polygon,point) | (1,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((0,0),(NaN,1),(1,1)) head | order <->(polygon,point) | (NaN,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((0,0),(NaN,1),(1,1)) only | order <->(polygon,point) | (1,NaN) seq=0 idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((0,0),(NaN,1),(1,1)) only | order <->(polygon,point) | (NaN,NaN) seq=0 idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((0,0),(NaN,1),(1,1)) sparse | order <->(polygon,point) | (1,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((0,0),(NaN,1),(1,1)) sparse | order <->(polygon,point) | (NaN,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((0,0),(NaN,1),(1,1)) tail | order <->(polygon,point) | (1,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((0,0),(NaN,1),(1,1)) tail | order <->(polygon,point) | (NaN,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) head | order <->(polygon,point) | (10,10) seq=0,0,0,0,0,1,1,1,1,1,1,1,1,1.4142135623730951,1.4142135623730 idx=0,0,0,0,1,1,1,1,1,1,1,1,1.4142135623730951,1.414213562373095
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) head | order <->(polygon,point) | (1,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) head | order <->(polygon,point) | (44,44) seq=0,0,0,0,0,1,1,1,1,1.4142135623730951,2,2,2,2,2.2360679774997 idx=0,0,0,0,1,1,1,1,1.4142135623730951,2,2,2,2,2.23606797749979,
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) head | order <->(polygon,point) | (99,99) seq=0,76.36753236814714,77.07788269017254,77.07788269017254,77.7 idx=76.36753236814714,77.07788269017254,77.07788269017254,77.781
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) head | order <->(polygon,point) | (NaN,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) only | order <->(polygon,point) | (1,NaN) seq=0 idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) only | order <->(polygon,point) | (NaN,NaN) seq=0 idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) sparse | order <->(polygon,point) | (10,10) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1.414213562373 idx=0,0,0,0,1,1,1,1,1,1,1,1,1.4142135623730951,1.414213562373095
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) sparse | order <->(polygon,point) | (1,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) sparse | order <->(polygon,point) | (44,44) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1.4142135623730951,2,2 idx=0,0,0,0,1,1,1,1,1.4142135623730951,2,2,2,2,2.23606797749979,
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) sparse | order <->(polygon,point) | (99,99) seq=0,0,0,0,0,0,0,0,0,0,0,76.36753236814714,77.07788269017254,77 idx=76.36753236814714,77.07788269017254,77.07788269017254,77.781
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) sparse | order <->(polygon,point) | (NaN,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) tail | order <->(polygon,point) | (10,10) seq=0,0,0,0,0,1,1,1,1,1,1,1,1,1.4142135623730951,1.4142135623730 idx=0,0,0,0,1,1,1,1,1,1,1,1,1.4142135623730951,1.414213562373095
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) tail | order <->(polygon,point) | (1,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) tail | order <->(polygon,point) | (44,44) seq=0,0,0,0,0,1,1,1,1,1.4142135623730951,2,2,2,2,2.2360679774997 idx=0,0,0,0,1,1,1,1,1.4142135623730951,2,2,2,2,2.23606797749979,
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) tail | order <->(polygon,point) | (99,99) seq=0,76.36753236814714,77.07788269017254,77.07788269017254,77.7 idx=76.36753236814714,77.07788269017254,77.07788269017254,77.781
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) tail | order <->(polygon,point) | (NaN,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) head | search ~=(polygon,polygon) | ((NaN,NaN),(0,0),(1,1)) seq=1 idx=0
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) sparse | search ~=(polygon,polygon) | ((NaN,NaN),(0,0),(1,1)) seq=11 idx=0
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) tail | search ~=(polygon,polygon) | ((NaN,NaN),(0,0),(1,1)) seq=1 idx=0
===== minimal case: box <-> point on GiST, one box with NaN in one coordinate
CREATE TABLE b (v box);
INSERT INTO b VALUES ('(1,NaN),(0,0)');
INSERT INTO b SELECT box(point(x, y), point(x + 1, y + 1)) FROM generate_series(0, 44) x, generate_series(0, 44) y;
CREATE INDEX ON b USING gist (v);
SET enable_seqscan = off;
SELECT v <-> point '(0.5,0.5)' AS d FROM b ORDER BY v <-> point '(0.5,0.5)' LIMIT 3;
RESET enable_seqscan;
SET enable_indexscan = off; SET enable_bitmapscan = off;
SELECT v <-> point '(0.5,0.5)' AS d FROM b ORDER BY v <-> point '(0.5,0.5)' LIMIT 3;
-- index scan: ERROR: inconsistent point values (unpatched and v5, master and REL_18)
-- with --enable-cassert: TRAP: failed Assert("box->low.y <= box->high.y") in computeDistance()
-- seq scan: 0, 0.5, 0.5
===== v5-0002 on REL_18: the only conflict (palloc_object is not in 18)
121- unionL = (BOX *) palloc(sizeof(BOX));
122-- *unionL = *cur;
123:+ snapBox(unionL, cur);
124- }
125- else
--
130- unionR = (BOX *) palloc(sizeof(BOX));
131-- *unionR = *cur;
132:+ snapBox(unionR, cur);
133- }
134- else
===== nan_matrix.sql
-- Differential check for bug #19705: does an index on a geometric column
-- return the same rows as a sequential scan, when some stored values have a
-- NaN coordinate?
--
-- nan_setup() creates one table per (index kind, NaN value, NaN position),
-- each with exactly one index, so the planner has nothing else to choose.
-- nan_check(label) then runs, for every operator the operator class declares
-- in pg_amop, a count(*) with a sequential scan and with the index, and for
-- ordering operators it compares the first 50 distances. It records
-- whether EXPLAIN really used the index.
--
-- Setup and check are separate so that the indexes can be built by one
-- binary and searched by another (a minor upgrade, no REINDEX).
CREATE TABLE IF NOT EXISTS nan_case (
tbl text PRIMARY KEY,
typ text, -- box, point, polygon, circle
am text,
opclass text,
nanval text,
pos text, -- head, tail, only, sparse
err text -- error from CREATE INDEX, if any (then nothing to check)
);
CREATE OR REPLACE FUNCTION nan_setup() RETURNS int LANGUAGE plpgsql AS $$
DECLARE
k record; nv text; pos text; t text; n int := 0; finite text; nanexpr text; err text;
BEGIN
SET LOCAL client_min_messages = warning;
FOR k IN SELECT * FROM (VALUES
('box', 'brin', 'box_inclusion_ops'),
('box', 'gist', 'box_ops'),
('box', 'spgist', 'box_ops'),
('point', 'gist', 'point_ops'),
('point', 'spgist', 'quad_point_ops'),
('point', 'spgist', 'kd_point_ops'),
('polygon', 'gist', 'poly_ops'),
('polygon', 'spgist', 'poly_ops'),
('circle', 'gist', 'circle_ops')) AS x(typ, am, opclass)
LOOP
-- 45 x 45 = 2025 finite values spread over a grid, so GiST and
-- SP-GiST split into several levels and BRIN has many ranges.
finite := CASE k.typ
WHEN 'box' THEN 'box(point(x, y), point(x + 1, y + 1))'
WHEN 'point' THEN 'point(x, y)'
WHEN 'polygon' THEN 'polygon(box(point(x, y), point(x + 1, y + 1)))'
WHEN 'circle' THEN 'circle(point(x, y), 1)' END;
FOREACH nv IN ARRAY CASE k.typ
WHEN 'box' THEN ARRAY['(NaN,NaN),(0,0)', '(1,NaN),(0,0)', '(NaN,NaN),(NaN,NaN)']
WHEN 'point' THEN ARRAY['(NaN,NaN)', '(NaN,1)', '(1,NaN)']
WHEN 'polygon' THEN ARRAY['((NaN,NaN),(0,0),(1,1))', '((0,0),(NaN,1),(1,1))']
WHEN 'circle' THEN ARRAY['<(NaN,NaN),1>', '<(0,0),NaN>', '<(1,NaN),1>'] END
LOOP
nanexpr := format('%L::%s', nv, k.typ);
FOREACH pos IN ARRAY ARRAY['head', 'tail', 'only', 'sparse'] LOOP
n := n + 1;
t := format('nan_%s_%s', k.typ, n);
EXECUTE format('DROP TABLE IF EXISTS %I', t);
EXECUTE format('CREATE TABLE %I (id serial, v %s)', t, k.typ);
IF pos = 'head' THEN
EXECUTE format('INSERT INTO %I (v) VALUES (%s)', t, nanexpr);
END IF;
IF pos <> 'only' THEN
EXECUTE format('INSERT INTO %I (v) SELECT CASE WHEN %L = ''sparse'' AND (x * 45 + y) %% 200 = 0 '
'THEN %s ELSE %s END FROM generate_series(0, 44) x, generate_series(0, 44) y',
t, pos, nanexpr, finite);
END IF;
IF pos IN ('tail', 'only') THEN
EXECUTE format('INSERT INTO %I (v) VALUES (%s)', t, nanexpr);
END IF;
err := NULL;
BEGIN
EXECUTE format('CREATE INDEX %I ON %I USING %s (v %s)%s', t || '_idx', t, k.am, k.opclass,
CASE WHEN k.am = 'brin' THEN ' WITH (pages_per_range = 1)' ELSE '' END);
EXCEPTION WHEN OTHERS THEN
err := SQLERRM;
END;
EXECUTE format('ANALYZE %I', t);
INSERT INTO nan_case VALUES (t, k.typ, k.am, k.opclass, nv, pos, err)
ON CONFLICT (tbl) DO UPDATE SET typ = EXCLUDED.typ, am = EXCLUDED.am,
opclass = EXCLUDED.opclass, nanval = EXCLUDED.nanval, pos = EXCLUDED.pos,
err = EXCLUDED.err;
END LOOP;
END LOOP;
END LOOP;
RETURN n;
END $$;
CREATE TABLE IF NOT EXISTS nan_result (
label text, tbl text, op text, probe text, kind text,
seq text, idx text, used_index bool, same bool
);
CREATE OR REPLACE FUNCTION nan_probes(typ text) RETURNS text[] LANGUAGE sql IMMUTABLE AS $$
SELECT CASE typ
WHEN 'box' THEN ARRAY['(-2,-2),(2,2)', '(0,0),(1,1)', '(10,10),(12,12)', '(40,40),(46,46)',
'(100,100),(200,200)', '(NaN,NaN),(0,0)', '(-Infinity,-Infinity),(Infinity,Infinity)']
WHEN 'point' THEN ARRAY['(0.5,0.5)', '(10,10)', '(44,44)', '(99,99)', '(-1,-1)', '(NaN,NaN)', '(1,NaN)']
WHEN 'polygon' THEN ARRAY['((0,0),(0,20),(20,20),(20,0))', '((5,5),(5,6),(6,6))',
'((0,0),(0,1),(1,1),(1,0))', '((NaN,NaN),(0,0),(1,1))']
WHEN 'circle' THEN ARRAY['<(0,0),5>', '<(20,20),3>', '<(44,44),0.5>', '<(NaN,NaN),1>', '<(0,0),Infinity>']
END
$$;
CREATE OR REPLACE FUNCTION nan_check(p_label text) RETURNS TABLE (checks int, mismatches int, not_indexed int)
LANGUAGE plpgsql AS $$
DECLARE
c record; o record; pr text; q text; s_res text; i_res text; plan text; line text; used bool;
nchk int := 0; nbad int := 0; nnoidx int := 0;
BEGIN
DELETE FROM nan_result WHERE label = p_label;
FOR c IN SELECT * FROM nan_case WHERE err IS NULL ORDER BY tbl LOOP
FOR o IN
SELECT DISTINCT ao.amoppurpose AS purpose, ao.amopopr::regoperator AS opr,
op.oprname, format_type(op.oprright, NULL) AS rtype
FROM pg_opclass oc
JOIN pg_am am ON am.oid = oc.opcmethod
JOIN pg_amop ao ON ao.amopfamily = oc.opcfamily
JOIN pg_operator op ON op.oid = ao.amopopr
WHERE oc.opcname = c.opclass AND am.amname = c.am
AND format_type(op.oprleft, NULL) = c.typ
LOOP
-- Probes: the probe set of the operator's right-hand type, if we have one.
IF o.rtype NOT IN ('box', 'point', 'polygon', 'circle') THEN
CONTINUE;
END IF;
FOREACH pr IN ARRAY nan_probes(o.rtype) LOOP
IF o.purpose = 's' THEN
q := format('SELECT count(*)::text FROM %I WHERE v %s %L::%s', c.tbl, o.oprname, pr, o.rtype);
ELSE
q := format('SELECT string_agg(d::text, '','' ORDER BY rn) FROM (SELECT d, row_number() OVER () rn '
'FROM (SELECT v %s %L::%s AS d FROM %I ORDER BY v %s %L::%s LIMIT 50) a) b',
o.oprname, pr, o.rtype, c.tbl, o.oprname, pr, o.rtype);
END IF;
PERFORM set_config('enable_seqscan', 'on', true);
PERFORM set_config('enable_indexscan', 'off', true);
PERFORM set_config('enable_bitmapscan', 'off', true);
BEGIN
EXECUTE q INTO s_res;
EXCEPTION WHEN OTHERS THEN
s_res := 'ERROR: ' || SQLERRM;
END;
PERFORM set_config('enable_seqscan', 'off', true);
PERFORM set_config('enable_indexscan', 'on', true);
PERFORM set_config('enable_bitmapscan', 'on', true);
-- Leaves the last index query in the server log, in case it crashes.
RAISE LOG 'nan_check % % [nan % %]: %', c.am, c.opclass, c.nanval, c.pos, q;
plan := '';
FOR line IN EXECUTE 'EXPLAIN (COSTS OFF) ' || q LOOP
plan := plan || line || E'\n';
END LOOP;
used := plan LIKE '%' || c.tbl || '_idx%';
BEGIN
EXECUTE q INTO i_res;
EXCEPTION WHEN OTHERS THEN
i_res := 'ERROR: ' || SQLERRM;
END;
nchk := nchk + 1;
IF NOT used THEN nnoidx := nnoidx + 1; END IF;
IF s_res IS DISTINCT FROM i_res THEN nbad := nbad + 1; END IF;
INSERT INTO nan_result VALUES (p_label, c.tbl, o.opr::text, pr,
CASE o.purpose WHEN 's' THEN 'search' ELSE 'order' END,
s_res, i_res, used, s_res IS NOT DISTINCT FROM i_res);
END LOOP;
END LOOP;
END LOOP;
RETURN QUERY SELECT nchk, nbad, nnoidx;
END $$;
===== run.sh
#!/bin/bash
# Runs nan_matrix.sql.
# run.sh fresh BUILD initdb, nan_setup() and nan_check() with BUILD
# run.sh upgrade OLD NEW nan_setup() with OLD, then nan_check() with NEW
# on the same data directory (a minor upgrade)
# Writes <label>.summary.txt and <label>.mismatch.txt next to this script.
set -eu
A=$(cd "$(dirname "$0")" && pwd)
W=$HOME/pg19705nan
MODE=$1; shift
if [ $MODE = fresh ]; then SETUP=$1; CHECK=$1; LABEL=$1; else SETUP=$1; CHECK=$2; LABEL=$1-to-$2; fi
D=$(mktemp -d /tmp/claude-1000/nan.XXXX); P=$((55800 + RANDOM % 100))
start() { $W/i-$1/bin/pg_ctl -D $D -l $D/log.$1 -o "-p $P -c unix_socket_directories=/tmp" -w start >/dev/null; }
stop() { $W/i-$1/bin/pg_ctl -D $D -m fast -w stop >/dev/null; }
q() { $W/i-$1/bin/psql -X -q -v ON_ERROR_STOP=1 -h /tmp -p $P -d postgres "${@:2}"; }
$W/i-$SETUP/bin/initdb -D $D -A trust --no-sync >/dev/null
start $SETUP
q $SETUP -f $A/nan_matrix.sql >/dev/null
q $SETUP -Atc "SELECT 'tables: ' || nan_setup()"
q $SETUP -Atc "SELECT format('CREATE INDEX error: %s %s | nan %s %s | %s', am, opclass, nanval, pos, err)
FROM nan_case WHERE err IS NOT NULL ORDER BY tbl" | tee $A/$LABEL.createerr.txt
if [ $CHECK != $SETUP ]; then stop $SETUP; start $CHECK; fi
q $CHECK -Atc "SELECT version()" | cut -c1-60
q $CHECK -Atc "SELECT format('checks %s, mismatches %s, index not used %s', checks, mismatches, not_indexed) FROM nan_check('$LABEL')" \
| tee $A/$LABEL.summary.txt
q $CHECK -Atc "
SELECT c.am || ' ' || c.opclass AS index, count(*) FILTER (WHERE NOT r.same) AS wrong, count(*) AS checks
FROM nan_result r JOIN nan_case c USING (tbl)
WHERE r.label = '$LABEL' GROUP BY 1 ORDER BY 1" | tee -a $A/$LABEL.summary.txt
q $CHECK -Atc "
SELECT format('%s %s | nan %s %s | %s %s | %s seq=%s idx=%s%s', c.am, c.opclass, c.nanval, c.pos,
r.kind, r.op, r.probe, left(r.seq, 60), left(r.idx, 60),
CASE WHEN r.used_index THEN '' ELSE ' (index not used)' END)
FROM nan_result r JOIN nan_case c USING (tbl)
WHERE r.label = '$LABEL' AND NOT r.same
ORDER BY c.am, c.opclass, r.op, c.nanval, c.pos, r.probe" > $A/$LABEL.mismatch.txt
echo "mismatch lines: $(wc -l < $A/$LABEL.mismatch.txt)"
stop $CHECK
# rm -rf $D (kept for the logs)
Attachments:
[text/plain] nocfbot-19705-v5-nan-matrix.txt (36.3K, ../../179038137563.2667046.1851730046607530976@gmail.com/2-nocfbot-19705-v5-nan-matrix.txt)
download | inline:
Differential check of the v5 series for bug #19705 (Kirill's v5-0001 BRIN, v5-0002 GiST,
and the NaN point check for GiST polygon/circle searches), all built without assertions.
master 87762cbaa4f; REL_18_STABLE e1b2a86d6e7 with v5-REL_18-0001 + v5-0002 + the NaN point
patch (code only; v5-0002 needs the rebase below).
For each index (one per table), NaN value and position of the NaN rows, every operator the
opclass lists in pg_amop is run with a seq scan and with the index (EXPLAIN checked that the
index was used in every case). Search operators compare count(*), ordering operators the
first 50 distances. A mismatch is any difference, including an ERROR on one side only.
===== m0nc (index | mismatches | checks)
checks 6769, mismatches 493, index not used 0
brin box_inclusion_ops|79|1092
gist box_ops|87|1092
gist circle_ops|58|804
gist point_ops|162|864
gist poly_ops|69|440
spgist box_ops|10|1092
spgist kd_point_ops|0|756
spgist poly_ops|28|440
spgist quad_point_ops|0|189
===== m5nc (index | mismatches | checks)
checks 6769, mismatches 129, index not used 0
brin box_inclusion_ops|0|1092
gist box_ops|20|1092
gist circle_ops|14|804
gist point_ops|18|864
gist poly_ops|39|440
spgist box_ops|10|1092
spgist kd_point_ops|0|756
spgist poly_ops|28|440
spgist quad_point_ops|0|189
===== r0nc (index | mismatches | checks)
checks 6769, mismatches 498, index not used 0
brin box_inclusion_ops|79|1092
gist box_ops|86|1092
gist circle_ops|62|804
gist point_ops|162|864
gist poly_ops|71|440
spgist box_ops|10|1092
spgist kd_point_ops|0|756
spgist poly_ops|28|440
spgist quad_point_ops|0|189
===== r5nc (index | mismatches | checks)
checks 6769, mismatches 129, index not used 0
brin box_inclusion_ops|0|1092
gist box_ops|20|1092
gist circle_ops|14|804
gist point_ops|18|864
gist poly_ops|39|440
spgist box_ops|10|1092
spgist kd_point_ops|0|756
spgist poly_ops|28|440
spgist quad_point_ops|0|189
===== r0nc-to-r5nc (index | mismatches | checks)
checks 6769, mismatches 126, index not used 0
brin box_inclusion_ops|0|1092
gist box_ops|20|1092
gist circle_ops|14|804
gist point_ops|15|864
gist poly_ops|39|440
spgist box_ops|10|1092
spgist kd_point_ops|0|756
spgist poly_ops|28|440
spgist quad_point_ops|0|189
(r0nc-to-r5nc: indexes built by unpatched REL_18, searched by patched REL_18, no REINDEX)
===== the 129 remaining mismatches with v5 on master
gist box_ops | nan (1,NaN),(0,0) head | order <->(box,point) | (0.5,0.5) seq=0,0.5,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898, idx=ERROR: inconsistent point values
gist box_ops | nan (1,NaN),(0,0) head | order <->(box,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist box_ops | nan (1,NaN),(0,0) only | order <->(box,point) | (0.5,0.5) seq=0.5 idx=ERROR: inconsistent point values
gist box_ops | nan (1,NaN),(0,0) only | order <->(box,point) | (1,NaN) seq=NaN idx=ERROR: inconsistent point values
gist box_ops | nan (1,NaN),(0,0) sparse | order <->(box,point) | (0.5,0.5) seq=0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.707106 idx=ERROR: inconsistent point values
gist box_ops | nan (1,NaN),(0,0) sparse | order <->(box,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist box_ops | nan (1,NaN),(0,0) tail | order <->(box,point) | (0.5,0.5) seq=0,0.5,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898, idx=ERROR: inconsistent point values
gist box_ops | nan (1,NaN),(0,0) tail | order <->(box,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist box_ops | nan (NaN,NaN),(0,0) head | order <->(box,point) | (0.5,0.5) seq=0,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,1.58 idx=0,0.5,0.5,0.7071067811865476,NaN,1.5,1.5,1.5811388300841898,
gist box_ops | nan (NaN,NaN),(0,0) head | order <->(box,point) | (-1,-1) seq=1.4142135623730951,2.23606797749979,2.23606797749979,2.82842 idx=NaN,1.4142135623730951,2.23606797749979,2.23606797749979,2.8
gist box_ops | nan (NaN,NaN),(0,0) head | order <->(box,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist box_ops | nan (NaN,NaN),(0,0) sparse | order <->(box,point) | (0.5,0.5) seq=0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,1.5811 idx=0.5,0.5,0.7071067811865476,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,N
gist box_ops | nan (NaN,NaN),(0,0) sparse | order <->(box,point) | (-1,-1) seq=2.23606797749979,2.23606797749979,2.8284271247461903,3.16227 idx=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,2.23606797749979
gist box_ops | nan (NaN,NaN),(0,0) sparse | order <->(box,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist box_ops | nan (NaN,NaN),(0,0) tail | order <->(box,point) | (0.5,0.5) seq=0,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,1.58 idx=0,0.5,0.5,0.7071067811865476,NaN,1.5,1.5,1.5811388300841898,
gist box_ops | nan (NaN,NaN),(0,0) tail | order <->(box,point) | (-1,-1) seq=1.4142135623730951,2.23606797749979,2.23606797749979,2.82842 idx=NaN,1.4142135623730951,2.23606797749979,2.23606797749979,2.8
gist box_ops | nan (NaN,NaN),(0,0) tail | order <->(box,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist box_ops | nan (NaN,NaN),(NaN,NaN) head | order <->(box,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist box_ops | nan (NaN,NaN),(NaN,NaN) sparse | order <->(box,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist box_ops | nan (NaN,NaN),(NaN,NaN) tail | order <->(box,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist circle_ops | nan <(0,0),NaN> head | order <->(circle,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist circle_ops | nan <(0,0),NaN> sparse | order <->(circle,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist circle_ops | nan <(0,0),NaN> tail | order <->(circle,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist circle_ops | nan <(1,NaN),1> head | order <->(circle,point) | (0.5,0.5) seq=0,0,0,0,0.5811388300841898,0.5811388300841898,0.581138830084 idx=ERROR: inconsistent point values
gist circle_ops | nan <(1,NaN),1> head | order <->(circle,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist circle_ops | nan <(1,NaN),1> only | order <->(circle,point) | (0.5,0.5) seq=NaN idx=ERROR: inconsistent point values
gist circle_ops | nan <(1,NaN),1> only | order <->(circle,point) | (1,NaN) seq=NaN idx=ERROR: inconsistent point values
gist circle_ops | nan <(1,NaN),1> sparse | order <->(circle,point) | (0.5,0.5) seq=0,0,0,0.5811388300841898,0.5811388300841898,0.58113883008418 idx=ERROR: inconsistent point values
gist circle_ops | nan <(1,NaN),1> sparse | order <->(circle,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist circle_ops | nan <(1,NaN),1> tail | order <->(circle,point) | (0.5,0.5) seq=0,0,0,0,0.5811388300841898,0.5811388300841898,0.581138830084 idx=ERROR: inconsistent point values
gist circle_ops | nan <(1,NaN),1> tail | order <->(circle,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist circle_ops | nan <(NaN,NaN),1> head | order <->(circle,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist circle_ops | nan <(NaN,NaN),1> sparse | order <->(circle,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist circle_ops | nan <(NaN,NaN),1> tail | order <->(circle,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist point_ops | nan (1,NaN) head | order <->(point,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist point_ops | nan (1,NaN) sparse | order <->(point,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist point_ops | nan (1,NaN) tail | order <->(point,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist point_ops | nan (NaN,1) head | order <->(point,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist point_ops | nan (NaN,1) sparse | order <->(point,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist point_ops | nan (NaN,1) tail | order <->(point,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist point_ops | nan (NaN,NaN) head | order <->(point,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist point_ops | nan (NaN,NaN) sparse | order <->(point,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist point_ops | nan (NaN,NaN) tail | order <->(point,point) | (1,NaN) seq=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN, idx=ERROR: inconsistent point values
gist point_ops | nan (1,NaN) head | search <@(point,polygon) | ((NaN,NaN),(0,0),(1,1)) seq=2026 idx=0
gist point_ops | nan (1,NaN) sparse | search <@(point,polygon) | ((NaN,NaN),(0,0),(1,1)) seq=2025 idx=0
gist point_ops | nan (1,NaN) tail | search <@(point,polygon) | ((NaN,NaN),(0,0),(1,1)) seq=2026 idx=0
gist point_ops | nan (NaN,1) head | search <@(point,polygon) | ((NaN,NaN),(0,0),(1,1)) seq=2026 idx=0
gist point_ops | nan (NaN,1) sparse | search <@(point,polygon) | ((NaN,NaN),(0,0),(1,1)) seq=2025 idx=0
gist point_ops | nan (NaN,1) tail | search <@(point,polygon) | ((NaN,NaN),(0,0),(1,1)) seq=2026 idx=0
gist point_ops | nan (NaN,NaN) head | search <@(point,polygon) | ((NaN,NaN),(0,0),(1,1)) seq=2026 idx=0
gist point_ops | nan (NaN,NaN) sparse | search <@(point,polygon) | ((NaN,NaN),(0,0),(1,1)) seq=2025 idx=0
gist point_ops | nan (NaN,NaN) tail | search <@(point,polygon) | ((NaN,NaN),(0,0),(1,1)) seq=2026 idx=0
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) head | order <->(polygon,point) | (0.5,0.5) seq=0,0,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,1. idx=ERROR: inconsistent point values
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) head | order <->(polygon,point) | (1,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: inconsistent point values
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) head | order <->(polygon,point) | (NaN,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) only | order <->(polygon,point) | (0.5,0.5) seq=0 idx=ERROR: inconsistent point values
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) only | order <->(polygon,point) | (10,10) seq=12.727922061357855 idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) only | order <->(polygon,point) | (1,NaN) seq=0 idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) only | order <->(polygon,point) | (44,44) seq=60.81118318204309 idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) only | order <->(polygon,point) | (99,99) seq=138.59292911256333 idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) only | order <->(polygon,point) | (NaN,NaN) seq=0 idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) sparse | order <->(polygon,point) | (0.5,0.5) seq=0,0,0,0,0,0,0,0,0,0,0,0.5,0.5,0.7071067811865476,1.5,1.5,1.5 idx=ERROR: inconsistent point values
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) sparse | order <->(polygon,point) | (1,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: inconsistent point values
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) sparse | order <->(polygon,point) | (NaN,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) tail | order <->(polygon,point) | (0.5,0.5) seq=0,0,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,1. idx=ERROR: inconsistent point values
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) tail | order <->(polygon,point) | (1,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: inconsistent point values
gist poly_ops | nan ((0,0),(NaN,1),(1,1)) tail | order <->(polygon,point) | (NaN,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) head | order <->(polygon,point) | (0.5,0.5) seq=0,0,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,1. idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) head | order <->(polygon,point) | (10,10) seq=0,0,0,0,0,1,1,1,1,1,1,1,1,1.4142135623730951,1.4142135623730 idx=0,0,0,0,1,1,1,1,1,1,1,1,1.4142135623730951,1.414213562373095
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) head | order <->(polygon,point) | (1,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: inconsistent point values
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) head | order <->(polygon,point) | (44,44) seq=0,0,0,0,0,1,1,1,1,1.4142135623730951,2,2,2,2,2.2360679774997 idx=0,0,0,0,1,1,1,1,1.4142135623730951,2,2,2,2,2.23606797749979,
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) head | order <->(polygon,point) | (99,99) seq=0,76.36753236814714,77.07788269017254,77.07788269017254,77.7 idx=76.36753236814714,77.07788269017254,77.07788269017254,77.781
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) head | order <->(polygon,point) | (NaN,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) only | order <->(polygon,point) | (0.5,0.5) seq=0 idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) only | order <->(polygon,point) | (10,10) seq=0 idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) only | order <->(polygon,point) | (1,NaN) seq=0 idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) only | order <->(polygon,point) | (44,44) seq=0 idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) only | order <->(polygon,point) | (99,99) seq=0 idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) only | order <->(polygon,point) | (NaN,NaN) seq=0 idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) sparse | order <->(polygon,point) | (0.5,0.5) seq=0,0,0,0,0,0,0,0,0,0,0,0.5,0.5,0.7071067811865476,1.5,1.5,1.5 idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) sparse | order <->(polygon,point) | (10,10) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1.414213562373 idx=0,0,0,0,1,1,1,1,1,1,1,1,1.4142135623730951,1.414213562373095
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) sparse | order <->(polygon,point) | (1,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: inconsistent point values
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) sparse | order <->(polygon,point) | (44,44) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1.4142135623730951,2,2 idx=0,0,0,0,1,1,1,1,1.4142135623730951,2,2,2,2,2.23606797749979,
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) sparse | order <->(polygon,point) | (99,99) seq=0,0,0,0,0,0,0,0,0,0,0,76.36753236814714,77.07788269017254,77 idx=76.36753236814714,77.07788269017254,77.07788269017254,77.781
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) sparse | order <->(polygon,point) | (NaN,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) tail | order <->(polygon,point) | (0.5,0.5) seq=0,0,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,1. idx=ERROR: index returned tuples in wrong order
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) tail | order <->(polygon,point) | (10,10) seq=0,0,0,0,0,1,1,1,1,1,1,1,1,1.4142135623730951,1.4142135623730 idx=0,0,0,0,1,1,1,1,1,1,1,1,1.4142135623730951,1.414213562373095
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) tail | order <->(polygon,point) | (1,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: inconsistent point values
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) tail | order <->(polygon,point) | (44,44) seq=0,0,0,0,0,1,1,1,1,1.4142135623730951,2,2,2,2,2.2360679774997 idx=0,0,0,0,1,1,1,1,1.4142135623730951,2,2,2,2,2.23606797749979,
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) tail | order <->(polygon,point) | (99,99) seq=0,76.36753236814714,77.07788269017254,77.07788269017254,77.7 idx=76.36753236814714,77.07788269017254,77.07788269017254,77.781
gist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) tail | order <->(polygon,point) | (NaN,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist box_ops | nan (NaN,NaN),(0,0) head | search ~=(box,box) | (NaN,NaN),(0,0) seq=1 idx=0
spgist box_ops | nan (NaN,NaN),(0,0) sparse | search ~=(box,box) | (NaN,NaN),(0,0) seq=11 idx=0
spgist box_ops | nan (NaN,NaN),(0,0) tail | search ~=(box,box) | (NaN,NaN),(0,0) seq=1 idx=0
spgist box_ops | nan (1,NaN),(0,0) tail | order <->(box,point) | (0.5,0.5) seq=0,0.5,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898, idx=0.5,0,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,
spgist box_ops | nan (NaN,NaN),(0,0) head | order <->(box,point) | (0.5,0.5) seq=0,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,1.58 idx=0,NaN,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,
spgist box_ops | nan (NaN,NaN),(0,0) head | order <->(box,point) | (-1,-1) seq=1.4142135623730951,2.23606797749979,2.23606797749979,2.82842 idx=1.4142135623730951,NaN,2.23606797749979,2.23606797749979,2.8
spgist box_ops | nan (NaN,NaN),(0,0) sparse | order <->(box,point) | (0.5,0.5) seq=0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,1.5811 idx=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,0.5,0.5,0.707106
spgist box_ops | nan (NaN,NaN),(0,0) sparse | order <->(box,point) | (-1,-1) seq=2.23606797749979,2.23606797749979,2.8284271247461903,3.16227 idx=NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,2.23606797749979
spgist box_ops | nan (NaN,NaN),(0,0) tail | order <->(box,point) | (0.5,0.5) seq=0,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,1.58 idx=NaN,0,0.5,0.5,0.7071067811865476,1.5,1.5,1.5811388300841898,
spgist box_ops | nan (NaN,NaN),(0,0) tail | order <->(box,point) | (-1,-1) seq=1.4142135623730951,2.23606797749979,2.23606797749979,2.82842 idx=NaN,1.4142135623730951,2.23606797749979,2.23606797749979,2.8
spgist poly_ops | nan ((0,0),(NaN,1),(1,1)) head | order <->(polygon,point) | (1,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((0,0),(NaN,1),(1,1)) head | order <->(polygon,point) | (NaN,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((0,0),(NaN,1),(1,1)) only | order <->(polygon,point) | (1,NaN) seq=0 idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((0,0),(NaN,1),(1,1)) only | order <->(polygon,point) | (NaN,NaN) seq=0 idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((0,0),(NaN,1),(1,1)) sparse | order <->(polygon,point) | (1,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((0,0),(NaN,1),(1,1)) sparse | order <->(polygon,point) | (NaN,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((0,0),(NaN,1),(1,1)) tail | order <->(polygon,point) | (1,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((0,0),(NaN,1),(1,1)) tail | order <->(polygon,point) | (NaN,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) head | order <->(polygon,point) | (10,10) seq=0,0,0,0,0,1,1,1,1,1,1,1,1,1.4142135623730951,1.4142135623730 idx=0,0,0,0,1,1,1,1,1,1,1,1,1.4142135623730951,1.414213562373095
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) head | order <->(polygon,point) | (1,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) head | order <->(polygon,point) | (44,44) seq=0,0,0,0,0,1,1,1,1,1.4142135623730951,2,2,2,2,2.2360679774997 idx=0,0,0,0,1,1,1,1,1.4142135623730951,2,2,2,2,2.23606797749979,
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) head | order <->(polygon,point) | (99,99) seq=0,76.36753236814714,77.07788269017254,77.07788269017254,77.7 idx=76.36753236814714,77.07788269017254,77.07788269017254,77.781
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) head | order <->(polygon,point) | (NaN,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) only | order <->(polygon,point) | (1,NaN) seq=0 idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) only | order <->(polygon,point) | (NaN,NaN) seq=0 idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) sparse | order <->(polygon,point) | (10,10) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1.414213562373 idx=0,0,0,0,1,1,1,1,1,1,1,1,1.4142135623730951,1.414213562373095
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) sparse | order <->(polygon,point) | (1,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) sparse | order <->(polygon,point) | (44,44) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1.4142135623730951,2,2 idx=0,0,0,0,1,1,1,1,1.4142135623730951,2,2,2,2,2.23606797749979,
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) sparse | order <->(polygon,point) | (99,99) seq=0,0,0,0,0,0,0,0,0,0,0,76.36753236814714,77.07788269017254,77 idx=76.36753236814714,77.07788269017254,77.07788269017254,77.781
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) sparse | order <->(polygon,point) | (NaN,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) tail | order <->(polygon,point) | (10,10) seq=0,0,0,0,0,1,1,1,1,1,1,1,1,1.4142135623730951,1.4142135623730 idx=0,0,0,0,1,1,1,1,1,1,1,1,1.4142135623730951,1.414213562373095
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) tail | order <->(polygon,point) | (1,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) tail | order <->(polygon,point) | (44,44) seq=0,0,0,0,0,1,1,1,1,1.4142135623730951,2,2,2,2,2.2360679774997 idx=0,0,0,0,1,1,1,1,1.4142135623730951,2,2,2,2,2.23606797749979,
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) tail | order <->(polygon,point) | (99,99) seq=0,76.36753236814714,77.07788269017254,77.07788269017254,77.7 idx=76.36753236814714,77.07788269017254,77.07788269017254,77.781
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) tail | order <->(polygon,point) | (NaN,NaN) seq=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, idx=ERROR: index returned tuples in wrong order
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) head | search ~=(polygon,polygon) | ((NaN,NaN),(0,0),(1,1)) seq=1 idx=0
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) sparse | search ~=(polygon,polygon) | ((NaN,NaN),(0,0),(1,1)) seq=11 idx=0
spgist poly_ops | nan ((NaN,NaN),(0,0),(1,1)) tail | search ~=(polygon,polygon) | ((NaN,NaN),(0,0),(1,1)) seq=1 idx=0
===== minimal case: box <-> point on GiST, one box with NaN in one coordinate
CREATE TABLE b (v box);
INSERT INTO b VALUES ('(1,NaN),(0,0)');
INSERT INTO b SELECT box(point(x, y), point(x + 1, y + 1)) FROM generate_series(0, 44) x, generate_series(0, 44) y;
CREATE INDEX ON b USING gist (v);
SET enable_seqscan = off;
SELECT v <-> point '(0.5,0.5)' AS d FROM b ORDER BY v <-> point '(0.5,0.5)' LIMIT 3;
RESET enable_seqscan;
SET enable_indexscan = off; SET enable_bitmapscan = off;
SELECT v <-> point '(0.5,0.5)' AS d FROM b ORDER BY v <-> point '(0.5,0.5)' LIMIT 3;
-- index scan: ERROR: inconsistent point values (unpatched and v5, master and REL_18)
-- with --enable-cassert: TRAP: failed Assert("box->low.y <= box->high.y") in computeDistance()
-- seq scan: 0, 0.5, 0.5
===== v5-0002 on REL_18: the only conflict (palloc_object is not in 18)
121- unionL = (BOX *) palloc(sizeof(BOX));
122-- *unionL = *cur;
123:+ snapBox(unionL, cur);
124- }
125- else
--
130- unionR = (BOX *) palloc(sizeof(BOX));
131-- *unionR = *cur;
132:+ snapBox(unionR, cur);
133- }
134- else
===== nan_matrix.sql
-- Differential check for bug #19705: does an index on a geometric column
-- return the same rows as a sequential scan, when some stored values have a
-- NaN coordinate?
--
-- nan_setup() creates one table per (index kind, NaN value, NaN position),
-- each with exactly one index, so the planner has nothing else to choose.
-- nan_check(label) then runs, for every operator the operator class declares
-- in pg_amop, a count(*) with a sequential scan and with the index, and for
-- ordering operators it compares the first 50 distances. It records
-- whether EXPLAIN really used the index.
--
-- Setup and check are separate so that the indexes can be built by one
-- binary and searched by another (a minor upgrade, no REINDEX).
CREATE TABLE IF NOT EXISTS nan_case (
tbl text PRIMARY KEY,
typ text, -- box, point, polygon, circle
am text,
opclass text,
nanval text,
pos text, -- head, tail, only, sparse
err text -- error from CREATE INDEX, if any (then nothing to check)
);
CREATE OR REPLACE FUNCTION nan_setup() RETURNS int LANGUAGE plpgsql AS $$
DECLARE
k record; nv text; pos text; t text; n int := 0; finite text; nanexpr text; err text;
BEGIN
SET LOCAL client_min_messages = warning;
FOR k IN SELECT * FROM (VALUES
('box', 'brin', 'box_inclusion_ops'),
('box', 'gist', 'box_ops'),
('box', 'spgist', 'box_ops'),
('point', 'gist', 'point_ops'),
('point', 'spgist', 'quad_point_ops'),
('point', 'spgist', 'kd_point_ops'),
('polygon', 'gist', 'poly_ops'),
('polygon', 'spgist', 'poly_ops'),
('circle', 'gist', 'circle_ops')) AS x(typ, am, opclass)
LOOP
-- 45 x 45 = 2025 finite values spread over a grid, so GiST and
-- SP-GiST split into several levels and BRIN has many ranges.
finite := CASE k.typ
WHEN 'box' THEN 'box(point(x, y), point(x + 1, y + 1))'
WHEN 'point' THEN 'point(x, y)'
WHEN 'polygon' THEN 'polygon(box(point(x, y), point(x + 1, y + 1)))'
WHEN 'circle' THEN 'circle(point(x, y), 1)' END;
FOREACH nv IN ARRAY CASE k.typ
WHEN 'box' THEN ARRAY['(NaN,NaN),(0,0)', '(1,NaN),(0,0)', '(NaN,NaN),(NaN,NaN)']
WHEN 'point' THEN ARRAY['(NaN,NaN)', '(NaN,1)', '(1,NaN)']
WHEN 'polygon' THEN ARRAY['((NaN,NaN),(0,0),(1,1))', '((0,0),(NaN,1),(1,1))']
WHEN 'circle' THEN ARRAY['<(NaN,NaN),1>', '<(0,0),NaN>', '<(1,NaN),1>'] END
LOOP
nanexpr := format('%L::%s', nv, k.typ);
FOREACH pos IN ARRAY ARRAY['head', 'tail', 'only', 'sparse'] LOOP
n := n + 1;
t := format('nan_%s_%s', k.typ, n);
EXECUTE format('DROP TABLE IF EXISTS %I', t);
EXECUTE format('CREATE TABLE %I (id serial, v %s)', t, k.typ);
IF pos = 'head' THEN
EXECUTE format('INSERT INTO %I (v) VALUES (%s)', t, nanexpr);
END IF;
IF pos <> 'only' THEN
EXECUTE format('INSERT INTO %I (v) SELECT CASE WHEN %L = ''sparse'' AND (x * 45 + y) %% 200 = 0 '
'THEN %s ELSE %s END FROM generate_series(0, 44) x, generate_series(0, 44) y',
t, pos, nanexpr, finite);
END IF;
IF pos IN ('tail', 'only') THEN
EXECUTE format('INSERT INTO %I (v) VALUES (%s)', t, nanexpr);
END IF;
err := NULL;
BEGIN
EXECUTE format('CREATE INDEX %I ON %I USING %s (v %s)%s', t || '_idx', t, k.am, k.opclass,
CASE WHEN k.am = 'brin' THEN ' WITH (pages_per_range = 1)' ELSE '' END);
EXCEPTION WHEN OTHERS THEN
err := SQLERRM;
END;
EXECUTE format('ANALYZE %I', t);
INSERT INTO nan_case VALUES (t, k.typ, k.am, k.opclass, nv, pos, err)
ON CONFLICT (tbl) DO UPDATE SET typ = EXCLUDED.typ, am = EXCLUDED.am,
opclass = EXCLUDED.opclass, nanval = EXCLUDED.nanval, pos = EXCLUDED.pos,
err = EXCLUDED.err;
END LOOP;
END LOOP;
END LOOP;
RETURN n;
END $$;
CREATE TABLE IF NOT EXISTS nan_result (
label text, tbl text, op text, probe text, kind text,
seq text, idx text, used_index bool, same bool
);
CREATE OR REPLACE FUNCTION nan_probes(typ text) RETURNS text[] LANGUAGE sql IMMUTABLE AS $$
SELECT CASE typ
WHEN 'box' THEN ARRAY['(-2,-2),(2,2)', '(0,0),(1,1)', '(10,10),(12,12)', '(40,40),(46,46)',
'(100,100),(200,200)', '(NaN,NaN),(0,0)', '(-Infinity,-Infinity),(Infinity,Infinity)']
WHEN 'point' THEN ARRAY['(0.5,0.5)', '(10,10)', '(44,44)', '(99,99)', '(-1,-1)', '(NaN,NaN)', '(1,NaN)']
WHEN 'polygon' THEN ARRAY['((0,0),(0,20),(20,20),(20,0))', '((5,5),(5,6),(6,6))',
'((0,0),(0,1),(1,1),(1,0))', '((NaN,NaN),(0,0),(1,1))']
WHEN 'circle' THEN ARRAY['<(0,0),5>', '<(20,20),3>', '<(44,44),0.5>', '<(NaN,NaN),1>', '<(0,0),Infinity>']
END
$$;
CREATE OR REPLACE FUNCTION nan_check(p_label text) RETURNS TABLE (checks int, mismatches int, not_indexed int)
LANGUAGE plpgsql AS $$
DECLARE
c record; o record; pr text; q text; s_res text; i_res text; plan text; line text; used bool;
nchk int := 0; nbad int := 0; nnoidx int := 0;
BEGIN
DELETE FROM nan_result WHERE label = p_label;
FOR c IN SELECT * FROM nan_case WHERE err IS NULL ORDER BY tbl LOOP
FOR o IN
SELECT DISTINCT ao.amoppurpose AS purpose, ao.amopopr::regoperator AS opr,
op.oprname, format_type(op.oprright, NULL) AS rtype
FROM pg_opclass oc
JOIN pg_am am ON am.oid = oc.opcmethod
JOIN pg_amop ao ON ao.amopfamily = oc.opcfamily
JOIN pg_operator op ON op.oid = ao.amopopr
WHERE oc.opcname = c.opclass AND am.amname = c.am
AND format_type(op.oprleft, NULL) = c.typ
LOOP
-- Probes: the probe set of the operator's right-hand type, if we have one.
IF o.rtype NOT IN ('box', 'point', 'polygon', 'circle') THEN
CONTINUE;
END IF;
FOREACH pr IN ARRAY nan_probes(o.rtype) LOOP
IF o.purpose = 's' THEN
q := format('SELECT count(*)::text FROM %I WHERE v %s %L::%s', c.tbl, o.oprname, pr, o.rtype);
ELSE
q := format('SELECT string_agg(d::text, '','' ORDER BY rn) FROM (SELECT d, row_number() OVER () rn '
'FROM (SELECT v %s %L::%s AS d FROM %I ORDER BY v %s %L::%s LIMIT 50) a) b',
o.oprname, pr, o.rtype, c.tbl, o.oprname, pr, o.rtype);
END IF;
PERFORM set_config('enable_seqscan', 'on', true);
PERFORM set_config('enable_indexscan', 'off', true);
PERFORM set_config('enable_bitmapscan', 'off', true);
BEGIN
EXECUTE q INTO s_res;
EXCEPTION WHEN OTHERS THEN
s_res := 'ERROR: ' || SQLERRM;
END;
PERFORM set_config('enable_seqscan', 'off', true);
PERFORM set_config('enable_indexscan', 'on', true);
PERFORM set_config('enable_bitmapscan', 'on', true);
-- Leaves the last index query in the server log, in case it crashes.
RAISE LOG 'nan_check % % [nan % %]: %', c.am, c.opclass, c.nanval, c.pos, q;
plan := '';
FOR line IN EXECUTE 'EXPLAIN (COSTS OFF) ' || q LOOP
plan := plan || line || E'\n';
END LOOP;
used := plan LIKE '%' || c.tbl || '_idx%';
BEGIN
EXECUTE q INTO i_res;
EXCEPTION WHEN OTHERS THEN
i_res := 'ERROR: ' || SQLERRM;
END;
nchk := nchk + 1;
IF NOT used THEN nnoidx := nnoidx + 1; END IF;
IF s_res IS DISTINCT FROM i_res THEN nbad := nbad + 1; END IF;
INSERT INTO nan_result VALUES (p_label, c.tbl, o.opr::text, pr,
CASE o.purpose WHEN 's' THEN 'search' ELSE 'order' END,
s_res, i_res, used, s_res IS NOT DISTINCT FROM i_res);
END LOOP;
END LOOP;
END LOOP;
RETURN QUERY SELECT nchk, nbad, nnoidx;
END $$;
===== run.sh
#!/bin/bash
# Runs nan_matrix.sql.
# run.sh fresh BUILD initdb, nan_setup() and nan_check() with BUILD
# run.sh upgrade OLD NEW nan_setup() with OLD, then nan_check() with NEW
# on the same data directory (a minor upgrade)
# Writes <label>.summary.txt and <label>.mismatch.txt next to this script.
set -eu
A=$(cd "$(dirname "$0")" && pwd)
W=$HOME/pg19705nan
MODE=$1; shift
if [ $MODE = fresh ]; then SETUP=$1; CHECK=$1; LABEL=$1; else SETUP=$1; CHECK=$2; LABEL=$1-to-$2; fi
D=$(mktemp -d /tmp/claude-1000/nan.XXXX); P=$((55800 + RANDOM % 100))
start() { $W/i-$1/bin/pg_ctl -D $D -l $D/log.$1 -o "-p $P -c unix_socket_directories=/tmp" -w start >/dev/null; }
stop() { $W/i-$1/bin/pg_ctl -D $D -m fast -w stop >/dev/null; }
q() { $W/i-$1/bin/psql -X -q -v ON_ERROR_STOP=1 -h /tmp -p $P -d postgres "${@:2}"; }
$W/i-$SETUP/bin/initdb -D $D -A trust --no-sync >/dev/null
start $SETUP
q $SETUP -f $A/nan_matrix.sql >/dev/null
q $SETUP -Atc "SELECT 'tables: ' || nan_setup()"
q $SETUP -Atc "SELECT format('CREATE INDEX error: %s %s | nan %s %s | %s', am, opclass, nanval, pos, err)
FROM nan_case WHERE err IS NOT NULL ORDER BY tbl" | tee $A/$LABEL.createerr.txt
if [ $CHECK != $SETUP ]; then stop $SETUP; start $CHECK; fi
q $CHECK -Atc "SELECT version()" | cut -c1-60
q $CHECK -Atc "SELECT format('checks %s, mismatches %s, index not used %s', checks, mismatches, not_indexed) FROM nan_check('$LABEL')" \
| tee $A/$LABEL.summary.txt
q $CHECK -Atc "
SELECT c.am || ' ' || c.opclass AS index, count(*) FILTER (WHERE NOT r.same) AS wrong, count(*) AS checks
FROM nan_result r JOIN nan_case c USING (tbl)
WHERE r.label = '$LABEL' GROUP BY 1 ORDER BY 1" | tee -a $A/$LABEL.summary.txt
q $CHECK -Atc "
SELECT format('%s %s | nan %s %s | %s %s | %s seq=%s idx=%s%s', c.am, c.opclass, c.nanval, c.pos,
r.kind, r.op, r.probe, left(r.seq, 60), left(r.idx, 60),
CASE WHEN r.used_index THEN '' ELSE ' (index not used)' END)
FROM nan_result r JOIN nan_case c USING (tbl)
WHERE r.label = '$LABEL' AND NOT r.same
ORDER BY c.am, c.opclass, r.op, c.nanval, c.pos, r.probe" > $A/$LABEL.mismatch.txt
echo "mismatch lines: $(wc -l < $A/$LABEL.mismatch.txt)"
stop $CHECK
# rm -rf $D (kept for the logs)
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows
2026-09-19 17:37 BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows PG Bug reporting form <noreply@postgresql.org>
2026-09-21 23:49 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
2026-09-22 12:12 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Kirill Reshke <reshkekirill@gmail.com>
2026-09-22 12:37 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Andrey Borodin <x4mmm@yandex-team.ru>
2026-09-23 03:43 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
2026-09-23 17:51 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Kirill Reshke <reshkekirill@gmail.com>
2026-09-24 02:43 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
2026-09-24 06:22 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Kirill Reshke <reshkekirill@gmail.com>
2026-09-25 04:29 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows shihao zhong <zhong950419@gmail.com>
2026-09-26 00:09 ` Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows Manu <manuelreyesbravo@gmail.com>
@ 2026-09-28 07:01 ` shihao zhong <zhong950419@gmail.com>
0 siblings, 0 replies; 11+ messages in thread
From: shihao zhong @ 2026-09-28 07:01 UTC (permalink / raw)
To: Manu <manuelreyesbravo@gmail.com>; +Cc: Kirill Reshke <reshkekirill@gmail.com>; Andrey Borodin <x4mmm@yandex-team.ru>; kehan5800@gmail.com; pgsql-bugs@lists.postgresql.org
Hi Manu,
Thanks for the matrix.
> Most of what is left is ordering: the index scan fails with
> "inconsistent point values"
computeDistance() assumes low <= high and a query point without NaN.
So this doesn't need NaN rows. On a table with only finite values,
ORDER BY v <-> point '(1,NaN)' fails for all four GiST opclasses.
v6-0004 gives distance 0 to keys and query points with a NaN, as 0002
already does for internal keys. A box_ops leaf must be exact, and
index-only scans can't recheck, so it calls the operator instead.
With your script, GiST has 9 mismatches left on master, all point <@
polygon with a NaN vertex. GiST indexes built by unpatched master give
the same.
0001 to 0003 are v5. 0003 is Kirill's polygon and circle patch after
pgindent.
> palloc_object(), which 18 does not have
18 has it. The conflict comes from 1b105f9472b, which changed those
lines in 19.
> I have not checked whether the patch there covers NaN too.
It doesn't. The v2 there leaves NaN on the same error on purpose.
Regards,
Shihao
Attachments:
[application/octet-stream] v6-0002-Fix-GiST-box-indexes-hiding-rows-next-to-a-NaN-bo.patch (13.8K, ../../CAGRkXqSN7vB_h41_jvsBbDb2sVNW=rmjxpndvL70oCDrYhbh=w@mail.gmail.com/3-v6-0002-Fix-GiST-box-indexes-hiding-rows-next-to-a-NaN-bo.patch)
download | inline diff:
From 946ad0c942371438c2797bd2207be5f908ecb462 Mon Sep 17 00:00:00 2001
From: Shihao <zhong950419@gmail.com>
Date: Wed, 23 Sep 2026 21:23:52 -0400
Subject: [PATCH v6 2/4] Fix GiST box indexes hiding rows next to a NaN box
GiST union keys kept NaN coordinates, and no box comparison is true
against a NaN bound. So an internal key covering a NaN box hid its
whole subtree from searches, and KNN scans could return rows from it
too late.
Map NaNs to infinities when building union keys. Internal keys that
still have a NaN, from indexes built before this fix, now match every
search and get distance zero. So those indexes return correct results
without a REINDEX, only less selectively. Also stop internal pages
from pruning ~= when the query has a NaN, since box_same() matches it.
For points, ~= also missed NaN queries at the leaf level, because it
compared with FPeq() while point_eq() compares exactly when there is a
NaN. Make the index agree with point_eq().
Author: Kirill Reshke <reshkekirill@gmail.com>
Author: Shihao Zhong <zhong950419@gmail.com>
Reported-by: Ke <kehan5800@gmail.com>
Reported-by: Andrey Borodin <x4mmm@yandex-team.ru>
Discussion: https://postgr.es/m/19705-548fda77321e062d@postgresql.org
Backpatch-through: 14
---
src/backend/access/gist/gistproc.c | 139 ++++++++++++++++++++++++-----
src/test/regress/expected/gist.out | 73 +++++++++++++++
src/test/regress/sql/gist.sql | 35 ++++++++
3 files changed, 225 insertions(+), 22 deletions(-)
diff --git a/src/backend/access/gist/gistproc.c b/src/backend/access/gist/gistproc.c
index f1044f49d6c..0f405d4450f 100644
--- a/src/backend/access/gist/gistproc.c
+++ b/src/backend/access/gist/gistproc.c
@@ -43,6 +43,45 @@ static bool gist_bbox_zorder_abbrev_abort(int memtupcount, SortSupport ssup);
/* Minimum accepted ratio of split */
#define LIMIT_RATIO 0.3
+/*
+ * Does the box have a NaN coordinate?
+ *
+ * No box comparison is true against a NaN, so a union key with one would
+ * hide everything below it. Union keys map NaNs to infinities instead (see
+ * adjustBox), but indexes built before that was done can still have NaN
+ * internal keys, and searches must descend into those.
+ */
+static inline bool
+box_has_nan(const BOX *box)
+{
+ return isnan(box->high.x) || isnan(box->high.y) ||
+ isnan(box->low.x) || isnan(box->low.y);
+}
+
+/* Is the entry an internal key with a NaN coordinate? */
+static inline bool
+nan_internal_key(const GISTENTRY *entry)
+{
+ return !GIST_LEAF(entry) && box_has_nan(DatumGetBoxP(entry->key));
+}
+
+/* float8_max() and float8_min(), but a NaN gives the matching infinity */
+static inline float8
+bound_max(float8 val1, float8 val2)
+{
+ if (isnan(val1) || isnan(val2))
+ return get_float8_infinity();
+ return float8_max(val1, val2);
+}
+
+static inline float8
+bound_min(float8 val1, float8 val2)
+{
+ if (isnan(val1) || isnan(val2))
+ return -get_float8_infinity();
+ return float8_min(val1, val2);
+}
+
/**************************************************
* Box ops
@@ -126,6 +165,10 @@ gist_box_consistent(PG_FUNCTION_ARGS)
if (DatumGetBoxP(entry->key) == NULL || query == NULL)
PG_RETURN_BOOL(false);
+ /* A NaN internal key tells us nothing, see box_has_nan() */
+ if (nan_internal_key(entry))
+ PG_RETURN_BOOL(true);
+
/*
* if entry is not leaf, use rtree_internal_consistent, else use
* gist_box_leaf_consistent
@@ -141,19 +184,31 @@ gist_box_consistent(PG_FUNCTION_ARGS)
}
/*
- * Increase BOX b to include addon.
+ * Increase BOX b to include addon. A NaN in either box grows b to the
+ * matching infinity.
*/
static void
adjustBox(BOX *b, const BOX *addon)
{
- if (float8_lt(b->high.x, addon->high.x))
- b->high.x = addon->high.x;
- if (float8_gt(b->low.x, addon->low.x))
- b->low.x = addon->low.x;
- if (float8_lt(b->high.y, addon->high.y))
- b->high.y = addon->high.y;
- if (float8_gt(b->low.y, addon->low.y))
- b->low.y = addon->low.y;
+ b->high.x = bound_max(b->high.x, addon->high.x);
+ b->low.x = bound_min(b->low.x, addon->low.x);
+ b->high.y = bound_max(b->high.y, addon->high.y);
+ b->low.y = bound_min(b->low.y, addon->low.y);
+}
+
+/* Copy a BOX, mapping NaNs to infinities. */
+static void
+snapBox(BOX *b, const BOX *box)
+{
+ *b = *box;
+ if (isnan(b->high.x))
+ b->high.x = get_float8_infinity();
+ if (isnan(b->high.y))
+ b->high.y = get_float8_infinity();
+ if (isnan(b->low.x))
+ b->low.x = -get_float8_infinity();
+ if (isnan(b->low.y))
+ b->low.y = -get_float8_infinity();
}
/*
@@ -174,7 +229,7 @@ gist_box_union(PG_FUNCTION_ARGS)
numranges = entryvec->n;
pageunion = palloc_object(BOX);
cur = DatumGetBoxP(entryvec->vector[0].key);
- memcpy(pageunion, cur, sizeof(BOX));
+ snapBox(pageunion, cur);
for (i = 1; i < numranges; i++)
{
@@ -239,7 +294,7 @@ fallbackSplit(GistEntryVector *entryvec, GIST_SPLITVEC *v)
if (unionL == NULL)
{
unionL = palloc_object(BOX);
- *unionL = *cur;
+ snapBox(unionL, cur);
}
else
adjustBox(unionL, cur);
@@ -252,7 +307,7 @@ fallbackSplit(GistEntryVector *entryvec, GIST_SPLITVEC *v)
if (unionR == NULL)
{
unionR = palloc_object(BOX);
- *unionR = *cur;
+ snapBox(unionR, cur);
}
else
adjustBox(unionR, cur);
@@ -526,7 +581,7 @@ gist_box_picksplit(PG_FUNCTION_ARGS)
{
box = DatumGetBoxP(entryvec->vector[i].key);
if (i == FirstOffsetNumber)
- context.boundingBox = *box;
+ snapBox(&context.boundingBox, box);
else
adjustBox(&context.boundingBox, box);
}
@@ -715,7 +770,7 @@ gist_box_picksplit(PG_FUNCTION_ARGS)
if (v->spl_nleft > 0) \
adjustBox(leftBox, box); \
else \
- *leftBox = *(box); \
+ snapBox(leftBox, box); \
v->spl_left[v->spl_nleft++] = off; \
} while(0)
@@ -724,7 +779,7 @@ gist_box_picksplit(PG_FUNCTION_ARGS)
if (v->spl_nright > 0) \
adjustBox(rightBox, box); \
else \
- *rightBox = *(box); \
+ snapBox(rightBox, box); \
v->spl_right[v->spl_nright++] = off; \
} while(0)
@@ -959,6 +1014,13 @@ rtree_internal_consistent(BOX *key, BOX *query, StrategyNumber strategy)
{
bool retval;
+ /*
+ * A NaN query can still match with ~=, since box_same() treats NaNs as
+ * equal, but box_contain() never matches it.
+ */
+ if (strategy == RTSameStrategyNumber && box_has_nan(query))
+ return true;
+
switch (strategy)
{
case RTLeftStrategyNumber:
@@ -1077,6 +1139,10 @@ gist_poly_consistent(PG_FUNCTION_ARGS)
if (DatumGetBoxP(entry->key) == NULL || query == NULL)
PG_RETURN_BOOL(false);
+ /* A NaN internal key tells us nothing, see box_has_nan() */
+ if (nan_internal_key(entry))
+ PG_RETURN_BOOL(true);
+
/*
* Since the operators require recheck anyway, we can just use
* rtree_internal_consistent even at leaf nodes. (This works in part
@@ -1147,6 +1213,10 @@ gist_circle_consistent(PG_FUNCTION_ARGS)
if (DatumGetBoxP(entry->key) == NULL || query == NULL)
PG_RETURN_BOOL(false);
+ /* A NaN internal key tells us nothing, see box_has_nan() */
+ if (nan_internal_key(entry))
+ PG_RETURN_BOOL(true);
+
/*
* Since the operators require recheck anyway, we can just use
* rtree_internal_consistent even at leaf nodes. (This works in part
@@ -1307,7 +1377,20 @@ gist_point_consistent_internal(StrategyNumber strategy,
result = FPlt(key->low.y, query->y);
break;
case RTSameStrategyNumber:
- if (isLeaf)
+
+ /*
+ * point_eq() compares exactly when there is a NaN, so a NaN query
+ * matches points with the same NaN coordinates. Internal keys
+ * cannot tell us where those are, so search everything below.
+ */
+ if (isnan(query->x) || isnan(query->y))
+ {
+ /* key.high must equal key.low, so we can disregard it */
+ result = !isLeaf ||
+ (float8_eq(key->low.x, query->x) &&
+ float8_eq(key->low.y, query->y));
+ }
+ else if (isLeaf)
{
/* key.high must equal key.low, so we can disregard it */
result = (FPeq(key->low.x, query->x) &&
@@ -1345,6 +1428,10 @@ gist_point_consistent(PG_FUNCTION_ARGS)
bool result;
StrategyNumber strategyGroup;
+ /* A NaN internal key tells us nothing, see box_has_nan() */
+ if (nan_internal_key(entry))
+ PG_RETURN_BOOL(true);
+
/*
* We have to remap these strategy numbers to get this klugy
* classification logic to work.
@@ -1465,9 +1552,13 @@ gist_point_distance(PG_FUNCTION_ARGS)
switch (strategyGroup)
{
case PointStrategyNumberGroup:
- distance = computeDistance(GIST_LEAF(entry),
- DatumGetBoxP(entry->key),
- PG_GETARG_POINT_P(1));
+ /* A NaN internal key tells us nothing, see box_has_nan() */
+ if (nan_internal_key(entry))
+ distance = 0.0;
+ else
+ distance = computeDistance(GIST_LEAF(entry),
+ DatumGetBoxP(entry->key),
+ PG_GETARG_POINT_P(1));
break;
default:
elog(ERROR, "unrecognized strategy number: %d", strategy);
@@ -1487,9 +1578,13 @@ gist_bbox_distance(GISTENTRY *entry, Datum query, StrategyNumber strategy)
switch (strategyGroup)
{
case PointStrategyNumberGroup:
- distance = computeDistance(false,
- DatumGetBoxP(entry->key),
- DatumGetPointP(query));
+ /* A NaN internal key tells us nothing, see box_has_nan() */
+ if (nan_internal_key(entry))
+ distance = 0.0;
+ else
+ distance = computeDistance(false,
+ DatumGetBoxP(entry->key),
+ DatumGetPointP(query));
break;
default:
elog(ERROR, "unrecognized strategy number: %d", strategy);
diff --git a/src/test/regress/expected/gist.out b/src/test/regress/expected/gist.out
index ac79f94aa80..174f8b4251f 100644
--- a/src/test/regress/expected/gist.out
+++ b/src/test/regress/expected/gist.out
@@ -463,3 +463,76 @@ create index gist_tbl_box_index on gist_tbl using gist (b);
insert into gist_tbl
select box(point(0.05*i, 0.05*i)) from generate_series(0,10) as i;
drop table gist_tbl;
+-- a box with a NaN coordinate must not poison the union keys (bug #19705)
+create table gist_nan_tbl (b box);
+insert into gist_nan_tbl select box '(1,1),(2,2)' from generate_series(1, 1000);
+insert into gist_nan_tbl values (box '(NaN,NaN),(0,0)');
+create index gist_nan_tbl_index on gist_nan_tbl using gist (b);
+-- also insert rows after the build, to exercise the insertion path
+insert into gist_nan_tbl select box '(3,3),(4,4)' from generate_series(1, 1000);
+insert into gist_nan_tbl values (box '(NaN,NaN),(0,0)');
+set enable_seqscan = off;
+set enable_bitmapscan = off;
+select count(*) from gist_nan_tbl where b && box(point(0,0), point(5,5));
+ count
+-------
+ 2000
+(1 row)
+
+select count(*) from gist_nan_tbl where b <@ box(point(0,0), point(5,5));
+ count
+-------
+ 2000
+(1 row)
+
+select count(*) from gist_nan_tbl where b @> point '(1.5,1.5)';
+ count
+-------
+ 1000
+(1 row)
+
+select count(*) from gist_nan_tbl where b ~= box '(NaN,NaN),(0,0)';
+ count
+-------
+ 2
+(1 row)
+
+reset enable_seqscan;
+reset enable_bitmapscan;
+drop table gist_nan_tbl;
+-- point ~= matches NaN coordinates exactly, the index must agree
+create table gist_nan_point_tbl (p point);
+insert into gist_nan_point_tbl
+ select point(i % 100, i / 100) from generate_series(0, 9999) i;
+insert into gist_nan_point_tbl
+ values ('(NaN,NaN)'), ('(NaN,3)'), ('(3,NaN)'), ('(NaN,NaN)');
+create index gist_nan_point_tbl_index on gist_nan_point_tbl using gist (p);
+set enable_seqscan = off;
+set enable_bitmapscan = off;
+select count(*) from gist_nan_point_tbl where p ~= point '(NaN,NaN)';
+ count
+-------
+ 2
+(1 row)
+
+select count(*) from gist_nan_point_tbl where p ~= point '(NaN,3)';
+ count
+-------
+ 1
+(1 row)
+
+select count(*) from gist_nan_point_tbl where p ~= point '(3,NaN)';
+ count
+-------
+ 1
+(1 row)
+
+select count(*) from gist_nan_point_tbl where p ~= point '(3,3)';
+ count
+-------
+ 1
+(1 row)
+
+reset enable_seqscan;
+reset enable_bitmapscan;
+drop table gist_nan_point_tbl;
diff --git a/src/test/regress/sql/gist.sql b/src/test/regress/sql/gist.sql
index 57dcc082450..654318fb4db 100644
--- a/src/test/regress/sql/gist.sql
+++ b/src/test/regress/sql/gist.sql
@@ -236,3 +236,38 @@ create index gist_tbl_box_index on gist_tbl using gist (b);
insert into gist_tbl
select box(point(0.05*i, 0.05*i)) from generate_series(0,10) as i;
drop table gist_tbl;
+
+-- a box with a NaN coordinate must not poison the union keys (bug #19705)
+create table gist_nan_tbl (b box);
+insert into gist_nan_tbl select box '(1,1),(2,2)' from generate_series(1, 1000);
+insert into gist_nan_tbl values (box '(NaN,NaN),(0,0)');
+create index gist_nan_tbl_index on gist_nan_tbl using gist (b);
+-- also insert rows after the build, to exercise the insertion path
+insert into gist_nan_tbl select box '(3,3),(4,4)' from generate_series(1, 1000);
+insert into gist_nan_tbl values (box '(NaN,NaN),(0,0)');
+set enable_seqscan = off;
+set enable_bitmapscan = off;
+select count(*) from gist_nan_tbl where b && box(point(0,0), point(5,5));
+select count(*) from gist_nan_tbl where b <@ box(point(0,0), point(5,5));
+select count(*) from gist_nan_tbl where b @> point '(1.5,1.5)';
+select count(*) from gist_nan_tbl where b ~= box '(NaN,NaN),(0,0)';
+reset enable_seqscan;
+reset enable_bitmapscan;
+drop table gist_nan_tbl;
+
+-- point ~= matches NaN coordinates exactly, the index must agree
+create table gist_nan_point_tbl (p point);
+insert into gist_nan_point_tbl
+ select point(i % 100, i / 100) from generate_series(0, 9999) i;
+insert into gist_nan_point_tbl
+ values ('(NaN,NaN)'), ('(NaN,3)'), ('(3,NaN)'), ('(NaN,NaN)');
+create index gist_nan_point_tbl_index on gist_nan_point_tbl using gist (p);
+set enable_seqscan = off;
+set enable_bitmapscan = off;
+select count(*) from gist_nan_point_tbl where p ~= point '(NaN,NaN)';
+select count(*) from gist_nan_point_tbl where p ~= point '(NaN,3)';
+select count(*) from gist_nan_point_tbl where p ~= point '(3,NaN)';
+select count(*) from gist_nan_point_tbl where p ~= point '(3,3)';
+reset enable_seqscan;
+reset enable_bitmapscan;
+drop table gist_nan_point_tbl;
--
2.37.1 (Apple Git-137.1)
[application/octet-stream] v6-0004-Fix-GiST-KNN-searches-with-a-NaN-in-a-key-or-the-.patch (6.7K, ../../CAGRkXqSN7vB_h41_jvsBbDb2sVNW=rmjxpndvL70oCDrYhbh=w@mail.gmail.com/4-v6-0004-Fix-GiST-KNN-searches-with-a-NaN-in-a-key-or-the-.patch)
download | inline diff:
From 6ce3c4bcccffb9b093567884ae41a103972d9e30 Mon Sep 17 00:00:00 2001
From: Shihao <zhong950419@gmail.com>
Date: Sun, 27 Sep 2026 09:26:57 -0700
Subject: [PATCH v6 4/4] Fix GiST KNN searches with a NaN in a key or the query
point
computeDistance() cannot place a point next to a box when either has a
NaN. It raised "inconsistent point values", failed an assertion, or
ranked the row too early. Use distance 0 there instead. Leaf keys of
box_ops must be exact, so they use the operator.
Reported-by: Manu <manuelreyesbravo@gmail.com>
Discussion: https://postgr.es/m/19705-548fda77321e062d@postgresql.org
---
src/backend/access/gist/gistproc.c | 30 ++++++++++++--
src/test/regress/expected/gist.out | 65 ++++++++++++++++++++++++++++++
src/test/regress/sql/gist.sql | 29 +++++++++++++
3 files changed, 120 insertions(+), 4 deletions(-)
diff --git a/src/backend/access/gist/gistproc.c b/src/backend/access/gist/gistproc.c
index f9160dc2bee..d45cc4b310d 100644
--- a/src/backend/access/gist/gistproc.c
+++ b/src/backend/access/gist/gistproc.c
@@ -1356,6 +1356,17 @@ computeDistance(bool isLeaf, BOX *box, Point *point)
return result;
}
+/*
+ * computeDistance() places the point in one of the regions around the box,
+ * which a NaN in either of them makes impossible. Callers use 0 instead, the
+ * only lower bound that is always safe.
+ */
+static inline bool
+distance_has_nan(const BOX *box, const Point *point)
+{
+ return box_has_nan(box) || isnan(point->x) || isnan(point->y);
+}
+
static bool
gist_point_consistent_internal(StrategyNumber strategy,
bool isLeaf, BOX *key, Point *query)
@@ -1566,8 +1577,10 @@ gist_point_distance(PG_FUNCTION_ARGS)
switch (strategyGroup)
{
case PointStrategyNumberGroup:
- /* A NaN internal key tells us nothing, see box_has_nan() */
- if (nan_internal_key(entry))
+ /* Leaf keys are points, which point_distance() handles */
+ if (!GIST_LEAF(entry) &&
+ distance_has_nan(DatumGetBoxP(entry->key),
+ PG_GETARG_POINT_P(1)))
distance = 0.0;
else
distance = computeDistance(GIST_LEAF(entry),
@@ -1592,8 +1605,8 @@ gist_bbox_distance(GISTENTRY *entry, Datum query, StrategyNumber strategy)
switch (strategyGroup)
{
case PointStrategyNumberGroup:
- /* A NaN internal key tells us nothing, see box_has_nan() */
- if (nan_internal_key(entry))
+ if (distance_has_nan(DatumGetBoxP(entry->key),
+ DatumGetPointP(query)))
distance = 0.0;
else
distance = computeDistance(false,
@@ -1622,6 +1635,15 @@ gist_box_distance(PG_FUNCTION_ARGS)
distance = gist_bbox_distance(entry, query, strategy);
+ /*
+ * A leaf key is the indexed box, so its distance must be exact. 0 is
+ * not, and index-only scans cannot recheck, so ask the operator.
+ */
+ if (GIST_LEAF(entry) &&
+ distance_has_nan(DatumGetBoxP(entry->key), DatumGetPointP(query)))
+ distance = DatumGetFloat8(DirectFunctionCall2(dist_bp,
+ entry->key, query));
+
PG_RETURN_FLOAT8(distance);
}
diff --git a/src/test/regress/expected/gist.out b/src/test/regress/expected/gist.out
index e7f1a27f866..c6e2c00396b 100644
--- a/src/test/regress/expected/gist.out
+++ b/src/test/regress/expected/gist.out
@@ -550,3 +550,68 @@ select count(*) from gist_nan_point_tbl where p <@ circle '<(50,50),50>';
reset enable_seqscan;
reset enable_bitmapscan;
drop table gist_nan_point_tbl;
+-- KNN with a NaN in a leaf key or in the query point
+create table gist_nan_knn_tbl (b box, p polygon, pt point);
+insert into gist_nan_knn_tbl
+ select b, polygon(b), center(b)
+ from (select box(point(i, j), point(i + 1, j + 1)) as b
+ from generate_series(0, 29) i, generate_series(0, 29) j) s;
+insert into gist_nan_knn_tbl values
+ ('(NaN,NaN),(0,0)', '((NaN,NaN),(0,0),(1,1))', '(NaN,NaN)'),
+ ('(1,NaN),(0,0)', '((0,0),(NaN,1),(1,1))', '(1,NaN)');
+create index gist_nan_knn_tbl_b on gist_nan_knn_tbl using gist (b);
+create index gist_nan_knn_tbl_p on gist_nan_knn_tbl using gist (p);
+create index gist_nan_knn_tbl_pt on gist_nan_knn_tbl using gist (pt);
+set enable_seqscan = off;
+set enable_bitmapscan = off;
+select b <-> point '(0.5,0.5)' from gist_nan_knn_tbl
+ order by b <-> point '(0.5,0.5)' limit 5;
+ ?column?
+--------------------
+ 0
+ 0.5
+ 0.5
+ 0.5
+ 0.7071067811865476
+(5 rows)
+
+-- the NaN distance sorts last
+select b <-> point '(0.5,0.5)' from gist_nan_knn_tbl
+ order by b <-> point '(0.5,0.5)' offset 901;
+ ?column?
+----------
+ NaN
+(1 row)
+
+select p <-> point '(0.5,0.5)' from gist_nan_knn_tbl
+ order by p <-> point '(0.5,0.5)' limit 5;
+ ?column?
+----------
+ 0
+ 0
+ 0
+ 0.5
+ 0.5
+(5 rows)
+
+select b <-> point '(1,NaN)' from gist_nan_knn_tbl
+ order by b <-> point '(1,NaN)' limit 3;
+ ?column?
+----------
+ NaN
+ NaN
+ NaN
+(3 rows)
+
+select pt <-> point '(1,NaN)' from gist_nan_knn_tbl
+ order by pt <-> point '(1,NaN)' limit 3;
+ ?column?
+----------
+ NaN
+ NaN
+ NaN
+(3 rows)
+
+reset enable_seqscan;
+reset enable_bitmapscan;
+drop table gist_nan_knn_tbl;
diff --git a/src/test/regress/sql/gist.sql b/src/test/regress/sql/gist.sql
index b9df6490ca5..b880da19a42 100644
--- a/src/test/regress/sql/gist.sql
+++ b/src/test/regress/sql/gist.sql
@@ -275,3 +275,32 @@ select count(*) from gist_nan_point_tbl where p <@ circle '<(50,50),50>';
reset enable_seqscan;
reset enable_bitmapscan;
drop table gist_nan_point_tbl;
+
+-- KNN with a NaN in a leaf key or in the query point
+create table gist_nan_knn_tbl (b box, p polygon, pt point);
+insert into gist_nan_knn_tbl
+ select b, polygon(b), center(b)
+ from (select box(point(i, j), point(i + 1, j + 1)) as b
+ from generate_series(0, 29) i, generate_series(0, 29) j) s;
+insert into gist_nan_knn_tbl values
+ ('(NaN,NaN),(0,0)', '((NaN,NaN),(0,0),(1,1))', '(NaN,NaN)'),
+ ('(1,NaN),(0,0)', '((0,0),(NaN,1),(1,1))', '(1,NaN)');
+create index gist_nan_knn_tbl_b on gist_nan_knn_tbl using gist (b);
+create index gist_nan_knn_tbl_p on gist_nan_knn_tbl using gist (p);
+create index gist_nan_knn_tbl_pt on gist_nan_knn_tbl using gist (pt);
+set enable_seqscan = off;
+set enable_bitmapscan = off;
+select b <-> point '(0.5,0.5)' from gist_nan_knn_tbl
+ order by b <-> point '(0.5,0.5)' limit 5;
+-- the NaN distance sorts last
+select b <-> point '(0.5,0.5)' from gist_nan_knn_tbl
+ order by b <-> point '(0.5,0.5)' offset 901;
+select p <-> point '(0.5,0.5)' from gist_nan_knn_tbl
+ order by p <-> point '(0.5,0.5)' limit 5;
+select b <-> point '(1,NaN)' from gist_nan_knn_tbl
+ order by b <-> point '(1,NaN)' limit 3;
+select pt <-> point '(1,NaN)' from gist_nan_knn_tbl
+ order by pt <-> point '(1,NaN)' limit 3;
+reset enable_seqscan;
+reset enable_bitmapscan;
+drop table gist_nan_knn_tbl;
--
2.37.1 (Apple Git-137.1)
[application/octet-stream] v6-REL_18-0001-Fix-BRIN-box_inclusion_ops-hiding-rows-next-to-a-.patch.nocfbot (5.6K, ../../CAGRkXqSN7vB_h41_jvsBbDb2sVNW=rmjxpndvL70oCDrYhbh=w@mail.gmail.com/5-v6-REL_18-0001-Fix-BRIN-box_inclusion_ops-hiding-rows-next-to-a-.patch.nocfbot)
download
[application/octet-stream] v6-0003-Check-for-NaN-point-in-GiST-polygon-and-circle-se.patch (6.2K, ../../CAGRkXqSN7vB_h41_jvsBbDb2sVNW=rmjxpndvL70oCDrYhbh=w@mail.gmail.com/6-v6-0003-Check-for-NaN-point-in-GiST-polygon-and-circle-se.patch)
download | inline diff:
From 8a8de919985069266631f9a0a32b9589f26cb3a8 Mon Sep 17 00:00:00 2001
From: reshke <reshke@double.cloud>
Date: Thu, 24 Sep 2026 08:10:07 +0300
Subject: [PATCH v6 3/4] Check for NaN point in GiST polygon and circle
searches.
The polygon/circle strategy groups of GiST consistent function check
leaf points with its bounding box and fast-filter check.
Points with NaN coordinate always fails, so such points were not returned by index
serach while poly_contain_pt/circle_contain_pt would actaully match them.
This was actaully the case in regression test - "SELECT count(*) FROM point_tbl WHERE f1
<@ polygon ..." in create_index.sql has answered 5 all along while the
index run answered 4.
Also asserts has been updated with NaN check.
Per BUG 19705
Discussion: https://postgr.es/m/19705-548fda77321e062d@postgresql.org
---
src/backend/access/gist/gistproc.c | 42 ++++++++++++++--------
src/test/regress/expected/create_index.out | 2 +-
src/test/regress/expected/gist.out | 14 ++++++++
src/test/regress/sql/gist.sql | 4 +++
4 files changed, 47 insertions(+), 15 deletions(-)
diff --git a/src/backend/access/gist/gistproc.c b/src/backend/access/gist/gistproc.c
index 0f405d4450f..f9160dc2bee 100644
--- a/src/backend/access/gist/gistproc.c
+++ b/src/backend/access/gist/gistproc.c
@@ -1482,11 +1482,16 @@ gist_point_consistent(PG_FUNCTION_ARGS)
{
POLYGON *query = PG_GETARG_POLYGON_P(1);
- result = DatumGetBool(DirectFunctionCall5(gist_poly_consistent,
- PointerGetDatum(entry),
- PolygonPGetDatum(query),
- Int16GetDatum(RTOverlapStrategyNumber),
- 0, PointerGetDatum(recheck)));
+ /*
+ * A NaN point fails the bounding-box prefilter, though the
+ * exact operator below may still match it, as on the heap.
+ */
+ result = box_has_nan(DatumGetBoxP(entry->key)) ||
+ DatumGetBool(DirectFunctionCall5(gist_poly_consistent,
+ PointerGetDatum(entry),
+ PolygonPGetDatum(query),
+ Int16GetDatum(RTOverlapStrategyNumber),
+ 0, PointerGetDatum(recheck)));
if (GIST_LEAF(entry) && result)
{
@@ -1496,8 +1501,10 @@ gist_point_consistent(PG_FUNCTION_ARGS)
*/
BOX *box = DatumGetBoxP(entry->key);
- Assert(box->high.x == box->low.x
- && box->high.y == box->low.y);
+ Assert((box->high.x == box->low.x ||
+ (isnan(box->high.x) && isnan(box->low.x))) &&
+ (box->high.y == box->low.y ||
+ (isnan(box->high.y) && isnan(box->low.y))));
result = DatumGetBool(DirectFunctionCall2(poly_contain_pt,
PolygonPGetDatum(query),
PointPGetDatum(&box->high)));
@@ -1509,11 +1516,16 @@ gist_point_consistent(PG_FUNCTION_ARGS)
{
CIRCLE *query = PG_GETARG_CIRCLE_P(1);
- result = DatumGetBool(DirectFunctionCall5(gist_circle_consistent,
- PointerGetDatum(entry),
- CirclePGetDatum(query),
- Int16GetDatum(RTOverlapStrategyNumber),
- 0, PointerGetDatum(recheck)));
+ /*
+ * A NaN point fails the bounding-box prefilter, though the
+ * exact operator below may still match it, as on the heap.
+ */
+ result = box_has_nan(DatumGetBoxP(entry->key)) ||
+ DatumGetBool(DirectFunctionCall5(gist_circle_consistent,
+ PointerGetDatum(entry),
+ CirclePGetDatum(query),
+ Int16GetDatum(RTOverlapStrategyNumber),
+ 0, PointerGetDatum(recheck)));
if (GIST_LEAF(entry) && result)
{
@@ -1523,8 +1535,10 @@ gist_point_consistent(PG_FUNCTION_ARGS)
*/
BOX *box = DatumGetBoxP(entry->key);
- Assert(box->high.x == box->low.x
- && box->high.y == box->low.y);
+ Assert((box->high.x == box->low.x ||
+ (isnan(box->high.x) && isnan(box->low.x))) &&
+ (box->high.y == box->low.y ||
+ (isnan(box->high.y) && isnan(box->low.y))));
result = DatumGetBool(DirectFunctionCall2(circle_contain_pt,
CirclePGetDatum(query),
PointPGetDatum(&box->high)));
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 7b2640f0e04..272b679bbac 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -380,7 +380,7 @@ SELECT count(*) FROM point_tbl WHERE f1 <@ polygon '(0,0),(0,100),(100,100),(50,
SELECT count(*) FROM point_tbl WHERE f1 <@ polygon '(0,0),(0,100),(100,100),(50,50),(100,0),(0,0)';
count
-------
- 4
+ 5
(1 row)
EXPLAIN (COSTS OFF)
diff --git a/src/test/regress/expected/gist.out b/src/test/regress/expected/gist.out
index 174f8b4251f..e7f1a27f866 100644
--- a/src/test/regress/expected/gist.out
+++ b/src/test/regress/expected/gist.out
@@ -533,6 +533,20 @@ select count(*) from gist_nan_point_tbl where p ~= point '(3,3)';
1
(1 row)
+-- a NaN point fails the bounding-box prefilter of the polygon and circle
+-- contains strategies, so the exact operator decides, as on the heap
+select count(*) from gist_nan_point_tbl where p <@ polygon '(0,0),(0,100),(100,100),(50,50),(100,0),(0,0)';
+ count
+-------
+ 7602
+(1 row)
+
+select count(*) from gist_nan_point_tbl where p <@ circle '<(50,50),50>';
+ count
+-------
+ 7843
+(1 row)
+
reset enable_seqscan;
reset enable_bitmapscan;
drop table gist_nan_point_tbl;
diff --git a/src/test/regress/sql/gist.sql b/src/test/regress/sql/gist.sql
index 654318fb4db..b9df6490ca5 100644
--- a/src/test/regress/sql/gist.sql
+++ b/src/test/regress/sql/gist.sql
@@ -268,6 +268,10 @@ select count(*) from gist_nan_point_tbl where p ~= point '(NaN,NaN)';
select count(*) from gist_nan_point_tbl where p ~= point '(NaN,3)';
select count(*) from gist_nan_point_tbl where p ~= point '(3,NaN)';
select count(*) from gist_nan_point_tbl where p ~= point '(3,3)';
+-- a NaN point fails the bounding-box prefilter of the polygon and circle
+-- contains strategies, so the exact operator decides, as on the heap
+select count(*) from gist_nan_point_tbl where p <@ polygon '(0,0),(0,100),(100,100),(50,50),(100,0),(0,0)';
+select count(*) from gist_nan_point_tbl where p <@ circle '<(50,50),50>';
reset enable_seqscan;
reset enable_bitmapscan;
drop table gist_nan_point_tbl;
--
2.37.1 (Apple Git-137.1)
[application/octet-stream] v6-0001-Fix-BRIN-box_inclusion_ops-hiding-rows-next-to-a-.patch (13.0K, ../../CAGRkXqSN7vB_h41_jvsBbDb2sVNW=rmjxpndvL70oCDrYhbh=w@mail.gmail.com/7-v6-0001-Fix-BRIN-box_inclusion_ops-hiding-rows-next-to-a-.patch)
download | inline diff:
From 76a9c50a97488c2f08c1d60e74c55fdcb2658fe1 Mon Sep 17 00:00:00 2001
From: Shihao <zhong950419@gmail.com>
Date: Wed, 23 Sep 2026 21:23:52 -0400
Subject: [PATCH v6 1/4] Fix BRIN box_inclusion_ops hiding rows next to a NaN
box
bound_box() lets a NaN coordinate into the range summary, and no box
operator matches a NaN bound, so BRIN skipped the whole range.
Add a mergeable support function, box_mergeable(), that rejects boxes
with a NaN coordinate. A range holding one is then marked unmergeable
and always scanned. BRIN also checks the first value of a range now,
which it used to copy into the summary unchecked. At scan time, a
summary that is not mergeable even with itself is treated as
unmergeable. So indexes built before this fix return correct results
without a REINDEX.
This needs a catalog change, so it is for master only.
Author: Kirill Reshke <reshkekirill@gmail.com>
Author: Shihao Zhong <zhong950419@gmail.com>
Reported-by: Ke <kehan5800@gmail.com>
Reported-by: Andrey Borodin <x4mmm@yandex-team.ru>
Discussion: https://postgr.es/m/19705-548fda77321e062d@postgresql.org
---
doc/src/sgml/brin.sgml | 6 +-
src/backend/access/brin/brin_inclusion.c | 22 +++++++
src/backend/utils/adt/geo_ops.c | 19 ++++++
src/include/catalog/pg_amproc.dat | 2 +
src/include/catalog/pg_proc.dat | 3 +
src/test/regress/expected/brin.out | 73 ++++++++++++++++++++++++
src/test/regress/expected/geometry.out | 13 +++++
src/test/regress/sql/brin.sql | 45 +++++++++++++++
src/test/regress/sql/geometry.sql | 4 ++
9 files changed, 185 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/brin.sgml b/doc/src/sgml/brin.sgml
index 64fb520db7e..2b3e41056b1 100644
--- a/doc/src/sgml/brin.sgml
+++ b/doc/src/sgml/brin.sgml
@@ -1187,8 +1187,10 @@ typedef struct BrinOpcInfo
<para>
Support function numbers 12 and 14 are provided to support
irregularities of built-in data types. Function number 12
- is used to support network addresses from different families which
- are not mergeable. Function number 14 is used to support
+ is used to support network addresses from different families, and
+ boxes with NaN coordinates, which are not mergeable. A value that is
+ not mergeable even with itself makes its block range always match.
+ Function number 14 is used to support
empty ranges. Function number 13 is an optional but
recommended one, which allows the new value to be checked before
it is passed to the union function. As the BRIN framework can shortcut
diff --git a/src/backend/access/brin/brin_inclusion.c b/src/backend/access/brin/brin_inclusion.c
index 5a2058d9aad..4dddc45f03d 100644
--- a/src/backend/access/brin/brin_inclusion.c
+++ b/src/backend/access/brin/brin_inclusion.c
@@ -191,8 +191,19 @@ brin_inclusion_add_value(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(false);
}
+ /*
+ * A new union is not checked for mergeability below, so check the value
+ * against itself. A box with a NaN coordinate fails this.
+ */
if (new)
+ {
+ finfo = inclusion_get_procinfo(bdesc, attno, PROCNUM_MERGEABLE, true);
+ if (finfo != NULL &&
+ !DatumGetBool(FunctionCall2Coll(finfo, colloid, newval, newval)))
+ column->bv_values[INCLUSION_UNMERGEABLE] = BoolGetDatum(true);
+
PG_RETURN_BOOL(true);
+ }
/* Check if the new value is already contained. */
finfo = inclusion_get_procinfo(bdesc, attno, PROCNUM_CONTAINS, true);
@@ -274,6 +285,17 @@ brin_inclusion_consistent(PG_FUNCTION_ARGS)
subtype = key->sk_subtype;
query = key->sk_argument;
unionval = column->bv_values[INCLUSION_UNION];
+
+ /*
+ * Likewise if the union is not mergeable even with itself. An index
+ * built before the opclass had a mergeable function can hold such a union
+ * without the flag, like a box union with NaN bounds.
+ */
+ finfo = inclusion_get_procinfo(bdesc, attno, PROCNUM_MERGEABLE, true);
+ if (finfo != NULL &&
+ !DatumGetBool(FunctionCall2Coll(finfo, colloid, unionval, unionval)))
+ PG_RETURN_BOOL(true);
+
switch (key->sk_strategy)
{
/*
diff --git a/src/backend/utils/adt/geo_ops.c b/src/backend/utils/adt/geo_ops.c
index 73324b91fe5..66bb58fb7d6 100644
--- a/src/backend/utils/adt/geo_ops.c
+++ b/src/backend/utils/adt/geo_ops.c
@@ -4427,6 +4427,25 @@ boxes_bound_box(PG_FUNCTION_ARGS)
PG_RETURN_BOX_P(container);
}
+/*
+ * Can the two boxes be merged into one bounding box?
+ *
+ * Not if either has a NaN coordinate: the bounding box would have NaN
+ * bounds too, and no box operator matches those. This is the mergeable
+ * support function of BRIN box_inclusion_ops.
+ */
+Datum
+box_mergeable(PG_FUNCTION_ARGS)
+{
+ BOX *box1 = PG_GETARG_BOX_P(0),
+ *box2 = PG_GETARG_BOX_P(1);
+
+ PG_RETURN_BOOL(!(isnan(box1->high.x) || isnan(box1->high.y) ||
+ isnan(box1->low.x) || isnan(box1->low.y) ||
+ isnan(box2->high.x) || isnan(box2->high.y) ||
+ isnan(box2->low.x) || isnan(box2->low.y)));
+}
+
/***********************************************************************
**
diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat
index 4a1efdbc899..db24e42e795 100644
--- a/src/include/catalog/pg_amproc.dat
+++ b/src/include/catalog/pg_amproc.dat
@@ -2033,6 +2033,8 @@
amproc => 'brin_inclusion_union' },
{ amprocfamily => 'brin/box_inclusion_ops', amproclefttype => 'box',
amprocrighttype => 'box', amprocnum => '11', amproc => 'bound_box' },
+{ amprocfamily => 'brin/box_inclusion_ops', amproclefttype => 'box',
+ amprocrighttype => 'box', amprocnum => '12', amproc => 'box_mergeable' },
{ amprocfamily => 'brin/box_inclusion_ops', amproclefttype => 'box',
amprocrighttype => 'box', amprocnum => '13', amproc => 'box_contain' },
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f46427258e3..bbbce58a962 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -2140,6 +2140,9 @@
{ oid => '4067', descr => 'bounding box of two boxes',
proname => 'bound_box', prorettype => 'box', proargtypes => 'box box',
prosrc => 'boxes_bound_box' },
+{ oid => '8400', descr => 'can two boxes be merged into a single summary',
+ proname => 'box_mergeable', prorettype => 'bool', proargtypes => 'box box',
+ prosrc => 'box_mergeable' },
{ oid => '981', descr => 'box diagonal',
proname => 'diagonal', prorettype => 'lseg', proargtypes => 'box',
prosrc => 'box_diagonal' },
diff --git a/src/test/regress/expected/brin.out b/src/test/regress/expected/brin.out
index e1db2280cf9..445efddd8f6 100644
--- a/src/test/regress/expected/brin.out
+++ b/src/test/regress/expected/brin.out
@@ -589,3 +589,76 @@ CREATE INDEX brin_insert_optimization_idx ON brin_insert_optimization USING brin
UPDATE brin_insert_optimization SET a = a;
REINDEX INDEX CONCURRENTLY brin_insert_optimization_idx;
DROP TABLE brin_insert_optimization;
+-- a box with a NaN coordinate must not hide the other rows of its page
+-- range (bug #19705)
+CREATE TABLE brin_box_nan (v box);
+INSERT INTO brin_box_nan SELECT box '(0,0),(1,1)' FROM generate_series(1, 100);
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+CREATE INDEX brin_box_nan_idx ON brin_box_nan USING brin (v);
+SET enable_seqscan = off;
+SELECT count(*) FROM brin_box_nan WHERE v && box '(-2,-2),(2,2)';
+ count
+-------
+ 100
+(1 row)
+
+SELECT count(*) FROM brin_box_nan WHERE v @> point '(0.5,0.5)';
+ count
+-------
+ 100
+(1 row)
+
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(0,0),(1,1)';
+ count
+-------
+ 100
+(1 row)
+
+-- the NaN row is found too: unmergeable ranges are always scanned
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(NaN,NaN),(0,0)';
+ count
+-------
+ 1
+(1 row)
+
+-- also when it is the only value in its range
+TRUNCATE brin_box_nan;
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+REINDEX INDEX brin_box_nan_idx;
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(NaN,NaN),(0,0)';
+ count
+-------
+ 1
+(1 row)
+
+RESET enable_seqscan;
+DROP TABLE brin_box_nan;
+-- An index built before box_inclusion_ops had a mergeable function can
+-- hold a NaN union. Mimic one with an opclass that gets the function
+-- only after the build.
+CREATE OPERATOR FAMILY brin_box_nan_ops USING brin;
+CREATE OPERATOR CLASS brin_box_nan_ops FOR TYPE box USING brin
+ FAMILY brin_box_nan_ops AS
+ OPERATOR 3 &&,
+ FUNCTION 1 brin_inclusion_opcinfo(internal),
+ FUNCTION 2 brin_inclusion_add_value(internal, internal, internal, internal),
+ FUNCTION 3 brin_inclusion_consistent(internal, internal, internal),
+ FUNCTION 4 brin_inclusion_union(internal, internal, internal),
+ FUNCTION 11 bound_box(box, box);
+CREATE TABLE brin_box_nan (v box);
+INSERT INTO brin_box_nan SELECT box '(0,0),(1,1)' FROM generate_series(1, 100);
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+CREATE INDEX brin_box_nan_idx ON brin_box_nan USING brin (v brin_box_nan_ops);
+ALTER OPERATOR FAMILY brin_box_nan_ops USING brin
+ ADD FUNCTION 12 (box, box) box_mergeable(box, box);
+\c -
+SET enable_seqscan = off;
+SELECT count(*) FROM brin_box_nan WHERE v && box '(-2,-2),(2,2)';
+ count
+-------
+ 100
+(1 row)
+
+RESET enable_seqscan;
+DROP TABLE brin_box_nan;
+DROP OPERATOR FAMILY brin_box_nan_ops USING brin;
diff --git a/src/test/regress/expected/geometry.out b/src/test/regress/expected/geometry.out
index 1d168b21cbc..9d03c4718a9 100644
--- a/src/test/regress/expected/geometry.out
+++ b/src/test/regress/expected/geometry.out
@@ -5321,3 +5321,16 @@ SELECT * FROM pg_input_error_info('(1,2),-1', 'circle');
invalid input syntax for type circle: "(1,2),-1" | | | 22P02
(1 row)
+-- box_mergeable() rejects NaN coordinates (BRIN box_inclusion_ops)
+SELECT box_mergeable(box '(0,0),(1,1)', box '(NaN,3),(0,2)');
+ box_mergeable
+---------------
+ f
+(1 row)
+
+SELECT box_mergeable(box '(0,0),(1,1)', box '(2,3),(0,2)');
+ box_mergeable
+---------------
+ t
+(1 row)
+
diff --git a/src/test/regress/sql/brin.sql b/src/test/regress/sql/brin.sql
index 7ea97f47c8d..33bfef7a5e6 100644
--- a/src/test/regress/sql/brin.sql
+++ b/src/test/regress/sql/brin.sql
@@ -534,3 +534,48 @@ CREATE INDEX brin_insert_optimization_idx ON brin_insert_optimization USING brin
UPDATE brin_insert_optimization SET a = a;
REINDEX INDEX CONCURRENTLY brin_insert_optimization_idx;
DROP TABLE brin_insert_optimization;
+
+-- a box with a NaN coordinate must not hide the other rows of its page
+-- range (bug #19705)
+CREATE TABLE brin_box_nan (v box);
+INSERT INTO brin_box_nan SELECT box '(0,0),(1,1)' FROM generate_series(1, 100);
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+CREATE INDEX brin_box_nan_idx ON brin_box_nan USING brin (v);
+SET enable_seqscan = off;
+SELECT count(*) FROM brin_box_nan WHERE v && box '(-2,-2),(2,2)';
+SELECT count(*) FROM brin_box_nan WHERE v @> point '(0.5,0.5)';
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(0,0),(1,1)';
+-- the NaN row is found too: unmergeable ranges are always scanned
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(NaN,NaN),(0,0)';
+-- also when it is the only value in its range
+TRUNCATE brin_box_nan;
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+REINDEX INDEX brin_box_nan_idx;
+SELECT count(*) FROM brin_box_nan WHERE v ~= box '(NaN,NaN),(0,0)';
+RESET enable_seqscan;
+DROP TABLE brin_box_nan;
+
+-- An index built before box_inclusion_ops had a mergeable function can
+-- hold a NaN union. Mimic one with an opclass that gets the function
+-- only after the build.
+CREATE OPERATOR FAMILY brin_box_nan_ops USING brin;
+CREATE OPERATOR CLASS brin_box_nan_ops FOR TYPE box USING brin
+ FAMILY brin_box_nan_ops AS
+ OPERATOR 3 &&,
+ FUNCTION 1 brin_inclusion_opcinfo(internal),
+ FUNCTION 2 brin_inclusion_add_value(internal, internal, internal, internal),
+ FUNCTION 3 brin_inclusion_consistent(internal, internal, internal),
+ FUNCTION 4 brin_inclusion_union(internal, internal, internal),
+ FUNCTION 11 bound_box(box, box);
+CREATE TABLE brin_box_nan (v box);
+INSERT INTO brin_box_nan SELECT box '(0,0),(1,1)' FROM generate_series(1, 100);
+INSERT INTO brin_box_nan VALUES (box '(NaN,NaN),(0,0)');
+CREATE INDEX brin_box_nan_idx ON brin_box_nan USING brin (v brin_box_nan_ops);
+ALTER OPERATOR FAMILY brin_box_nan_ops USING brin
+ ADD FUNCTION 12 (box, box) box_mergeable(box, box);
+\c -
+SET enable_seqscan = off;
+SELECT count(*) FROM brin_box_nan WHERE v && box '(-2,-2),(2,2)';
+RESET enable_seqscan;
+DROP TABLE brin_box_nan;
+DROP OPERATOR FAMILY brin_box_nan_ops USING brin;
diff --git a/src/test/regress/sql/geometry.sql b/src/test/regress/sql/geometry.sql
index c3ea368da5e..994797c4d24 100644
--- a/src/test/regress/sql/geometry.sql
+++ b/src/test/regress/sql/geometry.sql
@@ -529,3 +529,7 @@ SELECT pg_input_is_valid('(1', 'circle');
SELECT * FROM pg_input_error_info('1,', 'circle');
SELECT pg_input_is_valid('(1,2),-1', 'circle');
SELECT * FROM pg_input_error_info('(1,2),-1', 'circle');
+
+-- box_mergeable() rejects NaN coordinates (BRIN box_inclusion_ops)
+SELECT box_mergeable(box '(0,0),(1,1)', box '(NaN,3),(0,2)');
+SELECT box_mergeable(box '(0,0),(1,1)', box '(2,3),(0,2)');
--
2.37.1 (Apple Git-137.1)
^ permalink raw reply [nested|flat] 11+ messages in thread
end of thread, other threads:[~2026-09-28 07:01 UTC | newest]
Thread overview: 11+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-09-19 17:37 BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows PG Bug reporting form <noreply@postgresql.org>
2026-09-21 23:49 ` shihao zhong <zhong950419@gmail.com>
2026-09-22 12:12 ` Kirill Reshke <reshkekirill@gmail.com>
2026-09-22 12:37 ` Andrey Borodin <x4mmm@yandex-team.ru>
2026-09-23 03:43 ` shihao zhong <zhong950419@gmail.com>
2026-09-23 17:51 ` Kirill Reshke <reshkekirill@gmail.com>
2026-09-24 02:43 ` shihao zhong <zhong950419@gmail.com>
2026-09-24 06:22 ` Kirill Reshke <reshkekirill@gmail.com>
2026-09-25 04:29 ` shihao zhong <zhong950419@gmail.com>
2026-09-26 00:09 ` Manu <manuelreyesbravo@gmail.com>
2026-09-28 07:01 ` shihao zhong <zhong950419@gmail.com>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox