pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: PG Bug reporting form <noreply@postgresql.org>
To: pgsql-bugs@lists.postgresql.org
Cc: rpsalmi@gmail.com
Subject: BUG #17545: Incorrect selectivity for IS NOT DISTINCT FROM and NULLs
Date: Mon, 11 Jul 2022 14:18:44 +0000
Message-ID: <17545-a0ca4de888953169@postgresql.org> (raw)

The following bug has been logged on the website:

Bug reference:      17545
Logged by:          Roope Salmi
Email address:      rpsalmi@gmail.com
PostgreSQL version: 14.4
Operating system:   Ubuntu 22.04
Description:        

Hi,

It appears that selectivity for "IS NOT DISTINCT FROM" clauses regard the
proportion of NULLs in the same way as "=", leading to polar opposite row
count
estimates.

In practice this causes nested loop joins to be used when that is not
appropriate. Not sure if this is a known issue, but I was unable to find
any
previous mentions of it.

Here I create a table with one column and 1000 rows of NULL. A cartesian
product
with "WHERE x.a = y.a" correctly estimates that there are around zero
matching
rows. "x.a IS NOT DISTINCT FROM y.a" incorrectly gives the same estimate,
whereas "x.a = y.a OR (x.a IS NULL AND y.a IS NULL)", which should be
equivalent, gives the correct 1000000.

postgres=# CREATE TABLE test(a INTEGER);
CREATE TABLE
postgres=# INSERT INTO test(a) SELECT NULL FROM generate_series(1, 1000);
INSERT 0 1000
postgres=# ANALYZE test;
ANALYZE
postgres=# EXPLAIN SELECT FROM test x, test y WHERE x.a = y.a;
                              QUERY PLAN                              
----------------------------------------------------------------------
 Merge Join  (cost=127.66..137.67 rows=1 width=0)
   Merge Cond: (x.a = y.a)
   ->  Sort  (cost=63.83..66.33 rows=1000 width=4)
         Sort Key: x.a
         ->  Seq Scan on test x  (cost=0.00..14.00 rows=1000 width=4)
   ->  Sort  (cost=63.83..66.33 rows=1000 width=4)
         Sort Key: y.a
         ->  Seq Scan on test y  (cost=0.00..14.00 rows=1000 width=4)
(8 rows)

postgres=# EXPLAIN SELECT FROM test x, test y
postgres=# WHERE x.a IS NOT DISTINCT FROM y.a;
                              QUERY PLAN                              
----------------------------------------------------------------------
 Nested Loop  (cost=0.00..15030.50 rows=1 width=0)
   Join Filter: (NOT (x.a IS DISTINCT FROM y.a))
   ->  Seq Scan on test x  (cost=0.00..14.00 rows=1000 width=4)
   ->  Materialize  (cost=0.00..19.00 rows=1000 width=4)
         ->  Seq Scan on test y  (cost=0.00..14.00 rows=1000 width=4)
(5 rows)

postgres=# EXPLAIN SELECT FROM test x, test y                               
    
postgres-# WHERE x.a = y.a OR (x.a IS NULL AND y.a IS NULL);
                              QUERY PLAN                              
----------------------------------------------------------------------
 Nested Loop  (cost=0.00..15030.50 rows=1000000 width=0)
   Join Filter: ((x.a = y.a) OR ((x.a IS NULL) AND (y.a IS NULL)))
   ->  Seq Scan on test x  (cost=0.00..14.00 rows=1000 width=4)
   ->  Materialize  (cost=0.00..19.00 rows=1000 width=4)
         ->  Seq Scan on test y  (cost=0.00..14.00 rows=1000 width=4)
(5 rows)



view thread (9+ messages)  latest in thread

Message-ID: <17545-a0ca4de888953169@postgresql.org>
Permalink:  ../17545-a0ca4de888953169@postgresql.org/
Also on:    postgresql.org/message-id/17545-a0ca4de888953169@postgresql.org

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-bugs@postgresql.org
  Cc: noreply@postgresql.org, rpsalmi@gmail.com
  Subject: Re: BUG #17545: Incorrect selectivity for IS NOT DISTINCT FROM and NULLs
  In-Reply-To: <17545-a0ca4de888953169@postgresql.org>
  References: <17545-a0ca4de888953169@postgresql.org>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox