pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedUsing index for IS NULL query
9+ messages / 7 participants
[nested] [flat]
* Using index for IS NULL query
@ 2008-11-11 19:55 Andrus <kobruleht2@hot.ee>
2008-11-11 20:45 ` Re: Using index for IS NULL query Tom Lane <tgl@sss.pgh.pa.us>
2008-11-11 20:47 ` Re: Using index for IS NULL query Andreas Kretschmer <akretschmer@spamfence.net>
2008-11-11 21:01 ` Re: Using index for IS NULL query Tomas Vondra <tv@fuzzy.cz>
0 siblings, 3 replies; 9+ messages in thread
From: Andrus @ 2008-11-11 19:55 UTC (permalink / raw)
To: pgsql-performance
Index is not used for
is null
condition:
create index makse_dokumnr_idx on makse(dokumnr);
explain select
sum( summa)
from MAKSE
where dokumnr is null
"Aggregate (cost=131927.95..131927.96 rows=1 width=10)"
" -> Seq Scan on makse (cost=0.00..131927.94 rows=1 width=10)"
" Filter: (dokumnr IS NULL)"
Table makse contains 1200000 rows and about 800 rows with dokumnr is null so
using index is much faster that seq scan.
How to fix ?
Andrus.
"PostgreSQL 8.1.4 on i686-pc-linux-gnu, compiled by GCC
i686-pc-linux-gnu-gcc (GCC) 3.4.6 (Gentoo 3.4.6-r1, ssp-3.4.5-1.0,
pie-8.7.9)"
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Using index for IS NULL query
2008-11-11 19:55 Using index for IS NULL query Andrus <kobruleht2@hot.ee>
@ 2008-11-11 20:45 ` Tom Lane <tgl@sss.pgh.pa.us>
2008-11-11 20:50 ` Re: Using index for IS NULL query Andreas Kretschmer <akretschmer@spamfence.net>
2008-11-12 13:29 ` Re: Using index for IS NULL query Matthew Wakeling <matthew@flymine.org>
2 siblings, 2 replies; 9+ messages in thread
From: Tom Lane @ 2008-11-11 20:45 UTC (permalink / raw)
To: Andrus <kobruleht2@hot.ee>; +Cc: pgsql-performance
"Andrus" <kobruleht2@hot.ee> writes:
> Index is not used for
> is null
> How to fix ?
Update to something newer than 8.1 (specifically, you'll need 8.3).
regards, tom lane
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Using index for IS NULL query
2008-11-11 19:55 Using index for IS NULL query Andrus <kobruleht2@hot.ee>
2008-11-11 20:45 ` Re: Using index for IS NULL query Tom Lane <tgl@sss.pgh.pa.us>
@ 2008-11-11 20:50 ` Andreas Kretschmer <akretschmer@spamfence.net>
1 sibling, 0 replies; 9+ messages in thread
From: Andreas Kretschmer @ 2008-11-11 20:50 UTC (permalink / raw)
To: pgsql-performance
Tom Lane <tgl@sss.pgh.pa.us> schrieb:
> "Andrus" <kobruleht2@hot.ee> writes:
> > Index is not used for
> > is null
>
> > How to fix ?
>
> Update to something newer than 8.1 (specifically, you'll need 8.3).
Right. For my example in the other mail:
test=*# create index idx_foo on foo(i);
CREATE INDEX
test=*# explain analyse select * from foo where i is null;
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------
Bitmap Heap Scan on foo (cost=95.11..4780.49 rows=5000 width=8) (actual time=0.052..0.053 rows=1 loops=1)
Recheck Cond: (i IS NULL)
-> Bitmap Index Scan on idx_foo (cost=0.00..93.86 rows=5000 width=0) (actual time=0.047..0.047 rows=1 loops=1)
Index Cond: (i IS NULL)
Total runtime: 0.076 ms
Andreas
--
Really, I'm not out to destroy Microsoft. That will just be a completely
unintentional side effect. (Linus Torvalds)
"If I was god, I would recompile penguin with --enable-fly." (unknown)
Kaufbach, Saxony, Germany, Europe. N 51.05082°, E 13.56889°
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Using index for IS NULL query
2008-11-11 19:55 Using index for IS NULL query Andrus <kobruleht2@hot.ee>
2008-11-11 20:45 ` Re: Using index for IS NULL query Tom Lane <tgl@sss.pgh.pa.us>
@ 2008-11-12 13:29 ` Matthew Wakeling <matthew@flymine.org>
1 sibling, 0 replies; 9+ messages in thread
From: Matthew Wakeling @ 2008-11-12 13:29 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Andrus <kobruleht2@hot.ee>; pgsql-performance
On Tue, 11 Nov 2008, Tom Lane wrote:
>> Index is not used for
>> is null
>> How to fix ?
>
> Update to something newer than 8.1 (specifically, you'll need 8.3).
Oooh, that's useful to know. We can get rid of all our extra nulls
indexes. Thanks.
Matthew
--
As you approach the airport, you see a sign saying "Beware - low
flying airplanes". There's not a lot you can do about that. Take
your hat off? -- Michael Flanders
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Using index for IS NULL query
2008-11-11 19:55 Using index for IS NULL query Andrus <kobruleht2@hot.ee>
@ 2008-11-11 20:47 ` Andreas Kretschmer <akretschmer@spamfence.net>
2 siblings, 0 replies; 9+ messages in thread
From: Andreas Kretschmer @ 2008-11-11 20:47 UTC (permalink / raw)
To: pgsql-performance
Andrus <kobruleht2@hot.ee> schrieb:
> Index is not used for
>
> is null
>
> condition:
>
> create index makse_dokumnr_idx on makse(dokumnr);
> explain select
> sum( summa)
> from MAKSE
> where dokumnr is null
>
> "Aggregate (cost=131927.95..131927.96 rows=1 width=10)"
> " -> Seq Scan on makse (cost=0.00..131927.94 rows=1 width=10)"
> " Filter: (dokumnr IS NULL)"
>
>
>
> Table makse contains 1200000 rows and about 800 rows with dokumnr is null
> so using index is much faster that seq scan.
> How to fix ?
Create a partial index like below:
test=# create table foo ( i float);
CREATE TABLE
Zeit: 1,138 ms
test=*# insert into foo select random() from generate_series(1,1000000);
INSERT 0 1000000
test=*# insert into foo values (NULL);
INSERT 0 1
test=*# create index idx_foo on foo(i) where i is null;
CREATE INDEX
test=*# explain analyse select * from foo where i is null;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------
Bitmap Heap Scan on foo (cost=5.51..4690.89 rows=5000 width=8) (actual
time=0.037..0.038 rows=1 loops=1)
Recheck Cond: (i IS NULL)
-> Bitmap Index Scan on idx_foo (cost=0.00..4.26 rows=5000 width=0)
(actual time=0.033..0.033 rows=1 loops=1)
Index Cond: (i IS NULL)
Total runtime: 0.068 ms
(5 Zeilen)
Maybe there are other solutions...
Andreas
--
Really, I'm not out to destroy Microsoft. That will just be a completely
unintentional side effect. (Linus Torvalds)
"If I was god, I would recompile penguin with --enable-fly." (unknown)
Kaufbach, Saxony, Germany, Europe. N 51.05082°, E 13.56889°
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Using index for IS NULL query
2008-11-11 19:55 Using index for IS NULL query Andrus <kobruleht2@hot.ee>
@ 2008-11-11 21:01 ` Tomas Vondra <tv@fuzzy.cz>
2008-11-11 23:00 ` Re: Using index for IS NULL query Vladimir Sitnikov <sitnikov.vladimir@gmail.com>
2 siblings, 1 reply; 9+ messages in thread
From: Tomas Vondra @ 2008-11-11 21:01 UTC (permalink / raw)
To: Andrus <kobruleht2@hot.ee>; +Cc: pgsql-performance
> Index is not used for
>
> is null
>
> condition:
>
> create index makse_dokumnr_idx on makse(dokumnr);
> explain select
> sum( summa)
> from MAKSE
> where dokumnr is null
>
> "Aggregate (cost=131927.95..131927.96 rows=1 width=10)"
> " -> Seq Scan on makse (cost=0.00..131927.94 rows=1 width=10)"
> " Filter: (dokumnr IS NULL)"
>
>
> Table makse contains 1200000 rows and about 800 rows with dokumnr is
> null so using index is much faster that seq scan.
> How to fix ?
Yes, NULL values are not stored in the index, but you may create
functional index on
(CASE WHEN dokumnr IS NULL THEN -1 ELSE dokumnr END)
and then use the same expression in the WHERE clause. You may replace
the '-1' value by something that's not used in the dokumnr column.
regards
Tomas
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Using index for IS NULL query
2008-11-11 19:55 Using index for IS NULL query Andrus <kobruleht2@hot.ee>
2008-11-11 21:01 ` Re: Using index for IS NULL query Tomas Vondra <tv@fuzzy.cz>
@ 2008-11-11 23:00 ` Vladimir Sitnikov <sitnikov.vladimir@gmail.com>
2008-11-11 23:34 ` Re: Using index for IS NULL query Scott Marlowe <scott.marlowe@gmail.com>
2008-11-13 00:27 ` Re: Using index for IS NULL query Tomas Vondra <tv@fuzzy.cz>
0 siblings, 2 replies; 9+ messages in thread
From: Vladimir Sitnikov @ 2008-11-11 23:00 UTC (permalink / raw)
To: Tomas Vondra <tv@fuzzy.cz>; +Cc: Andrus <kobruleht2@hot.ee>; pgsql-performance
> Yes, NULL values are not stored in the index, but you may create functional
> index on
>
Are you sure NULL values are not stored? btree, gist and bitmap index and
search for NULL values.
select amname, amindexnulls, amsearchnulls from pg_am;
amname | amindexnulls | amsearchnulls
--------+--------------+---------------
btree | t | t
hash | f | f
gist | t | t
gin | f | f
bitmap | t | t
(5 rows)
Sincerely yours,
Vladimir Sitnikov
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Using index for IS NULL query
2008-11-11 19:55 Using index for IS NULL query Andrus <kobruleht2@hot.ee>
2008-11-11 21:01 ` Re: Using index for IS NULL query Tomas Vondra <tv@fuzzy.cz>
2008-11-11 23:00 ` Re: Using index for IS NULL query Vladimir Sitnikov <sitnikov.vladimir@gmail.com>
@ 2008-11-11 23:34 ` Scott Marlowe <scott.marlowe@gmail.com>
1 sibling, 0 replies; 9+ messages in thread
From: Scott Marlowe @ 2008-11-11 23:34 UTC (permalink / raw)
To: Vladimir Sitnikov <sitnikov.vladimir@gmail.com>; +Cc: Tomas Vondra <tv@fuzzy.cz>; Andrus <kobruleht2@hot.ee>; pgsql-performance
On Tue, Nov 11, 2008 at 4:00 PM, Vladimir Sitnikov
<sitnikov.vladimir@gmail.com> wrote:
>
>> Yes, NULL values are not stored in the index, but you may create
>> functional index on
>
> Are you sure NULL values are not stored? btree, gist and bitmap index and
> search for NULL values.
It's not that they're not stored, it's that before 8.3 pg didn't know
how to compare to them I believe. The standard trick was to create a
partial index with "where x is null" on the table / column. 8.3 knows
how to compare them and doesn't need the partial index.
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Using index for IS NULL query
2008-11-11 19:55 Using index for IS NULL query Andrus <kobruleht2@hot.ee>
2008-11-11 21:01 ` Re: Using index for IS NULL query Tomas Vondra <tv@fuzzy.cz>
2008-11-11 23:00 ` Re: Using index for IS NULL query Vladimir Sitnikov <sitnikov.vladimir@gmail.com>
@ 2008-11-13 00:27 ` Tomas Vondra <tv@fuzzy.cz>
1 sibling, 0 replies; 9+ messages in thread
From: Tomas Vondra @ 2008-11-13 00:27 UTC (permalink / raw)
To: ; +Cc: pgsql-performance
Well, you're obviously right - I didn't know this. I guess I've found
that the index is not used for null values, and deduced somehow that
NULL values are not stored in the index.
Thanks, it's nice to find out a 'bug' before it's too late :-)
regards
Tomas
> Are you sure NULL values are not stored? btree, gist and bitmap index
> and search for NULL values.
>
> select amname, amindexnulls, amsearchnulls from pg_am;
>
> amname | amindexnulls | amsearchnulls
> --------+--------------+---------------
> btree | t | t
> hash | f | f
> gist | t | t
> gin | f | f
> bitmap | t | t
> (5 rows)
>
>
> Sincerely yours,
> Vladimir Sitnikov
^ permalink raw reply [nested|flat] 9+ messages in thread
end of thread, other threads:[~2008-11-13 00:27 UTC | newest]
Thread overview: 9+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2008-11-11 19:55 Using index for IS NULL query Andrus <kobruleht2@hot.ee>
2008-11-11 20:45 ` Tom Lane <tgl@sss.pgh.pa.us>
2008-11-11 20:50 ` Andreas Kretschmer <akretschmer@spamfence.net>
2008-11-12 13:29 ` Matthew Wakeling <matthew@flymine.org>
2008-11-11 20:47 ` Andreas Kretschmer <akretschmer@spamfence.net>
2008-11-11 21:01 ` Tomas Vondra <tv@fuzzy.cz>
2008-11-11 23:00 ` Vladimir Sitnikov <sitnikov.vladimir@gmail.com>
2008-11-11 23:34 ` Scott Marlowe <scott.marlowe@gmail.com>
2008-11-13 00:27 ` Tomas Vondra <tv@fuzzy.cz>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox