pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Rod Taylor <rbt@rbt.ca>
To: Becky Neville <rebecca.neville@yale.edu>
Cc: Joe Conway <mail@joeconway.com>
Cc: Postgresql Performance <pgsql-performance@postgresql.org>
Subject: Re: NOT IN doesn't use index? (fwd)
Date: 03 May 2003 17:16:41 -0400
Message-ID: <1051996600.1656.36.camel@jester> (raw)
In-Reply-To: <Pine.LNX.4.44.0305031544190.9898-100000@newt.zoo.cs.yale.edu>
References: <Pine.LNX.4.44.0305031544190.9898-100000@newt.zoo.cs.yale.edu>

On Sat, 2003-05-03 at 15:56, Becky Neville wrote:
> I think that list is actually (gulp) hard coded.  It's not my query.  I am 
> trying to speed it up for someone else - to hopefully learn something in 
> the process that isn't dependent on what version of postgres i'm 
> running :)

An interesting test might be to see if the overhead of doing a character
based comparison (as opposed to integer based) is significant.  If it
is, previous tests show it can be significant for CPU bound queries,
convert all of those codes into integers and use a lookup table table to
do the conversion.

Another interesting thought, since you have a long running query would
be to attempt an inversion.  Create a temporary table with the *valid*
codes if count(valid codes) < 2 * count(invalid codes).  Run the query
replacing NOT IN with a join to the temporary table.  This will reduce
the number of comparisons required, as a match can move onto the next
datum, but a NOT IN must check all values. If this helps, try indexing
(and analyzing) the temporary table.

By far the fastest results can be achieved by not allowing invalid
billing codes to be inserted into the table via a constraint of somekind
(check or fkey to summary table).

-- 
Rod Taylor <rbt@rbt.ca>

PGP Key: http://www.rbt.ca/rbtpub.asc

Attachments:

  signature.asc (application/pgp-signature, 186B) download

view thread (9+ messages)  latest in thread

Message-ID: <1051996600.1656.36.camel@jester>
Permalink:  ../1051996600.1656.36.camel@jester/
Also on:    postgresql.org/message-id/1051996600.1656.36.camel@jester

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-performance@postgresql.org
  Cc: rbt@rbt.ca, rebecca.neville@yale.edu, mail@joeconway.com
  Subject: Re: NOT IN doesn't use index? (fwd)
  In-Reply-To: <1051996600.1656.36.camel@jester>
  References: <Pine.LNX.4.44.0305031544190.9898-100000@newt.zoo.cs.yale.edu> <1051996600.1656.36.camel@jester>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox