pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
slow query - where not in
6+ messages / 4 participants
[nested] [flat]

* slow query - where not in
@ 2003-03-28 15:38  Jeremiah Elliott <jelliott@artcobell.com>
  0 siblings, 2 replies; 6+ messages in thread

From: Jeremiah Elliott @ 2003-03-28 15:38 UTC (permalink / raw)
  To: pgsql-performance

here is the query that is killing me:

select shoporder from sodetailtabletrans where shoporder not in(select 
shoporder from soheadertable)

This is just an example query. Any time I use 'where not in(' it takes several 
hours to return a resultset. The postgres version is 7.2.3 although I have 
tried it on my test server which has 7.3 on it and it runs just as slow. The 
server is a fast server 2GHz with a gig of ram.  I have tried several 
differant index setups but nothing seems to help.

soheadertable - 5104 rows
CREATE TABLE "soheadertable" (
   "shoporder" numeric(10,0) NOT NULL,
   "initrundate" date,
   "actualrundate" date,
   "processedminutes" numeric(10,0),
   "starttime" timestamptz,
   "endtime" timestamptz,
   "line" int4,
   "personcount" numeric(10,0),
   "product" varchar(15),
   "qtytomake" numeric(10,3),
   "linestatus" numeric(2,0) DEFAULT 1,
   "finishlinestatus" numeric(2,0) DEFAULT 1,
   "qtyinqueue" numeric(10,3),
   "lastcartonprinted" numeric(10,0),
   "qtydone" int8,
   "warehouse" text,
   "rescheduledate" date,
   "calculateddatetorun" date
);
CREATE  UNIQUE INDEX "shoporder_soheadertable_ukey" ON "soheadertable" 
("shoporder");

sodetailtabletrans - 31494 rows
CREATE TABLE "sodetailtabletrans" (
   "shoporder" numeric(10,0) NOT NULL,
   "soseq" numeric(5,0) NOT NULL,
   "product" char(15) NOT NULL,
   "qtyqueued" numeric(17,2),
   "qtyneeded" numeric(17,2),
   "qtyallocated" numeric(17,2),
   "qtyused" numeric(17,2),
   "linestatus" numeric(2,0) DEFAULT 1,
   "unitsperenditem" numeric(10,1),
   CONSTRAINT "sodetailtrans_pk" PRIMARY KEY ("shoporder", "soseq")

-Jeremiah Elliott
jelliott@artcobell.com
);




^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* Re: slow query - where not in
@ 2003-03-28 15:59  Bruno Wolff III <bruno@wolff.to>
  parent: Jeremiah Elliott <jelliott@artcobell.com>
  1 sibling, 1 reply; 6+ messages in thread

From: Bruno Wolff III @ 2003-03-28 15:59 UTC (permalink / raw)
  To: Jeremiah Elliott <jelliott@artcobell.com>; +Cc: pgsql-performance

On Fri, Mar 28, 2003 at 09:38:50 -0600,
  Jeremiah Elliott <jelliott@artcobell.com> wrote:
> here is the query that is killing me:
> 
> select shoporder from sodetailtabletrans where shoporder not in(select 
> shoporder from soheadertable)

This will probably work better in 7.4.

For now there are several ways to rewrite this query.

If there are no null values for shoporder in soheadertable or
sodetailtabletrans you can use not exists instead of not in:
select shoporder from sodetailtabletrans where shoporder not exists(select 
shoporder from soheadertable)

You can use set difference to calculate the result:
select shoporder from sodetailtabletrans except all select 
shoporder from soheadertable

If there are no null values for shoporder in one of sodetailtabletrans
or soheadertable you can user an outer join with a restriction that limits
the rows of interest to those that don't match:
select sodetailtabletrans.shoporder from sodetailtabletrans left join
soheadertable using (shoporder) where soheadertable.shoporder is null




^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* Re: slow query - where not in
@ 2003-03-28 16:20  Greg Stark <gsstark@mit.edu>
  parent: Jeremiah Elliott <jelliott@artcobell.com>
  1 sibling, 0 replies; 6+ messages in thread

From: Greg Stark @ 2003-03-28 16:20 UTC (permalink / raw)
  To: jelliott@artcobell.com; +Cc: pgsql-performance

Jeremiah Elliott <jelliott@artcobell.com> writes:

> here is the query that is killing me:
> 
> select shoporder from sodetailtabletrans where shoporder not in(select 
> shoporder from soheadertable)
> 
> This is just an example query. Any time I use 'where not in(' it takes several 
> hours to return a resultset. The postgres version is 7.2.3 although I have 
> tried it on my test server which has 7.3 on it and it runs just as slow. The 
> server is a fast server 2GHz with a gig of ram.  I have tried several 
> differant index setups but nothing seems to help.

This should be improved with 7.4, however there are some other things you can
try now.

try

SELECT shoporder 
  FROM sodetailtabletrans 
 WHERE NOT EXISTS (
       SELECT 1
         FROM soheadertable
        WHERE shoporder = sodetailtabletrans.shoporder
       )

or else try something like

         SELECT a.shoporder
           FROM sodetailtabletrans as a
LEFT OUTER JOIN soheadertable as b ON (a.shoporder = b.shoporder)
          WHERE b.shoporder IS NULL


--
greg




^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* Re: slow query - where not in
@ 2003-03-28 16:53  Michael Paesold <mpaesold@gmx.at>
  parent: Bruno Wolff III <bruno@wolff.to>
  0 siblings, 1 reply; 6+ messages in thread

From: Michael Paesold @ 2003-03-28 16:53 UTC (permalink / raw)
  To: Bruno Wolff III <bruno@wolff.to>; Jeremiah Elliott <jelliott@artcobell.com>; +Cc: pgsql-performance

Bruno Wolff III <bruno@wolff.to> wrote:

> Jeremiah Elliott <jelliott@artcobell.com> wrote:
> > here is the query that is killing me:
> >
> > select shoporder from sodetailtabletrans where shoporder not in(select
> > shoporder from soheadertable)
>

> If there are no null values for shoporder in soheadertable or
> sodetailtabletrans you can use not exists instead of not in:
> select shoporder from sodetailtabletrans where shoporder not exists(select
> shoporder from soheadertable)

I think this should rather be:

SELECT shoporder FROM sodetailtabletrans
  WHERE NOT EXISTS (
    SELECT 1 FROM soheadertable
    WHERE soheadertable.shoporder = sodetailtabletrans.shoporder
    )

Regards,
Michael Paesold




^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* Re: slow query - where not in
@ 2003-03-28 17:53  Bruno Wolff III <bruno@wolff.to>
  parent: Michael Paesold <mpaesold@gmx.at>
  0 siblings, 1 reply; 6+ messages in thread

From: Bruno Wolff III @ 2003-03-28 17:53 UTC (permalink / raw)
  To: Michael Paesold <mpaesold@gmx.at>; +Cc: Jeremiah Elliott <jelliott@artcobell.com>; pgsql-performance

On Fri, Mar 28, 2003 at 17:53:46 +0100,
  Michael Paesold <mpaesold@gmx.at> wrote:
> Bruno Wolff III <bruno@wolff.to> wrote:
> 
> I think this should rather be:
> 
> SELECT shoporder FROM sodetailtabletrans
>   WHERE NOT EXISTS (
>     SELECT 1 FROM soheadertable
>     WHERE soheadertable.shoporder = sodetailtabletrans.shoporder
>     )

Thanks for catching my mistake.




^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* Re: slow query - where not in
@ 2003-03-28 20:58  Jeremiah Elliott <jelliott@artcobell.com>
  parent: Bruno Wolff III <bruno@wolff.to>
  0 siblings, 0 replies; 6+ messages in thread

From: Jeremiah Elliott @ 2003-03-28 20:58 UTC (permalink / raw)
  To: pgsql-performance

thanks guys - Greg, Bruno and Michael. That made a world of diferance. 

thx 
-Jeremiah




^ permalink  raw  reply  [nested|flat] 6+ messages in thread


end of thread, other threads:[~2003-03-28 20:58 UTC | newest]

Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2003-03-28 15:38 slow query - where not in Jeremiah Elliott <jelliott@artcobell.com>
2003-03-28 15:59 ` Bruno Wolff III <bruno@wolff.to>
2003-03-28 16:53   ` Michael Paesold <mpaesold@gmx.at>
2003-03-28 17:53     ` Bruno Wolff III <bruno@wolff.to>
2003-03-28 20:58       ` Jeremiah Elliott <jelliott@artcobell.com>
2003-03-28 16:20 ` Greg Stark <gsstark@mit.edu>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox