X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 9204832ABC7 for ; Thu, 21 Oct 2004 16:42:04 +0100 (BST) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 62336-08 for ; Thu, 21 Oct 2004 15:41:55 +0000 (GMT) Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130]) by svr1.postgresql.org (Postfix) with ESMTP id 7C82332A452 for ; Thu, 21 Oct 2004 16:41:51 +0100 (BST) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.13.1/8.13.1) with ESMTP id i9LFfnUk016018; Thu, 21 Oct 2004 11:41:49 -0400 (EDT) To: Victor Ciurus Cc: pgsql-performance@postgresql.org Subject: Re: Simple machine-killing query! In-reply-to: References: Comments: In-reply-to Victor Ciurus message dated "Thu, 21 Oct 2004 17:34:17 +0300" Date: Thu, 21 Oct 2004 11:41:48 -0400 Message-ID: <16017.1098373308@sss.pgh.pa.us> From: Tom Lane X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.0 tagged_above=0.0 required=5.0 tests= X-Spam-Level: X-Archive-Number: 200410/357 X-Sequence-Number: 8833 Victor Ciurus writes: > What I am requested to do is to keep all records from 'BIGMA' that do > not apear in 'DIRTY' > So far I have tried solving this by going for: > [explain] select * from BIGMA where string not in (select * from DIRTY); > QUERY PLAN > ------------------------------------------------------------------------ > Seq Scan on bigma (cost=0.00..24582291.25 rows=500 width=145) > Filter: (NOT (subplan)) > SubPlan > -> Seq Scan on dirty (cost=0.00..42904.63 rows=2503963 width=82) > (4 rows) If you are using PG 7.4, you can get reasonable performance out of this approach, but you need to jack sort_mem up to the point where the whole DIRTY table will fit into sort_mem (so that you get a hashed-subplan plan and not a plain subplan). If you find yourself setting sort_mem to more than say half of your machine's available RAM, you should probably forget that idea. > [explain] select * from bigma,dirty where bigma.email!=dirty.email; This of course does not give the right answer at all. A trick that people sometimes use is an outer join: select * from bigma left join dirty on (bigma.email=dirty.email) where dirty.email is null; Understanding why this works is left as an exercise for the reader ... but it does work, and pretty well too. If you're using pre-7.4 PG then this is about the only effective solution AFAIR. regards, tom lane