pg.ddx.io  pgsql-sql@postgresql.org mailing list archive  
help / color / mirror / Atom feed
Long-running DELETE...WHERE...
5+ messages / 5 participants
[nested] [flat]

* Long-running DELETE...WHERE...
@ 2002-01-14 18:27  Jeff Boes <jboes@nexcerpt.com>
  0 siblings, 3 replies; 5+ messages in thread

From: jboes@nexcerpt.com @ 2002-01-14 18:27 UTC (permalink / raw)
  To: pgsql-sql

Why would a delete involving a subselect run so much longer than the
individual delete commands?

My situation: table A has 200,000 rows. I've made up a temporary table
which holds the single-valued primary key for 80,000 rows which I want
to delete.

  DELETE FROM a WHERE id IN (select ID from tmp LIMIT 800);

runs for several minutes.  But if I do

\o tmpfile
\t
SELECT 'DELETE FROM a WHERE id = ' || id || ';' from tmp limit 800;
\o
\i tmpfile

this completes in about 15 seconds, or 1/50 of the time for the
single-statement delete above.

In trying to optimize this process, I disabled all the relational
integrity triggers (foreign keys) involving the table, and then I
dropped all the indexes EXCEPT that of the primary key.  All the
experiments were done within a single transaction using BEGIN. The
database version is 7.1.3, and the table was vacuumed very recently.



^ permalink  raw  reply  [nested|flat] 5+ messages in thread

* Re: Long-running DELETE...WHERE...
@ 2002-01-14 22:47  Jochem van Dieten <nomail@dev.null>
  parent: Jeff Boes <jboes@nexcerpt.com>
  2 siblings, 0 replies; 5+ messages in thread

From: Jochem van Dieten @ 2002-01-14 22:47 UTC (permalink / raw)
  To: pgsql-sql

Jeff Boes wrote:

> Why would a delete involving a subselect run so much longer than the
> individual delete commands?


Some SQL statements are faster than others. Try to rewrite your query to 
use EXISTS instead of IN and see if it makes any difference. I would 
recommend using EXPLAIN to find out what PostgreSQL is really doing and 
optimize from there.

jochem




^ permalink  raw  reply  [nested|flat] 5+ messages in thread

* Re: Long-running DELETE...WHERE...
@ 2002-01-15 01:21  Tom Lane <tgl@sss.pgh.pa.us>
  parent: Jeff Boes <jboes@nexcerpt.com>
  2 siblings, 1 reply; 5+ messages in thread

From: Tom Lane @ 2002-01-15 01:21 UTC (permalink / raw)
  To: jboes@nexcerpt.com; +Cc: pgsql-sql

jboes@nexcerpt.com (Jeff Boes) writes:
> My situation: table A has 200,000 rows. I've made up a temporary table
> which holds the single-valued primary key for 80,000 rows which I want
> to delete.
>   DELETE FROM a WHERE id IN (select ID from tmp LIMIT 800);
> runs for several minutes.

WHERE ... IN ... is notoriously inefficient.  I'd try

	DELETE FROM aa WHERE id = tmp.id;

which is not standard SQL but should be able to produce a decent plan.

You might find that a VACUUM ANALYZE on both tables beforehand would be
a good idea, too; never-vacuumed temp tables have some default
statistics assumed that are a lot less than 80k rows.

			regards, tom lane



^ permalink  raw  reply  [nested|flat] 5+ messages in thread

* Re: Long-running DELETE...WHERE...
@ 2002-01-15 02:04  Chester Carlton Young <chestercyoung@yahoo.com>
  parent: Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 5+ messages in thread

From: Chester Carlton Young @ 2002-01-15 02:04 UTC (permalink / raw)
  To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-sql

Nice syntax.  Could not find in doc.  Do you have any ideas where I
could find it?

--- Tom Lane <tgl@sss.pgh.pa.us> wrote:
> jboes@nexcerpt.com (Jeff Boes) writes:
> > My situation: table A has 200,000 rows. I've made up a temporary
> table
> > which holds the single-valued primary key for 80,000 rows which I
> want
> > to delete.
> >   DELETE FROM a WHERE id IN (select ID from tmp LIMIT 800);
> > runs for several minutes.
> 
> WHERE ... IN ... is notoriously inefficient.  I'd try
> 
> 	DELETE FROM aa WHERE id = tmp.id;
> 
> which is not standard SQL but should be able to produce a decent
> plan.
> 
> You might find that a VACUUM ANALYZE on both tables beforehand would
> be
> a good idea, too; never-vacuumed temp tables have some default
> statistics assumed that are a lot less than 80k rows.
> 
> 			regards, tom lane
> 
> ---------------------------(end of
> broadcast)---------------------------
> TIP 3: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo@postgresql.org so that your
> message can get through to the mailing list cleanly


__________________________________________________
Do You Yahoo!?
Send FREE video emails in Yahoo! Mail!
http://promo.yahoo.com/videomail/



^ permalink  raw  reply  [nested|flat] 5+ messages in thread

* Re: Long-running DELETE...WHERE...
@ 2002-01-15 19:22  Frank Bax <fbax@sympatico.ca>
  parent: Jeff Boes <jboes@nexcerpt.com>
  2 siblings, 0 replies; 5+ messages in thread

From: Frank Bax @ 2002-01-15 19:22 UTC (permalink / raw)
  To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: jboes@nexcerpt.com; pgsql-sql

At 08:21 PM 1/14/02 -0500, Tom Lane wrote:
>jboes@nexcerpt.com (Jeff Boes) writes:
>> My situation: table A has 200,000 rows. I've made up a temporary table
>> which holds the single-valued primary key for 80,000 rows which I want
>> to delete.
>>   DELETE FROM a WHERE id IN (select ID from tmp LIMIT 800);
>> runs for several minutes.
>
>WHERE ... IN ... is notoriously inefficient.  I'd try
>
>	DELETE FROM aa WHERE id = tmp.id;
>
>which is not standard SQL but should be able to produce a decent plan.

Nice alternative.  Is there an alternate format for this one:

DELETE  FROM  teamwork  WHERE  emp  NOT IN
    ( SELECT  DISTINCT  emp  FROM  timesheet
       WHERE  lo_shift > (now()-'90days'::interval) )

Frank




^ permalink  raw  reply  [nested|flat] 5+ messages in thread


end of thread, other threads:[~2002-01-15 19:22 UTC | newest]

Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2002-01-14 18:27 Long-running DELETE...WHERE... Jeff Boes <jboes@nexcerpt.com>
2002-01-14 22:47 ` Jochem van Dieten <nomail@dev.null>
2002-01-15 01:21 ` Tom Lane <tgl@sss.pgh.pa.us>
2002-01-15 02:04   ` Chester Carlton Young <chestercyoung@yahoo.com>
2002-01-15 19:22 ` Frank Bax <fbax@sympatico.ca>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox