X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [64.117.224.130]) by svr1.postgresql.org (Postfix) with ESMTP id 07B93D1CA5E for ; Fri, 3 Oct 2003 20:40:40 +0000 (GMT) Received: from svr1.postgresql.org ([64.117.224.193]) by localhost (neptune.hub.org [64.117.224.130]) (amavisd-new, port 10024) with ESMTP id 39688-03 for ; Fri, 3 Oct 2003 17:39:51 -0300 (ADT) Received: from rlx13.zapatec.com (66-117-144-213.zapatec.lmi.net [66.117.144.213]) by svr1.postgresql.org (Postfix) with ESMTP id A5266D1B582 for ; Fri, 3 Oct 2003 17:21:35 -0300 (ADT) Received: from rlx11.zapatec.com (rlx11.pr.zapatec.com [192.168.1.132]) by rlx13.zapatec.com (Postfix) with ESMTP id 83072ACAC for ; Fri, 3 Oct 2003 20:21:21 +0000 (GMT) Received: (from dror@localhost) by rlx11.zapatec.com (8.12.3/8.12.3/Submit) id h93KLK2N097195 for pgsql-performance@postgresql.org; Fri, 3 Oct 2003 13:21:20 -0700 (PDT) (envelope-from dror) Date: Fri, 3 Oct 2003 13:21:20 -0700 From: Dror Matalon To: pgsql-performance@postgresql.org Subject: Speeding up Aggregates Message-ID: <20031003202120.GO87525@rlx11.zapatec.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Virus-Scanned: by amavisd-new at postgresql.org X-Archive-Number: 200310/103 X-Sequence-Number: 3851 Hi, I have a query that ran quite well initially, but slowed down quite a bit once I introduced an aggregate into the equation. The average execution time went up from around 15 msec to around 300 msec. The original query fetches a bunch of articles: select articlenumber, channel, description, title, link, dtstamp from items, my_channels where items.channel = '22222' and my_channels.id = '22222' and owner = 'drormata' and dtstamp > last_viewed and articlenumber not in (select item from viewed_items where channel ='22222' and owner = 'drormata'); I then added a call to a function: and (dtstamp = item_max_date(22222, link)) item_max_date() looks like this: select max(dtstamp) from items where channel = $1 and link = $2; This should eliminate duplicate articles and only show the most recent one. resulting in the following query select articlenumber, channel, description, title, link, dtstamp from items, my_channels where items.channel = '22222' and my_channels.id = '22222' and owner = 'drormata' and dtstamp > last_viewed and articlenumber not in (select item from viewed_items where channel ='22222' and owner = 'drormata') and (dtstamp = item_max_date(22222, link)); Any suggestions on optimizing the query/function? It makes sense that it slowed down, but I wonder if I can do better. I'm including index list as well as "explain analyze" of both versions. Indexes: "item_channel_link" btree (channel, link) "item_created" btree (dtstamp) "item_signature" btree (signature) "items_channel_article" btree (channel, articlenumber) explain analyze select articlenumber, channel, description, title, link, dtstamp from items, my_channels where items.channel = '22222' and my_channels.id = '22222' and owner = 'drormata' and dtstamp > last_viewed and articlenumber not in (select item from viewed_items where channel ='22222' and owner = 'drormata'); QUERY PLAN ------------------------------------------------------------------------------------------------------------------------------------------- Nested Loop (cost=8.19..6982.58 rows=302 width=259) (actual time=16.95..17.16 rows=8 loops=1) Join Filter: ("inner".dtstamp > "outer".last_viewed) -> Seq Scan on my_channels (cost=0.00..3.23 rows=1 width=8) (actual time=0.36..0.38 rows=1 loops=1) Filter: ((id = 22222) AND (("owner")::text = 'drormata'::text)) -> Index Scan using items_channel_article on items (cost=8.19..6968.05 rows=904 width=259) (actual time=0.68..13.94 rows=899 loops=1) Index Cond: (channel = 22222) Filter: (NOT (hashed subplan)) SubPlan -> Seq Scan on viewed_items (cost=0.00..8.19 rows=2 width=4) (actual time=0.48..0.48 rows=0 loops=1) Filter: ((channel = 22222) AND (("owner")::text = 'drormata'::text)) Total runtime: 17.42 msec (11 rows) explain analyze select articlenumber, channel, description, title, link, dtstamp from items, my_channels where items.channel = '22222' and my_channels.id = '22222' and owner = 'drormata' and dtstamp > last_viewed and articlenumber not in (select item from viewed_items where channel ='22222' and owner = 'drormata') and (dtstamp = item_max_date(22222, link)); QUERY PLAN ------------------------------------------------------------------------------------------------------------------------------------------ Nested Loop (cost=8.19..6980.33 rows=1 width=259) (actual time=262.94..265.14 rows=7 loops=1) Join Filter: ("outer".dtstamp > "inner".last_viewed) -> Index Scan using items_channel_article on items (cost=8.19..6977.08 rows=1 width=259) (actual time=1.94..150.55 rows=683 loops=1) Index Cond: (channel = 22222) Filter: ((dtstamp = item_max_date(22222, link)) AND (NOT (hashed subplan))) SubPlan -> Seq Scan on viewed_items (cost=0.00..8.19 rows=2 width=4) (actual time=0.43..0.43 rows=0 loops=1) Filter: ((channel = 22222) AND (("owner")::text = 'drormata'::text)) -> Seq Scan on my_channels (cost=0.00..3.23 rows=1 width=8) (actual time=0.14..0.15 rows=1 loops=683) Filter: ((id = 22222) AND (("owner")::text = 'drormata'::text)) Total runtime: 265.39 msec -- Dror Matalon Zapatec Inc 1700 MLK Way Berkeley, CA 94709 http://www.zapatec.com