X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id ECBC653829 for ; Tue, 26 Apr 2005 17:17:01 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 02795-07 for ; Tue, 26 Apr 2005 20:16:59 +0000 (GMT) Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.206]) by svr1.postgresql.org (Postfix) with ESMTP id B0989537EA for ; Tue, 26 Apr 2005 17:16:57 -0300 (ADT) Received: by wproxy.gmail.com with SMTP id 68so65990wri for ; Tue, 26 Apr 2005 13:16:57 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=HBvTo4hWSpKI3UzcqmxvmxM5WYcZhy1EDEY72if2VOfbpW1+N98vNoGihUWOaWWxut/7U3cJXCPRE77T5JTS3R0x8yUF+wZRJTxjXVXww+VW6Ix+W/WljAVmiICCyyuKl8hhobiBNCLUKB1nHjUa1J27f2RMxXeqdkpNP8if6xo= Received: by 10.54.78.16 with SMTP id a16mr84201wrb; Tue, 26 Apr 2005 13:16:57 -0700 (PDT) Received: by 10.54.22.6 with HTTP; Tue, 26 Apr 2005 13:16:56 -0700 (PDT) Message-ID: Date: Tue, 26 Apr 2005 15:16:57 -0500 From: Matthew Nuzum Reply-To: newz@bearfruit.org To: pgsql-performance@postgresql.org Subject: speed up query with max() and odd estimates Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.336 tagged_above=0 required=5 tests=AWL, DNS_FROM_RFC_ABUSE, RCVD_BY_IP X-Spam-Level: X-Archive-Number: 200504/703 X-Sequence-Number: 12100 I have this query that takes a little over 8 min to run: select client,max(atime) as atime from usage_access where atime >=3D (select atime - '1 hour'::interval from usage_access order by atime desc limit 1) group by client; I think it can go a lot faster. Any suggestions on improving this? DB is 7.3.4 I think. (There is no index on client because it is very big and this data is used infrequently.) explain ANALYZE select client,max(atime) as atime from usage_access where atime >=3D (select atime - '1 hour'::interval from usage_access order by atime desc limit 1) group by client; =20 QUERY PLAN ---------------------------------------------------------------------------= ---------------------------------------------------------------------------= -------------- Aggregate (cost=3D3525096.28..3620450.16 rows=3D1271385 width=3D20) (actual time=3D482676.95..482693.69 rows=3D126 loops=3D1) InitPlan -> Limit (cost=3D0.00..0.59 rows=3D1 width=3D8) (actual time=3D0.40..0.41 rows=3D1 loops=3D1) -> Index Scan Backward using usage_access_atime on usage_access (cost=3D0.00..22657796.18 rows=3D38141552 width=3D8) (actual time=3D0.39..0.40 rows=3D2 loops=3D1) -> Group (cost=3D3525096.28..3588665.53 rows=3D12713851 width=3D20) (actual time=3D482676.81..482689.29 rows=3D3343 loops=3D1) -> Sort (cost=3D3525096.28..3556880.90 rows=3D12713851 width=3D20) (actual time=3D482676.79..482679.16 rows=3D3343 loops=3D1) Sort Key: client -> Seq Scan on usage_access (cost=3D0.00..1183396.40 rows=3D12713851 width=3D20) (actual time=3D482641.57..482659.18 rows=3D3343 loops=3D1) Filter: (atime >=3D $0) Total runtime: 482694.65 msec I'm starting to understand this, which is quite frightening to me. I thought that maybe if I shrink the number of rows down I could improve things a bit, but my first attempt didn't work. I thought I'd replace the "from usage_access" with this query instead: select * from usage_access where atime >=3D (select atime - '1 hour'::interval from usage_access order by atime desc limit 1); =20 QUERY PLAN ---------------------------------------------------------------------------= ---------------------------------------------------------------------------= -------------- Seq Scan on usage_access (cost=3D0.00..1183396.40 rows=3D12713851 width=3D116) (actual time=3D481796.22..481839.43 rows=3D3343 loops=3D1) Filter: (atime >=3D $0) InitPlan -> Limit (cost=3D0.00..0.59 rows=3D1 width=3D8) (actual time=3D0.41..0.42 rows=3D1 loops=3D1) -> Index Scan Backward using usage_access_atime on usage_access (cost=3D0.00..22657796.18 rows=3D38141552 width=3D8) (actual time=3D0.40..0.41 rows=3D2 loops=3D1) Total runtime: 481842.47 msec It doesn't look like this will help at all. This table is primarily append, however I just recently deleted a few million rows from the table, if that helps anyone. --=20 Matthew Nuzum www.bearfruit.org