From: Chad Wagner <chad.wagner@gmail.com>
To: Simon Riggs <simon@2ndquadrant.com>
Cc: Igor Lobanov <ilobanov@swsoft.com>
Cc: Richard Huxton <dev@archonet.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: Querying distinct values from a large table
Date: Tue, 30 Jan 2007 09:13:27 -0500
Message-ID: <81961ff50701300613g25ea4ce6jb357c82fb1ed6733@mail.gmail.com> (raw)
In-Reply-To: <1170165856.3681.54.camel@silverbirch.site>
References: <45BF02DE.7080605@swsoft.com>
<45BF0C13.3070003@archonet.com>
<45BF10D7.3040203@swsoft.com>
<1170165856.3681.54.camel@silverbirch.site>
On 1/30/07, Simon Riggs <simon@2ndquadrant.com> wrote:
>
> > explain analyze select distinct a, b from tbl
> >
> > EXPLAIN ANALYZE output is:
> >
> > Unique (cost=500327.32..525646.88 rows=1848 width=6) (actual
> > time=52719.868..56126.356 rows=5390 loops=1)
> > -> Sort (cost=500327.32..508767.17 rows=3375941 width=6) (actual
> > time=52719.865..54919.989 rows=3378864 loops=1)
> > Sort Key: a, b
> > -> Seq Scan on tbl (cost=0.00..101216.41 rows=3375941
> > width=6) (actual time=16.643..20652.610 rows=3378864 loops=1)
> > Total runtime: 57307.394 ms
>
> All your time is in the sort, not in the SeqScan.
>
> Increase your work_mem.
>
Sounds like an opportunity to implement a "Sort Unique" (sort of like a
hash, I guess), there is no need to push 3M rows through a sort algorithm to
only shave it down to 1848 unique records.
I am assuming this optimization just isn't implemented in PostgreSQL?
--
Chad
http://www.postgresqlforums.com/
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: chad.wagner@gmail.com, simon@2ndquadrant.com, ilobanov@swsoft.com, dev@archonet.com
Subject: Re: Querying distinct values from a large table
In-Reply-To: <81961ff50701300613g25ea4ce6jb357c82fb1ed6733@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox