From: Luke Lonergan <llonergan@greenplum.com>
To: Chad Wagner <chad.wagner@gmail.com>
To: Simon Riggs <simon@2ndquadrant.com>
Cc: Igor Lobanov <ilobanov@swsoft.com>
Cc: Richard Huxton <dev@archonet.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: Querying distinct values from a large table
Date: Tue, 30 Jan 2007 06:56:57 -0800
Message-ID: <C1E49CB9.199A4%llonergan@greenplum.com> (raw)
In-Reply-To: <81961ff50701300613g25ea4ce6jb357c82fb1ed6733@mail.gmail.com>
Chad,
On 1/30/07 6:13 AM, "Chad Wagner" <chad.wagner@gmail.com> wrote:
> Sounds like an opportunity to implement a "Sort Unique" (sort of like a hash,
> I guess), there is no need to push 3M rows through a sort algorithm to only
> shave it down to 1848 unique records.
>
> I am assuming this optimization just isn't implemented in PostgreSQL?
Not that it helps Igor, but we've implemented single pass sort/unique,
grouping and limit optimizations and it speeds things up to a single seqscan
over the data, from 2-5 times faster than a typical external sort.
I can't think of a way that indexing would help this situation given the
required visibility check of each tuple.
- Luke
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: llonergan@greenplum.com, chad.wagner@gmail.com, simon@2ndquadrant.com, ilobanov@swsoft.com, dev@archonet.com
Subject: Re: Querying distinct values from a large table
In-Reply-To: <C1E49CB9.199A4%llonergan@greenplum.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox