From: Stefan Kaltenbrunner <stefan@kaltenbrunner.cc>
To: Scott Marlowe <scott.marlowe@gmail.com>
Cc: George Pavlov <gpavlov@mynewplace.com>
Cc: Kynn Jones <kynnjo@gmail.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: How to "unique-ify" HUGE table?
Date: Tue, 23 Dec 2008 20:47:36 +0100
Message-ID: <49514058.9090005@kaltenbrunner.cc> (raw)
In-Reply-To: <dcc563d10812231037p1c5b08cfpe34a363ccaa6b37b@mail.gmail.com>
References: <c2350ba40812230925jb50fed7h3dc58cc311888c7f@mail.gmail.com>
<8C5B026B51B6854CBE88121DBF097A86033B2DED@ehost010-33.exch010.intermedia.net>
<dcc563d10812231037p1c5b08cfpe34a363ccaa6b37b@mail.gmail.com>
Scott Marlowe wrote:
> On Tue, Dec 23, 2008 at 11:14 AM, George Pavlov <gpavlov@mynewplace.com> wrote:
>> You don't say what PG version you are on, but just for kicks you may try
>> using GROUP BY instead of DISTINCT. Yes, the two should perform the
>> same, but with 8.1 (or maybe 8.0) I had seen situations where GROUP BY
>> was faster (admittedly this happened with more complex queries). So, try
>> this:
>
> Even in 8.3 it looks like group by is faster. Tested it on a decent
> sized table and group by used a hash agg and ran in ~600 ms, while
> distinct used a sort and ran in 1300 ms. That was on 500k rows. On a
> much larger table, one with about 10M rows, a similar statement runs
> in 1500 ms with group by and in 2390 ms when run with distinct.
Not surprising - this is a known limitation in all released versions of
postgresql (GROUP BY can use hashing and sorting - DISTINCT only
sorting). 8.4 is going to improve that though.
Stefan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: stefan@kaltenbrunner.cc, scott.marlowe@gmail.com, gpavlov@mynewplace.com, kynnjo@gmail.com
Subject: Re: How to "unique-ify" HUGE table?
In-Reply-To: <49514058.9090005@kaltenbrunner.cc>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox