Message-ID: <49514058.9090005@kaltenbrunner.cc>
Date: Tue, 23 Dec 2008 20:47:36 +0100
From: Stefan Kaltenbrunner <stefan@kaltenbrunner.cc>
User-Agent: Mozilla-Thunderbird 2.0.0.17 (X11/20081018)
MIME-Version: 1.0
To: Scott Marlowe <scott.marlowe@gmail.com>
CC: George Pavlov <gpavlov@mynewplace.com>, Kynn Jones <kynnjo@gmail.com>,
	pgsql-performance@postgresql.org
Subject: Re: How to "unique-ify" HUGE table?
References: <c2350ba40812230925jb50fed7h3dc58cc311888c7f@mail.gmail.com>	
	<8C5B026B51B6854CBE88121DBF097A86033B2DED@ehost010-33.exch010.intermedia.net>
	<dcc563d10812231037p1c5b08cfpe34a363ccaa6b37b@mail.gmail.com>
In-Reply-To: <dcc563d10812231037p1c5b08cfpe34a363ccaa6b37b@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Scott Marlowe wrote:
> On Tue, Dec 23, 2008 at 11:14 AM, George Pavlov <gpavlov@mynewplace.com> wrote:
>> You don't say what PG version you are on, but just for kicks you may try
>> using GROUP BY instead of DISTINCT. Yes, the two should perform the
>> same, but with 8.1 (or maybe 8.0) I had seen situations where GROUP BY
>> was faster (admittedly this happened with more complex queries). So, try
>> this:
> 
> Even in 8.3 it looks like group by is faster.  Tested it on a decent
> sized table and group by used a hash agg and ran in ~600 ms, while
> distinct used a sort and ran in 1300 ms.  That was on 500k rows.  On a
> much larger table, one with about 10M rows, a similar statement runs
> in 1500 ms with group by and in 2390 ms when run with distinct.

Not surprising - this is a known limitation in all released versions of 
postgresql (GROUP BY can use hashing and sorting - DISTINCT only 
sorting). 8.4 is going to improve that though.


Stefan