DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma;
	h=message-id:date:from:to:subject:cc:in-reply-to:mime-version
	:content-type:content-transfer-encoding:content-disposition
	:references;
	b=qcPPdZR7ILlJ+yiqZ6CiLXKkuL+jwKPRVYXZFPcA0G56Ez/PCFEkaNDUj5aj7ldwIz
	iH9oVaiw0kJqCokdXsTijxJCfBhs8N+aNkgePDuC4babxeew/qLbSBqdG0FHm4R6Ym/O
	4bavCgv4bo19KpJ9B+IcFUsAZNSKK/eYHSlX0=
Message-ID: <dcc563d10812231037p1c5b08cfpe34a363ccaa6b37b@mail.gmail.com>
Date: Tue, 23 Dec 2008 11:37:07 -0700
From: "Scott Marlowe" <scott.marlowe@gmail.com>
To: "George Pavlov" <gpavlov@mynewplace.com>
Subject: Re: How to "unique-ify" HUGE table?
Cc: "Kynn Jones" <kynnjo@gmail.com>, pgsql-performance@postgresql.org
In-Reply-To: 
 <8C5B026B51B6854CBE88121DBF097A86033B2DED@ehost010-33.exch010.intermedia.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <c2350ba40812230925jb50fed7h3dc58cc311888c7f@mail.gmail.com>
	<8C5B026B51B6854CBE88121DBF097A86033B2DED@ehost010-33.exch010.intermedia.net>

On Tue, Dec 23, 2008 at 11:14 AM, George Pavlov <gpavlov@mynewplace.com> wrote:
> You don't say what PG version you are on, but just for kicks you may try
> using GROUP BY instead of DISTINCT. Yes, the two should perform the
> same, but with 8.1 (or maybe 8.0) I had seen situations where GROUP BY
> was faster (admittedly this happened with more complex queries). So, try
> this:

Even in 8.3 it looks like group by is faster.  Tested it on a decent
sized table and group by used a hash agg and ran in ~600 ms, while
distinct used a sort and ran in 1300 ms.  That was on 500k rows.  On a
much larger table, one with about 10M rows, a similar statement runs
in 1500 ms with group by and in 2390 ms when run with distinct.