Received: from localhost (wm.hub.org [200.46.204.128]) by postgresql.org (Postfix) with ESMTP id 34B3F9FB1C8 for ; Wed, 27 Sep 2006 17:24:33 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.128]) (amavisd-new, port 10024) with ESMTP id 27120-06 for ; Wed, 27 Sep 2006 20:24:29 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey- Received: from wolff.to (wolff.to [66.93.197.194]) by postgresql.org (Postfix) with SMTP id 8AD059FA21B for ; Wed, 27 Sep 2006 17:24:28 -0300 (ADT) Received: (qmail 9807 invoked by uid 500); 27 Sep 2006 20:27:46 -0000 Date: Wed, 27 Sep 2006 15:27:46 -0500 From: Bruno Wolff III To: TJ O'Donnell Cc: pgsql-sql@postgresql.org Subject: Re: unique rows Message-ID: <20060927202746.GA9702@wolff.to> Mail-Followup-To: Bruno Wolff III , TJ O'Donnell , pgsql-sql@postgresql.org References: <4512208C.9050607@acm.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4512208C.9050607@acm.org> User-Agent: Mutt/1.4.2.1i X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=0.315 tagged_above=0 required=5 tests=SARE_MILLIONSOF X-Spam-Level: X-Archive-Number: 200609/171 X-Sequence-Number: 26382 On Wed, Sep 20, 2006 at 22:18:04 -0700, TJ O'Donnell wrote: > I want to make a table that has unique rows - no problem. > Create Table x (smarts Varchar Unique); > > I have input data that has many duplicates and until now > I was creating a temp table allowing duplicates, and then > Insert into x (smarts) select distinct smarts from tmpx; > This is now failing when I have hundreds of millions of > rows, running out of temp space (while trying to sort to > accomplish the distinct?) Another option would be to do a select with a GROUP BY instead of DISTINCT. That would allow for a hash aggregate plan (don't forget to analyse the temp table first) which would use less memory.