Received: from magus.postgresql.org (magus.postgresql.org [87.238.57.229]) by mail.postgresql.org (Postfix) with ESMTP id 229A110D80A7 for ; Thu, 17 Nov 2011 20:46:33 -0400 (AST) Received: from smtp.01.com ([216.115.88.169]) by magus.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1RRCb9-00083j-Cy for pgsql-performance@postgresql.org; Fri, 18 Nov 2011 00:46:32 +0000 Received: from localhost (localhost.localdomain [127.0.0.1]) by smtp-out-2.01.com (Postfix) with ESMTP id 80B20A08294 for ; Thu, 17 Nov 2011 18:46:11 -0600 (CST) X-Virus-Scanned: amavisd-new at smtp-out-2.01.com Received: from smtp.01.com ([127.0.0.1]) by localhost (smtp-out-2.01.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eU0LlAQ5287P for ; Thu, 17 Nov 2011 18:46:11 -0600 (CST) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp-out-2.01.com (Postfix) with ESMTP id 62923A082A0 for ; Thu, 17 Nov 2011 18:46:11 -0600 (CST) Received: from Sidney-Stratton.local (70-36-143-92.dsl.dynamic.sonic.net [70.36.143.92]) by smtp-out-2.01.com (Postfix) with ESMTPSA id 36B16A08294 for ; Thu, 17 Nov 2011 18:46:11 -0600 (CST) Message-ID: <4EC5AACD.9010802@agliodbs.com> Date: Thu, 17 Nov 2011 16:46:05 -0800 From: Josh Berkus Organization: PostgreSQL Experts Inc. User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.2.24) Gecko/20111103 Lightning/1.0b2 Thunderbird/3.1.16 MIME-Version: 1.0 To: pgsql-performance@postgresql.org Subject: Re: external sort performance References: In-Reply-To: X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Archive-Number: 201111/227 X-Sequence-Number: 45738 > The first log statement is expected. The second log statement, however, isn't. > The total table size is (as noted earlier) about 5GB and, in fact, fit > into one nice hash table (approx 15GB in size). > Is the sorting that is necessary for index creation unable to use a > hash table? (This is a standard btree index). How big is the source table? You're not sorting the *result* table, you're sorting the source table if you're summarizing it. If the original source data is only 5GB, I'd check your code for a cartesian join. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com