X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [64.117.224.130]) by svr1.postgresql.org (Postfix) with ESMTP id 8A239D1B4F6 for ; Wed, 10 Sep 2003 17:17:03 +0000 (GMT) Received: from svr1.postgresql.org ([64.117.224.193]) by localhost (neptune.hub.org [64.117.224.130]) (amavisd-new, port 10024) with ESMTP id 37505-03 for ; Wed, 10 Sep 2003 14:16:33 -0300 (ADT) Received: from davinci.ethosmedia.com (unknown [209.128.84.228]) by svr1.postgresql.org (Postfix) with ESMTP id B59C7D1B4E3 for ; Wed, 10 Sep 2003 14:16:31 -0300 (ADT) Received: from [63.195.55.98] (HELO spooky) by davinci.ethosmedia.com (CommuniGate Pro SMTP 4.0.2) with ESMTP id 3627535; Wed, 10 Sep 2003 10:17:00 -0700 Content-Type: text/plain; charset="iso-8859-1" From: Josh Berkus Organization: Aglio Database Solutions To: Chris Huston Subject: Re: Reading data in bulk - help? Date: Wed, 10 Sep 2003 10:16:14 -0700 User-Agent: KMail/1.4.3 Cc: pgsql-performance@postgresql.org References: <91D7A447-E361-11D7-A142-000393011B1A@bangjafwac.com> In-Reply-To: <91D7A447-E361-11D7-A142-000393011B1A@bangjafwac.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: <200309101016.14652.josh@agliodbs.com> X-Virus-Scanned: by amavisd-new at postgresql.org X-Archive-Number: 200309/172 X-Sequence-Number: 3538 Chris, > The system is currently running on a single processor 500Mhz G4. We're > likely to move to a two processor 2Ghz G5 in the next few months. Then > each block may take only a 30-60 milliseconds to complete and their can > be two concurrent blocks processing at once. What about explaining your disk setup? Or mentioning postgresql.conf? For somebody who wants help, you're ignoring a lot of advice and questions. Personally, I'm not going to be of any further help until you report back on the other 3 of 4 options. > RELATED QUESTION: How now do I speed up the following query: "select > distinct group_id from datatable"? Which results in a sequential scan > of the db. Why doesn't it use the group_id index? I only do this once > per run so it's not as critical as the fetch speed which is done 6817 > times. Because it can't until PostgreSQL 7.4, which has hash aggregates. Up to 7.3, we have to use seq scans for all group bys. I'd suggest that you keep a table of group_ids, instead. -- Josh Berkus Aglio Database Solutions San Francisco