X-Original-To: pgsql-patches-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id A2E00529D8 for ; Thu, 21 Jul 2005 18:55:08 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 54040-08 for ; Thu, 21 Jul 2005 21:55:02 +0000 (GMT) Received: from smtp.osdl.org (smtp.osdl.org [65.172.181.4]) by svr1.postgresql.org (Postfix) with ESMTP id D9A3B52999 for ; Thu, 21 Jul 2005 18:54:58 -0300 (ADT) Received: from ibm-b.pdx.osdl.net (fw.osdl.org [65.172.181.6]) (authenticated bits=0) by smtp.osdl.org (8.12.8/8.12.8) with ESMTP id j6LLsdjA002847 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO); Thu, 21 Jul 2005 14:54:39 -0700 Message-Id: <200507212154.j6LLsdjA002847@smtp.osdl.org> Date: Thu, 21 Jul 2005 14:55:07 -0700 From: Mark Wong To: "Luke Lonergan" Cc: "Andrew Dunstan" , "Alvaro Herrera" , "Bruce Momjian" , "Alon Goldshuv" , pgsql-patches@postgresql.org, maryedie@osdl.org Subject: Re: COPY FROM performance improvements In-Reply-To: References: <42C05F88.7000105@dunslane.net> Organization: OSDL X-Mailer: Sylpheed-Claws 1.0.4a (GTK+ 1.2.10; i686-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-MIMEDefang-Filter: osdl$Revision: 1.113 $ X-Scanned-By: MIMEDefang 2.36 X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.057 tagged_above=0 required=5 tests=AWL, FORGED_RCVD_HELO X-Spam-Level: X-Archive-Number: 200507/424 X-Sequence-Number: 16764 I just ran through a few tests with the v14 patch against 100GB of data from dbt3 and found a 30% improvement; 3.6 hours vs 5.3 hours. Just to give a few details, I only loaded data and started a COPY in parallel for each the data files: http://www.testing.osdl.org/projects/dbt3testing/results/fast_copy/ Here's a visual of my disk layout, for those familiar with the database schema: http://www.testing.osdl.org/projects/dbt3testing/results/fast_copy/layout-dev4-010-dbt3.html I have 6 arrays of fourteen 15k rpm drives in a split-bus configuration attached to a 4-way itanium2 via 6 compaq smartarray pci-x controllers. Let me know if you have any questions. Mark