Received: from localhost (unknown [200.46.204.183]) by postgresql.org (Postfix) with ESMTP id 450172E3D11 for ; Tue, 5 Feb 2008 11:07:31 -0400 (AST) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.183]) (amavisd-maia, port 10024) with ESMTP id 46429-01-7 for ; Tue, 5 Feb 2008 11:07:27 -0400 (AST) Received: from fhw-relay07.plus.net (fhw-relay07.plus.net [212.159.14.215]) by postgresql.org (Postfix) with ESMTP id A66792E4350 for ; Tue, 5 Feb 2008 11:05:24 -0400 (AST) Received: from [84.51.143.99] (helo=server3.office.archonet.com) by fhw-relay07.plus.net with esmtp (Exim) id 1JMPMM-0001XC-MD; Tue, 05 Feb 2008 15:05:18 +0000 Received: from [192.168.1.36] (dell36.office.archonet.com [192.168.1.36]) by server3.office.archonet.com (Postfix) with ESMTP id 04F1F27406F; Tue, 5 Feb 2008 15:05:18 +0000 (GMT) Message-ID: <47A87B2E.7050205@archonet.com> Date: Tue, 05 Feb 2008 15:05:18 +0000 From: Richard Huxton User-Agent: Thunderbird 2.0.0.9 (X11/20071031) MIME-Version: 1.0 To: Simon Riggs CC: Dimitri Fontaine , pgsql-performance@postgresql.org, "Jignesh K. Shah" , Greg Smith Subject: Re: Benchmark Data requested References: <1202150236.4252.521.camel@ebony.site> <47A797E5.4000507@sun.com> <200802051506.51581.dfontaine@hi-media.com> <1202221496.4252.680.camel@ebony.site> <47A87618.1090203@archonet.com> <1202223268.4252.707.camel@ebony.site> In-Reply-To: <1202223268.4252.707.camel@ebony.site> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Plusnet-Relay: 2d0d7e1480d3fa91abcd7227eff296e2 X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200802/36 X-Sequence-Number: 28578 Simon Riggs wrote: > On Tue, 2008-02-05 at 14:43 +0000, Richard Huxton wrote: >> Simon Riggs wrote: >>> On Tue, 2008-02-05 at 15:06 +0100, Dimitri Fontaine wrote: >>>> Le lundi 04 f=C3=A9vrier 2008, Jignesh K. Shah a =C3=A9crit : >>>>> Multiple table loads ( 1 per table) spawned via script is bit bett= er >>>>> but hits wal problems. >>>> pgloader will too hit the WAL problem, but it still may have its ben= efits, or=20 >>>> at least we will soon (you can already if you take it from CVS) be a= ble to=20 >>>> measure if the parallel loading at the client side is a good idea pe= rf. wise. >>> Should be able to reduce lock contention, but not overall WAL volume. >> In the case of a bulk upload to an empty table (or partition?) could y= ou=20 >> not optimise the WAL away? That is, shouldn't the WAL basically be a=20 >> simple transformation of the on-disk blocks? You'd have to explicitly=20 >> sync the file(s) for the table/indexes of course, and you'd need some=20 >> work-around for WAL shipping, but it might be worth it for you chaps=20 >> with large imports. >=20 > Only by locking the table, which serializes access, which then slows yo= u > down or at least restricts other options. Plus if you use pg_loader the= n > you'll find only the first few rows optimized and all the rest not. Hmm - the table-locking requirement is true enough, but why would=20 pg_loader cause problems after the first few rows? --=20 Richard Huxton Archonet Ltd