Subject: Re: Benchmark Data requested
From: Simon Riggs <simon@2ndquadrant.com>
To: Richard Huxton <dev@archonet.com>
Cc: Dimitri Fontaine <dfontaine@hi-media.com>,
	pgsql-performance@postgresql.org, "Jignesh K. Shah" <J.K.Shah@sun.com>,
	Greg Smith <gsmith@gregsmith.com>
In-Reply-To: <47A87618.1090203@archonet.com>
References: <1202150236.4252.521.camel@ebony.site>
	<Pine.GSO.4.64.0802041500460.799@westnet.com>
	<47A797E5.4000507@sun.com>
	<200802051506.51581.dfontaine@hi-media.com>
	<1202221496.4252.680.camel@ebony.site> <47A87618.1090203@archonet.com>
Content-Type: text/plain; charset=utf-8
Date: Tue, 05 Feb 2008 14:54:28 +0000
Message-Id: <1202223268.4252.707.camel@ebony.site>
Mime-Version: 1.0
Content-Transfer-Encoding: 8bit

On Tue, 2008-02-05 at 14:43 +0000, Richard Huxton wrote:
> Simon Riggs wrote:
> > On Tue, 2008-02-05 at 15:06 +0100, Dimitri Fontaine wrote:
> >>
> >> Le lundi 04 février 2008, Jignesh K. Shah a écrit :
> 
> >>> Multiple table loads ( 1 per table) spawned via script  is bit better
> >>> but hits wal problems.
> >> pgloader will too hit the WAL problem, but it still may have its benefits, or 
> >> at least we will soon (you can already if you take it from CVS) be able to 
> >> measure if the parallel loading at the client side is a good idea perf. wise.
> > 
> > Should be able to reduce lock contention, but not overall WAL volume.
> 
> In the case of a bulk upload to an empty table (or partition?) could you 
> not optimise the WAL away? That is, shouldn't the WAL basically be a 
> simple transformation of the on-disk blocks? You'd have to explicitly 
> sync the file(s) for the table/indexes of course, and you'd need some 
> work-around for WAL shipping, but it might be worth it for you chaps 
> with large imports.

Only by locking the table, which serializes access, which then slows you
down or at least restricts other options. Plus if you use pg_loader then
you'll find only the first few rows optimized and all the rest not.

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com