Subject: Re: Benchmark Data requested
From: Simon Riggs <simon@2ndquadrant.com>
To: Dimitri Fontaine <dfontaine@hi-media.com>
Cc: pgsql-performance@postgresql.org, "Jignesh K. Shah" <J.K.Shah@sun.com>,
	Greg Smith <gsmith@gregsmith.com>
In-Reply-To: <200802051815.25800.dfontaine@hi-media.com>
References: <1202150236.4252.521.camel@ebony.site>
	<200802051506.51581.dfontaine@hi-media.com>
	<1202221496.4252.680.camel@ebony.site>
	<200802051815.25800.dfontaine@hi-media.com>
Content-Type: text/plain; charset=utf-8
Date: Tue, 05 Feb 2008 17:32:33 +0000
Message-Id: <1202232753.4252.789.camel@ebony.site>
Mime-Version: 1.0
Content-Transfer-Encoding: 8bit

On Tue, 2008-02-05 at 18:15 +0100, Dimitri Fontaine wrote:
> Le mardi 05 février 2008, Simon Riggs a écrit :
> > I'll look at COPY FROM internals to make this faster. I'm looking at
> > this now to refresh my memory; I already had some plans on the shelf.
> 
> Maybe stealing some ideas from pg_bulkload could somewhat help here?
>   http://pgfoundry.org/docman/view.php/1000261/456/20060709_pg_bulkload.pdf

> IIRC it's mainly about how to optimize index updating while loading data, and 
> I've heard complaints on the line "this external tool has to know too much 
> about PostgreSQL internals to be trustworthy as non-core code"... so...

Yeh, the batch index updates are a cool feature. Should be able to do
that internally also.

Not going to try the no-WAL route again though. If we can get it running
efficiently and in parallel, then that will be OK. 

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com