Date: Tue, 05 Feb 2008 13:47:13 -0500
From: "Jignesh K. Shah" <J.K.Shah@Sun.COM>
Subject: Re: Benchmark Data requested
In-reply-to: <1202232753.4252.789.camel@ebony.site>
To: Simon Riggs <simon@2ndquadrant.com>
Cc: Dimitri Fontaine <dfontaine@hi-media.com>,
	pgsql-performance@postgresql.org, Greg Smith <gsmith@gregsmith.com>
Message-id: <47A8AF31.8070304@sun.com>
Organization: Sun Microsystems
MIME-version: 1.0
Content-type: text/plain; format=flowed; charset=UTF-8
Content-transfer-encoding: 8BIT
References: <1202150236.4252.521.camel@ebony.site>
	<200802051506.51581.dfontaine@hi-media.com>
	<1202221496.4252.680.camel@ebony.site>
	<200802051815.25800.dfontaine@hi-media.com>
	<1202232753.4252.789.camel@ebony.site>
User-Agent: Thunderbird 2.0.0.4 (X11/20070622)


That sounds cool to me too..

How much work is to make pg_bulkload to work on 8.3? An Integrated 
version is certainly more beneficial.

Specially I think it will also help for other setups like TPC-E too 
where this is a problem.

Regards,
Jignesh


Simon Riggs wrote:
> On Tue, 2008-02-05 at 18:15 +0100, Dimitri Fontaine wrote:
>   
>> Le mardi 05 février 2008, Simon Riggs a écrit :
>>     
>>> I'll look at COPY FROM internals to make this faster. I'm looking at
>>> this now to refresh my memory; I already had some plans on the shelf.
>>>       
>> Maybe stealing some ideas from pg_bulkload could somewhat help here?
>>   http://pgfoundry.org/docman/view.php/1000261/456/20060709_pg_bulkload.pdf
>>     
>
>   
>> IIRC it's mainly about how to optimize index updating while loading data, and 
>> I've heard complaints on the line "this external tool has to know too much 
>> about PostgreSQL internals to be trustworthy as non-core code"... so...
>>     
>
> Yeh, the batch index updates are a cool feature. Should be able to do
> that internally also.
>
> Not going to try the no-WAL route again though. If we can get it running
> efficiently and in parallel, then that will be OK. 
>
>