From: Richard Huxton <dev@archonet.com>
To: Simon Riggs <simon@2ndquadrant.com>
Cc: Dimitri Fontaine <dfontaine@hi-media.com>
Cc: pgsql-performance@postgresql.org, "Jignesh K. Shah" <J.K.Shah@sun.com>
Cc: Greg Smith <gsmith@gregsmith.com>
Subject: Re: Benchmark Data requested
Date: Tue, 05 Feb 2008 15:05:18 +0000
Message-ID: <47A87B2E.7050205@archonet.com> (raw)
In-Reply-To: <1202223268.4252.707.camel@ebony.site>
References: <1202150236.4252.521.camel@ebony.site>
<Pine.GSO.4.64.0802041500460.799@westnet.com>
<47A797E5.4000507@sun.com>
<200802051506.51581.dfontaine@hi-media.com>
<1202221496.4252.680.camel@ebony.site>
<47A87618.1090203@archonet.com>
<1202223268.4252.707.camel@ebony.site>
Simon Riggs wrote:
> On Tue, 2008-02-05 at 14:43 +0000, Richard Huxton wrote:
>> Simon Riggs wrote:
>>> On Tue, 2008-02-05 at 15:06 +0100, Dimitri Fontaine wrote:
>>>> Le lundi 04 février 2008, Jignesh K. Shah a écrit :
>>>>> Multiple table loads ( 1 per table) spawned via script is bit better
>>>>> but hits wal problems.
>>>> pgloader will too hit the WAL problem, but it still may have its benefits, or
>>>> at least we will soon (you can already if you take it from CVS) be able to
>>>> measure if the parallel loading at the client side is a good idea perf. wise.
>>> Should be able to reduce lock contention, but not overall WAL volume.
>> In the case of a bulk upload to an empty table (or partition?) could you
>> not optimise the WAL away? That is, shouldn't the WAL basically be a
>> simple transformation of the on-disk blocks? You'd have to explicitly
>> sync the file(s) for the table/indexes of course, and you'd need some
>> work-around for WAL shipping, but it might be worth it for you chaps
>> with large imports.
>
> Only by locking the table, which serializes access, which then slows you
> down or at least restricts other options. Plus if you use pg_loader then
> you'll find only the first few rows optimized and all the rest not.
Hmm - the table-locking requirement is true enough, but why would
pg_loader cause problems after the first few rows?
--
Richard Huxton
Archonet Ltd
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: dev@archonet.com, simon@2ndquadrant.com, dfontaine@hi-media.com, J.K.Shah@sun.com, gsmith@gregsmith.com
Subject: Re: Benchmark Data requested
In-Reply-To: <47A87B2E.7050205@archonet.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox