From: Luke Lonergan <llonergan@greenplum.com>
To: Greg Smith <gsmith@gregsmith.com>
To: pgsql-performance@postgresql.org
Subject: Re: Benchmark Data requested --- pgloader CE design ideas
Date: Wed, 06 Feb 2008 08:17:42 -0800
Message-ID: <C3CF1DA6.53067%llonergan@greenplum.com> (raw)
In-Reply-To: <Pine.GSO.4.64.0802061041230.15780@westnet.com>
Hi Greg,
On 2/6/08 7:56 AM, "Greg Smith" <gsmith@gregsmith.com> wrote:
> If I'm loading a TB file, odds are good I can split that into 4 or more
> vertical pieces (say rows 1-25%, 25-50%, 50-75%, 75-100%), start 4 loaders
> at once, and get way more than 1 disk worth of throughput reading. You
> have to play with the exact number because if you push the split too far
> you introduce seek slowdown instead of improvements, but that's the basic
> design I'd like to see one day. It's not parallel loading that's useful
> for the cases I'm thinking about until something like this comes around.
Just load 4 relfiles. You have to be able to handle partial relfiles, which
changes the storage mgmt a bit, but the benefits are easier to achieve.
- Luke
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: llonergan@greenplum.com, gsmith@gregsmith.com
Subject: Re: Benchmark Data requested --- pgloader CE design ideas
In-Reply-To: <C3CF1DA6.53067%llonergan@greenplum.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox