From: Michael Artz <mlartz@gmail.com>
To: Luke Lonergan <llonergan@greenplum.com>
Cc: Worky Workerson <worky.workerson@gmail.com>
Cc: Merlin Moncure <mmoncure@gmail.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: Best COPY Performance
Date: Sat, 28 Oct 2006 08:03:10 -0400
Message-ID: <e9c163070610280503t4788ab5ah857ea249b5027e2@mail.gmail.com> (raw)
In-Reply-To: <C1682586.55BF%llonergan@greenplum.com>
References: <ce4072df0610272047w4fdf36dfj130b0e9171374d3a@mail.gmail.com>
<C1682586.55BF%llonergan@greenplum.com>
> > Are you saying that I should be able to issue multiple COPY commands
> > because my I/O wait is low? I was under the impression that I am I/O
> > bound, so multiple simeoultaneous loads would have a detrimental
> > effect ...
>
> The reason I asked how many CPUs was to make sense of the 12% usr CPU time
> in the above. That means you are CPU bound and are fully using one CPU. So
> you aren't being limited by the I/O in this case, it's the CPU.
>
> I agree with Merlin that you can speed things up by breaking the file up.
> Alternately you can use the OSS Bizgres java loader, which lets you specify
> the number of I/O threads with the "-n" option on a single file.
Thanks, I'll try that on Monday.
> The other thing to wonder about though is why you are so CPU bound at 5
> MB/s. What version of Postgres is this?
I was wondering about that as well, and the only thing that I can
think of is that its the PK btree index creation on the IP4.
PG 8.1.3 x86_64. I installed it via a RH rpm for their "Web Services
Beta", or something like that. I know I'm a bit behind the times, but
getting stuff in (and out) of my isolated lab is a bit of a pain.
I'll compile up a 8.2 beta as well and see how that works out.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: mlartz@gmail.com, llonergan@greenplum.com, worky.workerson@gmail.com, mmoncure@gmail.com
Subject: Re: Best COPY Performance
In-Reply-To: <e9c163070610280503t4788ab5ah857ea249b5027e2@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox