From: Alan Hodgson <ahodgson@simkin.ca>
To: pgsql-performance@postgresql.org
Subject: Re: Improve COPY performance for large data sets
Date: Wed, 10 Sep 2008 10:14:34 -0700
Message-ID: <200809101014.34249@hal.medialogik.com> (raw)
In-Reply-To: <48C7FA69.1080507@brightbuilders.com>
References: <48C7FA69.1080507@brightbuilders.com>
On Wednesday 10 September 2008, Ryan Hansen <ryan.hansen@brightbuilders.com>
wrote:
>Currently it's taking about 12 hours to complete on a 64 bit
> server with 3 GB memory allocated (shared_buffer), single SATA 320 GB
> drive. I don't seem to get any improvement running the same operation
> on a dual opteron dual-core, 16 GB server.
>
> I'm not asking for someone to solve my problem, just some direction in
> the best ways to tune for faster bulk loading, since this will be a
> fairly regular operation for our application (assuming it can work this
> way). I've toyed with the maintenance_work_mem and some of the other
> params, but it's still way slower than it seems like it should be.
> So any contributions are much appreciated.
Your drive subsystem, such as it is, is inappropriate for a database. Your
bottleneck is your drive.
Turning fsync off might help. You should also drop all indexes on the table
before the COPY and add them back after (which would eliminate a lot of
random I/O during the COPY).
--
Alan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: ahodgson@simkin.ca
Subject: Re: Improve COPY performance for large data sets
In-Reply-To: <200809101014.34249@hal.medialogik.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox