From: Claus Guttesen <kometen@gmail.com>
To: Kelly Burkhart <kelly@tradebotsystems.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: 8.x index insert performance
Date: Mon, 31 Oct 2005 20:02:26 +0100
Message-ID: <b41c75520510311102t31a6eal@mail.gmail.com> (raw)
In-Reply-To: <1130778725.7026.33.camel@krb06.tradebot.com>
References: <1130778725.7026.33.camel@krb06.tradebot.com>
> We are running some performance tests in which we are attempting to
> insert about 100,000,000 rows in a database at a sustained rate. About
> 50M rows in, our performance drops dramatically.
>
> This test is with data that we believe to be close to what we will
> encounter in production. However in tests with purely generated,
> sequential data, we did not notice this slowdown. I'm trying to figure
> out what patterns in the "real" data may be causing us problems.
>
> I have log,data and indexes on separate LUNs on an EMC SAN. Prior to
> slowdown, each partition is writing at a consistent rate. Index
> partition is reading at a much lower rate. At the time of slowdown,
> index partition read rate increases, all write rates decrease. CPU
> utilization drops.
I'm doing some test-inserts (albeit with much fewer records) into
8.0.4 (on FreeBSD 6.0 RC1) and the import-time decreased three-fold
when I increased the below mentioned values:
shared_buffers = 8192
commit_delay = 100000
commit_siblings = 1000
When I increased shared_buffers the kernel needed minor tweaking.
regards
Claus
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: kometen@gmail.com, kelly@tradebotsystems.com
Subject: Re: 8.x index insert performance
In-Reply-To: <b41c75520510311102t31a6eal@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox