From: Florian Weimer <fw@deneb.enyo.de>
To: Vivek Khera <khera@kcilink.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: Bulk INSERT performance in 7.4.1
Date: Thu, 4 Mar 2004 11:06:08 +0100
Message-ID: <20040304100608.GA5521@deneb.enyo.de> (raw)
In-Reply-To: <x7r7w9smii.fsf@yertle.int.kciLink.com>
References: <20040303092528.GA6647@deneb.enyo.de>
<20040303141032.GA12276@deneb.enyo.de>
<x7r7w9smii.fsf@yertle.int.kciLink.com>
Vivek Khera wrote:
> If you've got the time, could you try also doing the full bulk insert
> test with the checkpoint log files on another physical disk? See if
> that's any faster.
We have been doing that for a few weeks, but the performance
improvements are less than what we expected. There is hardly any disk
activity on the log RAID, even during checkpointing.
After I activated the tuned configuration, we are again mostly CPU-bound
(it seems that updating all four indices is quite expensive). The
bulk INSERT process runs single-threaded right now, and if we switched
to multiple processes for that, we could reach some 1,500 INSERTs per
second, I believe. This is more than sufficient for us; our real-time
data collector is tuned to emit about 150 records per second, on the
average. (There is an on-disk queue to compensate temporary problems,
such as spikes in the data rate and database updates gone awry.)
--
Current mail filters: many dial-up/DSL/cable modem hosts, and the
following domains: atlas.cz, bigpond.com, freenet.de, hotmail.com,
libero.it, netscape.net, postino.it, tiscali.co.uk, tiscali.cz,
tiscali.it, voila.fr, wanadoo.fr, yahoo.com.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: fw@deneb.enyo.de, khera@kcilink.com
Subject: Re: Bulk INSERT performance in 7.4.1
In-Reply-To: <20040304100608.GA5521@deneb.enyo.de>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox