From: Ian Westmacott <ianw@intellivid.com>
To: Daniel J. Luke <dluke@geeklair.net>
Cc: pgsql-performance@postgresql.org
Subject: Re: Getting even more insert performance (250m+rows/day)
Date: Wed, 24 May 2006 23:20:24 -0400
Message-ID: <000301c67faa$296831a0$1fd61e42@hsd1.ma.comcast.net> (raw)
In-Reply-To: <20060524200349.GA10753@uio.no>
We were able to achieve 2B (small) rows per day sustained with
very little latency. It is beefy hardware, but things that did
help include WAL on its own I/O channel, XFS, binary copy,
and tuning bgwriter and checkpoint settings for the application
and hardware. Things that didn't help much were shared_buffers
and wal_buffers. But our application is single-writer, and a
small number of readers.
Although there is tons of great advice in this and other forums,
I think you just have to do a lot of experimentation with careful
measurement to find what's right for your application/environment.
i.e., YMMV.
-----Original Message-----
From: pgsql-performance-owner@postgresql.org
[mailto:pgsql-performance-owner@postgresql.org]On Behalf Of Steinar H.
Gunderson
Sent: Wednesday, May 24, 2006 4:04 PM
To: Daniel J. Luke
Cc: pgsql-performance@postgresql.org
Subject: Re: [PERFORM] Getting even more insert performance
(250m+rows/day)
On Wed, May 24, 2006 at 03:45:17PM -0400, Daniel J. Luke wrote:
> Things I've already done that have made a big difference:
> - modified postgresql.conf shared_buffers value
> - converted to COPY from individual insert statements
> - changed BLCKSZ to 32768
Have you tried fiddling with the checkpointing settings? Check your logs --
if you get a warning about checkpoints being too close together, that should
give you quite some boost.
Apart from that, you should have quite a bit to go on -- somebody on this
list reported 2 billion rows/day earlier, but it might have been on beefier
hardware, of course. :-)
/* Steinar */
--
Homepage: http://www.sesse.net/
---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: ianw@intellivid.com, dluke@geeklair.net
Subject: Re: Getting even more insert performance (250m+rows/day)
In-Reply-To: <000301c67faa$296831a0$1fd61e42@hsd1.ma.comcast.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox