From: Josh Berkus <josh@agliodbs.com>
To: Marinos J. Yannikos <mjy@geizhals.at>
To: pgsql-performance@postgresql.org
Subject: Re: optimization ideas for frequent, large(ish) updates in frequently accessed DB?
Date: Thu, 12 Feb 2004 22:28:41 -0800
Message-ID: <200402122228.41680.josh@agliodbs.com> (raw)
In-Reply-To: <402C213A.4030105@geizhals.at>
References: <402C213A.4030105@geizhals.at>
Marinos,
> shared_buffers=100000
> (I tried many values, this seems to work well for us - 12GB RAM)
> wal_buffers=500
> sort_mem=800000
> checkpoint_segments=16
> effective_cache_size=1000000
> etc.
800MB for sort mem? Are you sure you typed that correctly? You must be
counting on not having a lot of concurrent queries. It sure will speed up
index updating, though!
I think you might do well to experiment with using the checkpoint_delay and
checkpoint_sibilings settings in order to get more efficient batch processing
of updates while selects are going on. I would also suggest increasing
checkpoint segments as much as your disk space will allow; I know one
reporting database I run that does batch loads is using 128 (which is about a
gig of disk, I think).
What have you set max_fsm_relations and max_fsm_pages to? The latter should
be very high for you, like 10,000,000
For that matter, what *version* of PostgreSQL are you running?
Also, make sure that your tables get vaccuumed regularly.
> Any help/suggestions would be greatly appreciated... Even if it's
> something like "you need a faster db box, there's no other way" ;-)
Well, a battery-backed RAID controller with a fast cache would certainly help.
You'll also be glad to know that a *lot* of the improvements in the upcoming
PostgreSQL 7.5 are aimed at giving better peformance on large, high-activity
databases like yours.
--
-Josh Berkus
Aglio Database Solutions
San Francisco
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: josh@agliodbs.com, mjy@geizhals.at
Subject: Re: optimization ideas for frequent, large(ish) updates in frequently accessed DB?
In-Reply-To: <200402122228.41680.josh@agliodbs.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox