From: Tom Lane <tgl@sss.pgh.pa.us>
To: Josh Berkus <josh@agliodbs.com>
Cc: Roman Fail <rfail@posportal.com>
Cc: sszabo@megazone23.bigpanda.com
Cc: pgsql-performance@postgresql.org
Subject: Re: 7.3.1 New install, large queries are slow
Date: Fri, 17 Jan 2003 12:33:11 -0500
Message-ID: <13109.1042824791@sss.pgh.pa.us> (raw)
In-Reply-To: <web-2317350@davinci.ethosmedia.com>
References: <web-2317350@davinci.ethosmedia.com>
"Josh Berkus" <josh@agliodbs.com> writes:
>>> shared_buffers = 131072
>>
>> Yipes! Try about a tenth that much. Or less.
> Why? He has 4GB RAM on the machine.
I think a gig of shared buffers is overkill no matter what.
One reason not to crank up shared_buffers "just because you can" is that
there are operations (such as CHECKPOINT) that have to scan through all
the buffers, linearly. I don't *think* any of these are in
performance-critical paths, but nonetheless you're wasting CPU. I trust
the kernel to manage a huge number of buffers efficiently more than I
trust Postgres.
There's another issue, which is somewhat platform-dependent; I'm not
sure if it applies to whatever OS Roman is using. But if you have a
machine where SysV shared memory is not locked into RAM, then a huge
shared buffer arena creates the probability that some of it will be
touched seldom enough that the kernel will decide to swap it out. When
that happens, you *really* pay through the nose --- a page that you
might have been able to get from kernel cache without incurring I/O will
now certainly cost you I/O to touch. It's even worse if the buffer
contained a dirty page at the time it was swapped out --- now that page
is going to require being read back in and written out again, a net cost
of three I/Os where there should have been one. Bottom line is that
shared_buffers should be kept small enough that the space all looks like
a hot spot to the kernel's memory allocation manager.
In short, I believe in keeping shared_buffers relatively small --- one
to ten thousand seems like the right ballpark --- and leaving the kernel
to allocate the rest of RAM as kernel disk cache.
I have been thinking recently about proposing that we change the factory
default shared_buffers to 1000, which if this line of reasoning is
correct would eliminate the need for average installations to tune it.
The reason the default is 64 is that on some older Unixen, the default
SHMMAX is only one meg --- but it's been a long time since our default
shared memory request was less than a meg anyway, because of bloat in
other components of shared memory. It's probably time to change the
default to something more reasonable from a performance standpoint, and
put some burden on users of older Unixen to either reduce the setting
or fix their SHMMAX parameter.
regards, tom lane
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-performance@postgresql.org
Cc: tgl@sss.pgh.pa.us, josh@agliodbs.com, rfail@posportal.com, sszabo@megazone23.bigpanda.com
Subject: Re: 7.3.1 New install, large queries are slow
In-Reply-To: <13109.1042824791@sss.pgh.pa.us>
References: <web-2317350@davinci.ethosmedia.com> <13109.1042824791@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox