From: Jignesh K. Shah <J.K.Shah@Sun.COM>
To: Greg Smith <gsmith@gregsmith.com>
Cc: Peter Schuller <peter.schuller@infidyne.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: shared_buffers in 8.3 w/ lots of RAM on dedicated PG machine
Date: Fri, 15 Feb 2008 14:34:33 -0500
Message-ID: <47B5E949.7070103@sun.com> (raw)
In-Reply-To: <Pine.GSO.4.64.0802150916380.28161@westnet.com>
References: <20080215123528.GA16532@hyperion.scode.org>
<20080215133734.GU12156@it.is.rice.edu>
<20080215135846.GA17327@hyperion.scode.org>
<Pine.GSO.4.64.0802150916380.28161@westnet.com>
Greg Smith wrote:
> On Fri, 15 Feb 2008, Peter Schuller wrote:
>
>> Or is it a matter of PostgreSQL doing non-direct I/O, such that
>> anything cached in shared_buffers will also be cached by the OS?
>
> PostgreSQL only uses direct I/O for writing to the WAL; everything
> else goes through the regular OS buffer cache unless you force it to
> do otherwise at the OS level (like some Solaris setups do with
> forcedirectio). This is one reason it still make not make sense to
> give an extremely high percentage of RAM to PostgreSQL even with
> improvements in managing it. Another is that shared_buffers memory
> has to be reconciled with disk at every checkpoint, where OS buffers
> do not. A third is that your OS may just be more efficient at
> buffering--it knows more about the underlying hardware, and the
> clock-sweep method used internally by PostgreSQL to simulate a LRU
> cache is not extremely sophisticated.
>
> However, don't feel limited by the general 25% rule; it's certainly
> worth exploring whether 50% or more works better for your workload.
> You'll have to benchmark that yourself though, and I'd suggest using
> pg_buffercache:
> http://www.postgresql.org/docs/8.3/static/pgbuffercache.html to get an
> idea just what the pages are being used for.
>
As per the test that I have done mostly with forcedirectio on Solaris, I
have seen gains with increasing the buffercache to about somewhere
between 10GB and thats when thing seem to take a turn...
So in my case I am generally comfortable for Postgres to use about
8-10GB beyond which I am cautious.
Also with tests with UFS buffered for table/index and forcedirectio it
seems to perform better with forcedirectio .. However if you do want to
exploit the extra RAM with UFS then you have to do some tunings for UFS
in Solaris.. Now with machines with 32GB becoming common this is
something worth pursuing depending on the storage if it can handle the
directio load or not.
Regards,
Jignesh
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: J.K.Shah@Sun.COM, gsmith@gregsmith.com, peter.schuller@infidyne.com
Subject: Re: shared_buffers in 8.3 w/ lots of RAM on dedicated PG machine
In-Reply-To: <47B5E949.7070103@sun.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox