pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Tom Lane <tgl@sss.pgh.pa.us>
To: josh@agliodbs.com
Cc: pgsql-performance@postgresql.org, testperf-general@pgfoundry.org, Jan Wieck <JanWieck@Yahoo.com>
Subject: Re: First set of OSDL Shared Mem scalability results, some wierdness ...
Date: Fri, 08 Oct 2004 18:21:28 -0400
Message-ID: <21365.1097274088@sss.pgh.pa.us> (raw)
In-Reply-To: <200410081443.16109.josh@agliodbs.com>
References: <200410081443.16109.josh@agliodbs.com>

Josh Berkus <josh@agliodbs.com> writes:
> Here's a top-level summary:

> shared_buffers	% RAM		NOTPM20*
> 1000			0.2%		1287
> 23000			5%		1507
> 46000			10%		1481
> 69000			15%		1382
> 92000			20%		1375
> 115000		25%		1380
> 138000		30%		1344

> As you can see, the "sweet spot" appears to be between 5% and 10% of RAM, 
> which is if anything *lower* than recommendations for 7.4!   

This doesn't actually surprise me a lot.  There are a number of aspects
of Postgres that will get slower the more buffers there are.

One thing that I hadn't focused on till just now, which is a new
overhead in 8.0, is that StrategyDirtyBufferList() scans the *entire*
buffer list *every time it's called*, which is to say once per bgwriter
loop.  And to add insult to injury, it's doing that with the BufMgrLock
held (not that it's got any choice).

We could alleviate this by changing the API between this function and
BufferSync, such that StrategyDirtyBufferList can stop as soon as it's
found all the buffers that are going to be written in this bgwriter
cycle ... but AFAICS that means abandoning the "bgwriter_percent" knob
since you'd never really know how many dirty pages there were
altogether.

BTW, what is the actual size of the test database (disk footprint wise)
and how much of that do you think is heavily accessed during the run?
It's possible that the test conditions are such that adjusting
shared_buffers isn't going to mean anything anyway.

			regards, tom lane



view thread (41+ messages)  latest in thread

Message-ID: <21365.1097274088@sss.pgh.pa.us>
Permalink:  ../21365.1097274088@sss.pgh.pa.us/
Also on:    postgresql.org/message-id/21365.1097274088@sss.pgh.pa.us

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-performance@postgresql.org
  Cc: tgl@sss.pgh.pa.us, josh@agliodbs.com, JanWieck@Yahoo.com
  Subject: Re: First set of OSDL Shared Mem scalability results, some wierdness ...
  In-Reply-To: <21365.1097274088@sss.pgh.pa.us>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox