Date: Thu, 26 Mar 2009 13:39:21 -0400 (EDT)
From: Greg Smith <gsmith@gregsmith.com>
To: Mark Kirkwood <markir@paradise.net.nz>
cc: "pgsql-performance@postgresql.org" <pgsql-performance@postgresql.org>
Subject: Re: Raid 10 chunksize
In-Reply-To: <49CB066C.5040203@paradise.net.nz>
Message-ID: <alpine.GSO.2.01.0903261309220.7258@westnet.com>
References: <49C98433.5080709@paradise.net.nz>
	<alpine.GSO.2.01.0903250251490.13683@westnet.com>
	<49CB066C.5040203@paradise.net.nz>
User-Agent: Alpine 2.01 (GSO 1184 2008-12-16)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII

On Thu, 26 Mar 2009, Mark Kirkwood wrote:

>> Also, if you want to minimize total I/O, you might drop 
>> bgwriter_lru_maxpages to 0.  That feature presumes you have some spare I/O 
>> capacity you use to prioritize lower latency, and it sounds like you don't. 
>> You get the lowest total I/O per transaction with the background writer 
>> turned off.
>> 
>
> Right - but then a big very noticeable stall when you do have to checkpoint? 
> We want to avoid that I think, even at the cost of a little overall 
> throughput.

There's not really a big difference if you're running with a large value 
for checkpoing_segments.  That spreads the checkpoint I/O over a longer 
period of time.  The current background writer doesn't aim to reduce 
writes at checkpoint time, because that never really worked out like 
people expected it to anyway.

It's aimed instead to write out buffers that database backend processes 
are going to need fairly soon, so they are less likely to block because 
they have to write them out themselves.  That leads to an occasional bit 
of wasted I/O, where the buffer written out gets used or dirtied against 
before it can be assigned to a backend.  I've got a long paper expanding 
on the documentation here you might find useful: 
http://www.westnet.com/~gsmith/content/postgresql/chkp-bgw-83.htm

> Yeah - with 64K chunksize I'm seeing a result more congruent with yours 
> (866 or so for 24 clients)

That's good to hear.  If adjusting that helped so much, you might consider 
aligning the filesystem partitions to the chunk size too; the partition 
header usually screws that up on Linux.  See these two references for 
ideas:  http://www.vmware.com/resources/techresources/608 
http://spiralbound.net/2008/06/09/creating-linux-partitions-for-clariion

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD