Message-ID: <4C07F1FA.8010909@2ndquadrant.com>
Date: Thu, 03 Jun 2010 14:18:34 -0400
From: Greg Smith <greg@2ndquadrant.com>
User-Agent: Thunderbird 2.0.0.24 (X11/20100317)
MIME-Version: 1.0
To: Kevin Grittner <Kevin.Grittner@wicourts.gov>
CC: Craig James <craig_james@emolecules.com>,
	Matthew Wakeling <matthew@flymine.org>, pgsql-performance@postgresql.org
Subject: Re: Weird XFS WAL problem
References: <AANLkTikS_QvlkYfv5_hbsXFH4SsTh1nP_WoTwRqpUUcz@mail.gmail.com>
	<AANLkTikegoyBvJXhqZGLZyTwRVyBs4qCuCQ7d1Vntk4w@mail.gmail.com>
	<AANLkTikjkEvj2hG-D1CMu49PQXXbxHVzpXm9JbLWzMQo@mail.gmail.com>
	<AANLkTimtdk11hrWd-OpvXGLrh-4pukGd9eHEUemEM3OR@mail.gmail.com>
	<AANLkTinAhrwwwQKh6NrX-nnvcpxT1EFFlZrUxhaDB87C@mail.gmail.com>
	<AANLkTinuOUTbuke9FTMtwSObcZBio0uOS1ysWrqFyK3t@mail.gmail.com>
	<4C06E994.2080905@emolecules.com>
	<4C06EC01.3060506@catalyst.net.nz>
	<4C07E103.9020909@emolecules.com>
	<alpine.DEB.2.00.1006031811340.4083@aragorn.flymine.org>
	<4C07A0830200002500031E74@gw.wicourts.gov>
In-Reply-To: <4C07A0830200002500031E74@gw.wicourts.gov>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Kevin Grittner wrote:
> I've seen this, too (with xfs).  Our RAID controller, in spite of
> having BBU cache configured for writeback, waits for actual
> persistence on disk for write barriers (unlike for fsync).  This
> does strike me as surprising to the point of bordering on qualifying
> as a bug.
Completely intentional, and documented at 
http://xfs.org/index.php/XFS_FAQ#Q._Should_barriers_be_enabled_with_storage_which_has_a_persistent_write_cache.3F

The issue is that XFS will actually send the full "flush your cache" 
call to the controller, rather than just the usual fsync call, and that 
eliminates the benefit of having a write cache there in the first 
place.  Good controllers respect that and flush their whole write cache 
out.  And ext4 has adopted the same mechanism.  This is very much a good 
thing from the perspective of database reliability for people with 
regular hard drives who don't have a useful write cache on their cheap 
hard drives.  It allows them to keep the disk's write cache on for other 
things, while still getting the proper cache flushes when the database 
commits demand them.  It does mean that everyone with a non-volatile 
battery backed cache, via RAID card typically, needs to turn barriers 
off manually.

I've already warned on this list that PostgreSQL commit performance on 
ext4 is going to appear really terrible to many people.  If you 
benchmark and don't recognize ext3 wasn't operating in a reliable mode 
before, the performance drop now that ext4 is doing the right thing with 
barriers looks impossibly bad.

-- 
Greg Smith  2ndQuadrant US  Baltimore, MD
PostgreSQL Training, Services and Support
greg@2ndQuadrant.com   www.2ndQuadrant.us