Message-ID: <4C408F01.2080501@2ndquadrant.com>
Date: Fri, 16 Jul 2010 12:55:29 -0400
From: Greg Smith <greg@2ndquadrant.com>
User-Agent: Thunderbird 2.0.0.24 (X11/20100317)
MIME-Version: 1.0
To: Scott Carey <scott@richrelevance.com>
CC: Ben Chobot <bench@silentmedia.com>,
	Craig James <craig_james@emolecules.com>,
	"Timothy.Noonan@emc.com" <Timothy.Noonan@emc.com>,
	PostgreSQL - Performance <pgsql-performance@postgresql.org>
Subject: Re: performance on new linux box
References: <AANLkTinVrej5r_uF3As3mF8BW4CYgLGiw2-v2HORsqjM@mail.gmail.com>
	<4C360677.1090608@emolecules.com>
	<AANLkTiny431UE-qThjeJuGRst2gR0lkdn1UKKTSWfvV9@mail.gmail.com>
	<AANLkTin24rAdlEMpohzAVdBXuI37DPofZrpnKxiQLPgL@mail.gmail.com>
	<95971335.20100708211351@erwied.eu>
	<AANLkTinsne2dhyxzXBb4KMVM_7yixWaUfVtG-WBPxMIq@mail.gmail.com>
	<488953776.20100708213233@erwied.eu>
	<AANLkTimx5Ug2iAQLv3l5scscART_6fm-wCl10k1nWyti@mail.gmail.com>
	<4C35E4CA020000250003327E@gw.wicourts.gov>
	<AANLkTino2ep4VyJDA9PVV6FObxHnBlLX3O9z8q19PzfN@mail.gmail.com>
	<4C363CAE.7010401@emolecules.com>
	<B9A709F368FAAF4DB4B33870F72A141D0106C0E8@CORPUSMX30A.corp.emc.com>
	<4C36462E.1080405@emolecules.com> <822
	F9CEE-570F-43F7-980F-A721EA7952D1@richrelevance.com>
	<A8F65D92-00A9-433F-83C0-04DEB55D57DF@silentmedia.com>
	<0BC62364-BD19-41E0-A483-677474CD5A5D@richrelevance.com>
	<6A7F9A16-F50B-43AE-B349-174726564292@silentmedia.com>
	<EAA36D96-AD81-4461-88B4-27F8CBACA636@richrelevance.com>
In-Reply-To: <EAA36D96-AD81-4461-88B4-27F8CBACA636@richrelevance.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Scott Carey wrote:
> As long as fsync() works _properly_ which is true for any file system + disk combination with a damn (not HFS+ on OSX, not FAT, not a few other things), then it will tell the drive to flush its cache _before_ fsync() returns.  There is NO REASON for a raid card to turn off a drive cache unless it does not trust the drive cache.  In write-through mode, it should not return to the OS with a fsync, direct write, or other "the OS thinks this data is persisted now" call until it has flushed the disk cache.  That does not mean it has to turn off the disk cache.
>   

Assuming that the operating system will pass through fsync calls to 
flush data all the way to drive level in all situations is an extremely 
dangerous assumption.  Most RAID controllers don't know how to force 
things out of the individual drive caches; that's why they turn off 
write caching on them.  Few filesystems get the details right to handle 
individual drive cache flushing correctly.  On Linux, XFS and ext4 are 
the only two with any expectation that will happen, and of those two 
ext4 is still pretty new and therefore should still be presumed to be buggy.

Please don't advise people about what is safe based on theoretical 
grounds here, in practice there are way too many bugs in the 
implementation of things like drive barriers to trust them most of the 
time.  There is no substitute for a pull the plug test using something 
that looks for bad cache flushes, i.e. diskchecker.pl:  
http://brad.livejournal.com/2116715.html  If you do that you'll discover 
you must turn off the individual drive caches when using a 
battery-backed RAID controller, and you can't ever trust barriers on 
ext3 because of bugs that were only fixed in ext4.

-- 
Greg Smith  2ndQuadrant US  Baltimore, MD
PostgreSQL Training, Services and Support
greg@2ndQuadrant.com   www.2ndQuadrant.us