Message-ID: <4C40442A.4030806@postnewspapers.com.au>
Date: Fri, 16 Jul 2010 19:36:10 +0800
From: Craig Ringer <craig@postnewspapers.com.au>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.9.1.9) Gecko/20100423 Lightning/1.0b1 Thunderbird/3.0.4
MIME-Version: 1.0
To: Scott Marlowe <scott.marlowe@gmail.com>
CC: "pgsql-performance@postgresql.org" <pgsql-performance@postgresql.org>
Subject: Re: performance on new linux box
References: <AANLkTinVrej5r_uF3As3mF8BW4CYgLGiw2-v2HORsqjM@mail.gmail.com>
	<4C35A23B0200002500033220@gw.wicourts.gov>
	<AANLkTilCjMHlvYuHCu5Hf7nd4q5C50Cu5NhTS55F3St9@mail.gmail.com>
	<4C360677.1090608@emolecules.com>
	<AANLkTiny431UE-qThjeJuGRst2gR0lkdn1UKKTSWfvV9@mail.gmail.com>
	<AANLkTin24rAdlEMpohzAVdBXuI37DPofZrpnKxiQLPgL@mail.gmail.com>
	<95971335.20100708211351@erwied.eu>
	<AANLkTinsne2dhyxzXBb4KMVM_7yixWaUfVtG-WBPxMIq@mail.gmail.com>
	<488953776.20100708213233@erwied.eu>
	<AANLkTimx5Ug2iAQLv3l5scscART_6fm-wCl10k1nWyti@mail.gmail.com>
	<4C35E4CA020000250003327E@gw.wicourts.gov>
	<AANLkTino2ep4VyJDA9PVV6FObxHnBlLX3O9z8q19PzfN@mail.gmail.com>
	<4C363CAE.7010401@emolecules.com>
	<B9A709F368FAAF4DB4B33870F72A141D0106C0E8@CORPUSMX30A.corp.emc.com>
	<4C36462E.1080405@emolecules.com>
	<A8F65D92-00A9-433F-83C0-04DEB55D57DF@silentmedia.com>
	<0BC62364-BD19-41E0-A483-677474CD5A5D@richrelevance.com>
	<AANLkTimf9SQ_zZX5E-au35ZUEEwow6TmSc9x3NQx90fn@mail.gmail.com>
In-Reply-To: <AANLkTimf9SQ_zZX5E-au35ZUEEwow6TmSc9x3NQx90fn@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

On 16/07/10 09:22, Scott Marlowe wrote:
> On Thu, Jul 15, 2010 at 10:30 AM, Scott Carey <scott@richrelevance.com> wrote:
>>
>> On Jul 14, 2010, at 7:50 PM, Ben Chobot wrote:
>>
>>> On Jul 14, 2010, at 6:57 PM, Scott Carey wrote:
>>>
>>>> But none of this explains why a 4-disk raid 10 is slower than a 1 disk system.  If there is no write-back caching on the RAID, it should still be similar to the one disk setup.
>>>
>>> Many raid controllers are smart enough to always turn off write caching on the drives, and also disable the feature on their own buffer without a BBU. Add a BBU, and the cache on the controller starts getting used, but *not* the cache on the drives.
>>
>> This does not make sense.
> 
> Basically, you can have cheap, fast and dangerous (drive with write
> cache enabled, which responds positively to fsync even when it hasn't
> actually fsynced the data.  You can have cheap, slow and safe with a
> drive that has a cache but since it'll be fsyncing it all the the time
> the write cache won't actually get used, or fast, expensive, and safe,
> which is what a BBU RAID card gets by saying the data is fsynced when
> it's actually just in cache, but a safe cache that won't get lost on
> power down.

Speaking of BBUs... do you ever find yourself wishing you could use
software RAID with battery backup?

I tend to use software RAID quite heavily on non-database servers, as
it's cheap, fast, portable from machine to machine, and (in the case of
Linux 'md' raid) reliable. Alas, I can't really use it for DB servers
due to the need for write-back caching.

There's no technical reason I know of why sw raid couldn't write-cache
to some non-volatile memory on the host. A dedicated  a battery-backed
pair of DIMMS on a PCI-E card mapped into memory would be ideal. Failing
that, a PCI-E card with onboard RAM+BATT or fast flash that presents an
AHCI interface so it can be used as a virtual HDD would do pretty well.
Even one of those SATA "RAM Drive" units would do the job, though
forcing everything though the SATA2 bus would be a performance downside.

The only issue I see with sw raid write caching is that it probably
couldn't be done safely on the root file system. The OS would have to
come up, init software raid, and find the caches before it'd be safe to
read or write volumes with s/w raid write caching enabled. It's not the
sort of thing that'd be practical to implement in GRUB's raid support.

--
Craig Ringer