Message-ID: <4E011871.1040603@2ndQuadrant.com>
Date: Tue, 21 Jun 2011 18:17:21 -0400
From: Greg Smith <greg@2ndQuadrant.com>
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.1.16) Gecko/20110505 Icedove/3.0.11
MIME-Version: 1.0
To: Merlin Moncure <mmoncure@gmail.com>
CC: Scott Marlowe <scott.marlowe@gmail.com>,
	Yeb Havinga <yebhavinga@gmail.com>,
	Florian Weimer <fweimer@bfk.de>, pgsql-performance@postgresql.org
Subject: Re: Contemplating SSD Hardware RAID
References: <4E0015F2.9060504@drivefaster.net>
	<4E003B44.9090005@2ndQuadrant.com>	<4E004D96.7010308@gmail.com>
	<4E004E74.5020209@gmail.com>	<82d3i777ow.fsf@mid.bfk.de>
	<4E00B4B9.50006@2ndQuadrant.com>	<4E00FABB.6030302@gmail.com>
	<4E00FE4B.3030204@gmail.com>
	<BANLkTi=zrKHjv1aRcgA6q_MJPNSOuJ9w=Q@mail.gmail.com>
	<BANLkTimoJGY6smRZmizkz_w6CxFH8Y0XaQ@mail.gmail.com>
In-Reply-To: <BANLkTimoJGY6smRZmizkz_w6CxFH8Y0XaQ@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

On 06/21/2011 05:35 PM, Merlin Moncure wrote:
> On Tue, Jun 21, 2011 at 3:32 PM, Scott Marlowe<scott.marlowe@gmail.com>  wrote:
>    
>> On Tue, Jun 21, 2011 at 2:25 PM, Yeb Havinga<yebhavinga@gmail.com>  wrote:
>>
>>      
>>> It's too bad however that OCZ doesn't let the user
>>> choose which firmware to run (the tool always picks the newest), so after
>>> every upgrade it'll be a surprise what values are supported or if any of the
>>>        
>> That right there pretty much eliminates them from consideration for
>> enterprise applications.
>>      
> As much as I've been irritated with Intel for being intentionally
> oblique on the write caching issue -- I think they remain more or less
> the only game in town for enterprise use.

That's at the core of why I have been so consistently cranky about 
them.  The sort of customers I deal with who are willing to spend money 
on banks of SSD will buy Intel, and the "Enterprise" feature set seems 
completely enough that it doesn't set off any alarms to them.  The same 
is not true of OCZ, which unfortunately means I never even get them onto 
the vendor grid in the first place.  Everybody runs out to buy the Intel 
units instead, they get burned by the write cache issues, lose data, and 
sometimes they even blame PostgreSQL for it.

I have a customer who has around 50 X25-E drives, a little stack of them 
in six servers running two similar databases.  They each run about a 
terabyte, and refill about every four months (old data eventually ages 
out, replaced by new).  At the point I started working with them, they 
had lost the entire recent history twice--terabyte gone, 
whoosh!--because the power reliability is poor in their area.  And 
network connectivity is bad enough that they can't ship this volume of 
updates to elsewhere either.

It happened again last month, and for the first time the database was 
recoverable.  I converted one server to be a cold spare, just archive 
the WAL files.  And that's the only one that lived through the nasty 
power spike+outage that corrupted the active databases on both the 
master and the warm standby of each set.  All four of the servers where 
PostgreSQL was writing data and expected proper fsync guarantees, all 
gone from one power issue.  At the point I got involved, they were about 
to cancel this entire PostgreSQL experiment because they assumed the 
database had to be garbage that this kept happening; until I told them 
about this known issue they never considered the drives were the 
problem.  That's what I think of when people ask me about the Intel X25-E.

I've very happy with the little 3rd generation consumer grade SSD I 
bought from Intel though (320 series).  If they just do the same style 
of write cache and reliability rework to the enterprise line, but using 
better flash, I agree that the first really serious yet affordable 
product for the database market may finally come out of that.  We're 
just not there yet, and unfortunately for the person who started this 
round of discussion throwing hardware RAID at the problem doesn't make 
this go away either.

-- 
Greg Smith   2ndQuadrant US    greg@2ndQuadrant.com   Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support  www.2ndQuadrant.us
"PostgreSQL 9.0 High Performance": http://www.2ndQuadrant.com/books