DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=dk20050327;
 d=earthlink.net;
	b=Pxe6ASL17YgAD/wTSb2Bh7o8Mi/125dDnsGdOvEOybFLmAzlD3459vnZ/kpmaUD9;
	h=Message-ID:Date:From:Reply-To:To:Subject:Mime-Version:Content-Type:Content-Transfer-Encoding:X-Mailer:X-ELNK-Trace:X-Originating-IP;
Message-ID: 
 <13498289.1153192075696.JavaMail.root@elwamui-huard.atl.sa.earthlink.net>
Date: Mon, 17 Jul 2006 23:07:55 -0400 (EDT)
From: Ron  Peacetree <rjpeace@earthlink.net>
Reply-To: Ron  Peacetree <rjpeace@earthlink.net>
To: Mikael Carneholm <Mikael.Carneholm@WirelessCar.com>,
	pgsql-performance@postgresql.org
Subject: Re: RAID stripe size question
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

-----Original Message-----
>From: Mikael Carneholm <Mikael.Carneholm@WirelessCar.com>
>Sent: Jul 17, 2006 5:16 PM
>To: Ron  Peacetree <rjpeace@earthlink.net>, pgsql-performance@postgresql.o=
rg
>Subject: RE: [PERFORM] RAID stripe size question
>
>>15Krpm HDs will have average access times of 5-6ms.  10Krpm ones of 7-8ms=
.
>
>Average seek time for that disk is listed as 4.9ms, maybe sounds a bit opt=
imistic?
>
Ah, the games vendors play.  "average seek time" for a 10Krpm HD may very w=
ell be 4.9ms.  However, what matters to you the user is "average =3Daccess=
=3D time".  The 1st is how long it takes to position the heads to the corre=
ct track.  The 2nd is how long it takes to actually find and get data from =
a specified HD sector.

>> 28HDs as above setup as 2 RAID 10's =3D> ~75MBps*5=3D ~375MB/s,  ~75*9=
=3D ~675MB/s.
>
>I guess it's still limited by the 2Gbit FC (192Mb/s), right?
>
No.  A decent HBA has multiple IO channels on it.  So for instance Areca's =
ARC-6080 (8/12/16-port 4Gbps Fibre-to-SATA ll Controller) has 2 4Gbps FCs i=
n it (...and can support up to 4GB of BB cache!).  Nominally, this card can=
 push 8Gbps=3D 800MBps.  ~600-700MBps is the RW number.

Assuming ~75MBps ASTR per HD, that's ~ enough bandwidth for a 16 HD RAID 10=
 set per ARC-6080. =20

>>Very, very few RAID controllers can do >=3D 1GBps One thing that help gre=
atly with=20
>>bursty IO patterns is to up your battery backed RAID cache as high as you=
 possibly
>>can.  Even multiple GBs of BBC can be worth it. =20
>>Another reason to have multiple controllers ;-)
>
>I use 90% of the raid cache for writes, don't think I could go higher than=
 that.=20
>Too bad the emulex only has 256Mb though :/
>
If your RAID cache hit rates are in the 90+% range, you probably would find=
 it profitable to make it greater.  I've definitely seen access patterns th=
at benefitted from increased RAID cache for any size I could actually insta=
ll.  For those access patterns, no amount of RAID cache commercially availa=
ble was enough to find the "flattening" point of the cache percentage curve=
.  256MB of BB RAID cache per HBA is just not that much for many IO pattern=
s.


>The controller is a FC2143 (http://h71016.www7.hp.com/dstore/MiddleFrame.a=
sp?page=3Dconfig&ProductLineId=3D450&FamilyId=3D1449&BaseId=3D17621&oi=3DE9=
CED&BEID=3D19701&SBLID=3D), which uses PCI-E. Don't know how it compares to=
 other controllers, haven't had the time to search for / read any reviews y=
et.
>
This is a relatively low end HBA with 1 4Gb FC on it.  Max sustained IO on =
it is going to be ~320MBps.  Or ~ enough for an 8 HD RAID 10 set made of 75=
MBps ASTR HD's.

28 such HDs are =3Ddefinitely=3D IO choked on this HBA. =20

The arithmatic suggests you need a better HBA or more HBAs or both.


>>WAL's are basically appends that are written in bursts of your chosen log=
 chunk size and that are almost never read afterwards.  Big DB pages and bi=
g RAID stripes makes sense for WALs.
>
>According to http://www.pcguide.com/ref/hdd/perf/raid/concepts/perfStripe-=
c.html, it seems to be the other way around? ("As stripe size is decreased,=
 files are broken into smaller and smaller pieces. This increases the numbe=
r of drives that an average file will use to hold all the blocks containing=
 the data of that file, theoretically increasing transfer performance, but =
decreasing positioning performance.")
>
>I guess I'll have to find out which theory that holds by good ol=EF=BF=BD =
trial and error... :)
>
IME, stripe sizes of 64, 128, or 256 are the most common found to be optima=
l for most access patterns + SW + FS + OS + HW.