X-Original-To: pgsql-performance-postgresql.org@postgresql.org Received: from localhost (wm.hub.org [200.46.204.128]) by postgresql.org (Postfix) with ESMTP id 3141E9F95E3 for ; Tue, 18 Jul 2006 15:57:05 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.128]) (amavisd-new, port 10024) with ESMTP id 57542-10 for ; Tue, 18 Jul 2006 18:56:55 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey- Received: from mail.mi8.com (d01gw04.mi8.com [63.240.6.44]) by postgresql.org (Postfix) with ESMTP id 460C19F94F3 for ; Tue, 18 Jul 2006 15:56:56 -0300 (ADT) Received: from 172.16.1.110 by mail.mi8.com with ESMTP (- Welcome to Mi8 Corporation www.Mi8.com (D4)); Tue, 18 Jul 2006 14:56:48 -0400 X-Server-Uuid: C8FB4D43-1108-484A-A898-3CBCC7906230 Received: from MI8NYCMAIL06.Mi8.com ([172.16.1.160]) by D01SMTP01.Mi8.com with Microsoft SMTPSVC(6.0.3790.1830); Tue, 18 Jul 2006 14:56:48 -0400 Received: from 67.103.45.218 ([67.103.45.218]) by MI8NYCMAIL06.Mi8.com ( [172.16.1.219]) via Exchange Front-End Server mi8owa.mi8.com ( [172.16.1.108]) with Microsoft Exchange Server HTTP-DAV ; Tue, 18 Jul 2006 18:56:47 +0000 User-Agent: Microsoft-Entourage/11.2.5.060620 Date: Tue, 18 Jul 2006 11:56:45 -0700 Subject: Re: RAID stripe size question From: "Luke Lonergan" To: "Mikael Carneholm" , "Ron Peacetree" , pgsql-performance@postgresql.org Message-ID: Thread-Topic: [PERFORM] RAID stripe size question Thread-Index: AcaqF5fOHF1DUDpsT6uhqF/w3/K+OgAVFNmAAAv/tqg= In-Reply-To: <7F10D26ECFA1FB458B89C5B4B0D72C2B4E4C31@sesrv12.wirelesscar.com> MIME-Version: 1.0 X-OriginalArrivalTime: 18 Jul 2006 18:56:48.0475 (UTC) FILETIME=[EC1F12B0:01C6AA9B] X-WSS-ID: 68A3F17A2YG5698144-01-01 Content-Type: multipart/alternative; boundary=B_3236068606_690732 X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200607/182 X-Sequence-Number: 20042 > This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. --B_3236068606_690732 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Mikael, On 7/18/06 6:34 AM, "Mikael Carneholm" wrote: > However, what's more important is the seeks/s - ~530/s on a 28 disk > array is quite lousy compared to the 1400/s on a 12 x 15Kdisk array I'm getting 2500 seeks/second on a 36 disk SATA software RAID (ZFS, Solaris 10) on a Sun X4500: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Single Stream =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D With a very recent update to the zfs module that improves I/O scheduling an= d prefetching, I get the following bonnie++ 1.03a results with a 36 drive RAID10, Solaris 10 U2 on an X4500 with 500GB Hitachi drives (zfs checksumming is off): Version 1.03 ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP thumperdw-i-1 32G 120453 99 467814 98 290391 58 109371 99 993344 94 1801 4 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 +++++ +++ +++++ +++ +++++ +++ 30850 99 +++++ +++ +++++ +++ =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Two Streams =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Bumping up the number of concurrent processes to 2, we get about 1.5x speed reads of RAID10 with a concurrent workload (you have to add the rates together):=20 Version 1.03 ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP thumperdw-i-1 32G 111441 95 212536 54 171798 51 106184 98 719472 88 1233 2 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 26085 90 +++++ +++ 5700 98 21448 97 +++++ +++ 4381 97 Version 1.03 ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP thumperdw-i-1 32G 116355 99 212509 54 171647 50 106112 98 715030 87 1274 3 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 26082 99 +++++ +++ 5588 98 21399 88 +++++ +++ 4272 97 So that=B9s 2500 seeks per second, 1440MB/s sequential block read, 212MB/s pe= r character sequential read. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D - Luke --B_3236068606_690732 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Re: [PERFORM] RAID stripe size question Mikae= l,

On 7/18/06 6:34 AM, "Mikael Carneholm" <Mikael.Carneholm@Wirel= essCar.com> wrote:

> However, what's more important is the seeks/s - = ~530/s on a 28 disk
> array is quite lousy compared to the 1400/s on a 12 x 15Kdisk array
I'm getting 2500 seeks/second on a 36 disk SATA software RAID (ZFS, Solaris= 10) on a Sun X4500:

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Single Stream =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

With a very recent update to the zfs module that improves I/O scheduling an= d prefetching, I get the following bonnie++ 1.03a results with a 36 drive RA= ID10, Solaris 10 U2 on an X4500 with 500GB Hitachi drives (zfs checksumming = is off):

Versi= on  1.03       ------Sequential Output---= ---    --Sequential Input-   --Random-
            &nb= sp;       -Per Chr-  --Block--  = ;-Rewrite-  -Per Chr-  --Block--  --Seeks--
Machine        Size K/sec  %CP K/se= c  %CP K/sec  %CP K/sec  %CP K/sec  %CP /sec %CP
thumperdw-i-1   32G 120453  99 467814  98 290391  = 58 109371  99 993344  94 1801   4
            &nb= sp;       ------Sequential Create------ -= -------Random Create--------
            &nb= sp;       -Create-- --Read--- -Delete-- -= Create-- --Read--- -Delete--
            &nb= sp; files  /sec %CP  /sec %CP  /sec %CP  /sec %CP &= nbsp;/sec %CP  /sec %CP
            &nb= sp;    16 +++++ +++ +++++ +++ +++++ +++ 30850  99 += ++++ +++ +++++ +++

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Two Streams =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

Bumping up the number of concurrent processes to 2, we get about 1.5x speed= reads of RAID10 with a concurrent workload (you have to add the rates toget= her):

Version  1.03     =   ------Sequential Output------   --Sequential Input- &n= bsp;   --Random-
            &nb= sp;       -Per Chr- --Block--  -Rewr= ite-  -Per Chr-  --Block--    --Seeks--
Machine        Size K/sec  %CP K/se= c  %CP K/sec  %CP K/sec  %CP K/sec  %CP  /sec %CP thumperdw-i-1   32G 111441  95 212536  54 171798  = 51 106184  98 719472  88  1233   2
            &nb= sp;       ------Sequential Create------ -= -------Random Create--------
            &nb= sp;       -Create-- --Read--- -Delete-- -= Create-- --Read--- -Delete--
            &nb= sp; files  /sec %CP  /sec %CP  /sec %CP  /sec %CP &= nbsp;/sec %CP  /sec %CP
            &nb= sp;    16 26085  90 +++++ +++  5700  98 2= 1448  97 +++++ +++  4381  97

Version  1.03       ------Sequential Out= put------   --Sequential Input-     --Random-<= BR>             &nb= sp;       -Per Chr-  --Block--  = ;-Rewrite-  -Per Chr-  --Block--   --Seeks--
Machine        Size K/sec  %CP K/se= c  %CP K/sec  %CP K/sec  %CP K/sec  %CP  /sec %CP thumperdw-i-1   32G 116355  99 212509  54 171647  = 50 106112  98 715030  87  1274   3
            &nb= sp;       ------Sequential Create------ -= -------Random Create--------
            &nb= sp;       -Create-- --Read--- -Delete-- -= Create-- --Read--- -Delete--
            &nb= sp; files  /sec %CP  /sec %CP  /sec %CP  /sec %CP &= nbsp;/sec %CP  /sec %CP
            &nb= sp;    16 26082  99 +++++ +++  5588  98 2= 1399  88 +++++ +++  4272  97

So that’s 2500 seeks pe= r second, 1440MB/s sequential block read, 212MB/s per character sequential r= ead.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

- Luke --B_3236068606_690732--