Received: from localhost (unknown [200.46.204.184]) by postgresql.org (Postfix) with ESMTP id 4DF0D2E0190 for ; Tue, 29 Jan 2008 11:52:35 -0400 (AST) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.184]) (amavisd-maia, port 10024) with ESMTP id 14743-06 for ; Tue, 29 Jan 2008 11:52:14 -0400 (AST) Received: from oxford.xeocode.com (unknown [62.232.55.118]) by postgresql.org (Postfix) with ESMTP id 71FBE2E0158 for ; Tue, 29 Jan 2008 11:52:23 -0400 (AST) Received: from localhost ([127.0.0.1] helo=oxford.xeocode.com) by oxford.xeocode.com with esmtp (Exim 4.69) (envelope-from ) id 1JJsl2-00077j-JK; Tue, 29 Jan 2008 15:52:20 +0000 To: "Matthew" Cc: "Mark Mielke" , Subject: Re: RAID arrays and performance In-Reply-To: (matthew@flymine.org's message of "Tue\, 29 Jan 2008 15\:09\:03 +0000 \(GMT\)") User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.1 (gnu/linux) X-Draft-From: ("nnmaildir+EDB-IMAP:INBOX" 10350) References: <87lk8aiqf3.fsf@oxford.xeocode.com> <475559F4.7060104@mark.mielke.cc> <4755648C.90905@mark.mielke.cc> <87bq96ii3b.fsf@oxford.xeocode.com> <4755810E.4000707@mark.mielke.cc> <87zlwq6yz9.fsf@oxford.xeocode.com> <87bq74vfys.fsf@oxford.xeocode.com> From: Gregory Stark Organization: EnterpriseDB Date: Tue, 29 Jan 2008 15:52:20 +0000 Message-ID: <877ihsvdcb.fsf@oxford.xeocode.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200801/291 X-Sequence-Number: 28530 "Matthew" writes: > On Tue, 29 Jan 2008, Gregory Stark wrote: >>> This was with 8192 random requests of size 8192 bytes from an 80GB test file. >>> Unsorted requests ranged from 1.8 MB/s with no prefetching to 28MB/s with lots >>> of prefetching. Sorted requests went from 2.4MB/s to 38MB/s. That's almost >>> exactly 16x improvement for both, and this is top of the line hardware. >> >> Neat. The curves look very similar to mine. I also like that with your >> hardware the benefit maxes out at pretty much exactly where I had >> mathematically predicted they would ((stripe size)^2 / 2). > > Why would that be the case? Does that mean that we can select a stripe size of > 100GB and get massive performance improvements? Doesn't seem logical to me. To > me, it maxes out at 16x speed because there are 16 discs. Sorry, I meant "number of drives in the array" not number of bytes. So with 16 drives you would need approximately 128 random pending i/o operations to expect all drives to be busy at all times. I got this from a back-of-the-envelope calculation which now that I'm trying to reproduce it seems to be wrong. Previously I thought it was n(n+1)/2 or about n^2/2. So at 16 I would have expected about 128 pending i/o requests before all the drives could be expected to be busy. Now that I'm working it out more carefully I'm getting that the expected number of pending i/o requests before all drives are busy is n + n/2 + n/3 + ... + n/n which is actually n * H(n) which is approximated closely by n * log(n). That would predict that 16 drives would actually max out at 44.4 pending i/o requests. It would predict that my three-drive array would max out well below that at 7.7 pending i/o requests. Empirically neither result seems to match reality. Other factors must be dominating. > Amusingly, there appears to be a spam filter preventing my message (with its > image) getting through to the performance mailing list. This has been plaguing us for a while. When we figure out who's misconfigured system is doing it I expect they'll be banned from the internet for life! -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's Slony Replication support!