X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org
Received: from localhost (av.hub.org [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id 37F6352DEA
	for <pgsql-performance-postgresql.org@localhost.postgresql.org>;
	Sat, 20 Aug 2005 23:34:26 -0300 (ADT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 55941-03
	for <pgsql-performance-postgresql.org@localhost.postgresql.org>;
	Sun, 21 Aug 2005 02:34:23 +0000 (GMT)
Received: from sccmmhc91.asp.att.net (sccmmhc91.asp.att.net [204.127.203.211])
	by svr1.postgresql.org (Postfix) with ESMTP id E2EAD52DC1
	for <pgsql-performance@postgresql.org>;
	Sat, 20 Aug 2005 23:34:21 -0300 (ADT)
Received: from juju.arbash-meinel.com ([12.214.18.81])
	by sccmmhc91.asp.att.net (sccmmhc91) with ESMTP
	id <20050821023210m9100g5klfe>; Sun, 21 Aug 2005 02:34:26 +0000
Received: by juju.arbash-meinel.com (Postfix, from userid 505)
	id 437A855FE1; Sat, 20 Aug 2005 21:32:10 -0500 (CDT)
Received: from [192.168.1.11] (liliana.arbash-meinel.com [192.168.1.11])
	by juju.arbash-meinel.com (Postfix) with ESMTP id 744C855FAD;
	Sat, 20 Aug 2005 21:32:04 -0500 (CDT)
Message-ID: <4307E7A4.7090300@arbash-meinel.com>
Date: Sat, 20 Aug 2005 21:32:04 -0500
From: John A Meinel <john@arbash-meinel.com>
User-Agent: Mozilla Thunderbird 1.0.2 (Windows/20050317)
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: Ron <rjpeace@earthlink.net>
Cc: Jeremiah Jahn <jeremiah@cs.earlham.edu>,
	postgres performance <pgsql-performance@postgresql.org>
Subject: Re: extremly low memory usage
References: <1124331108.27881.96.camel@bluejay.goodinassociates.com>
	<EA4219B9-E2B0-4756-ABF2-D84F245FDFB5@torgo.978.org>
	<1124386761.27881.119.camel@bluejay.goodinassociates.com>
	<4304CB77.4070403@arbash-meinel.com>
	<1124470109.27881.152.camel@bluejay.goodinassociates.com>
	<43061472.2080201@arbash-meinel.com>
	<1124481386.27881.167.camel@bluejay.goodinassociates.com>
	<43064908.7010505@arbash-meinel.com>
	<1124564003.27881.229.camel@bluejay.goodinassociates.com>
	<6.2.3.4.0.20050820154422.06021150@pop.earthlink.net>
In-Reply-To: <6.2.3.4.0.20050820154422.06021150@pop.earthlink.net>
X-Enigmail-Version: 0.92.0.0
Content-Type: multipart/signed; micalg=pgp-sha1;
	protocol="application/pgp-signature";
	boundary="------------enig6CF0AF8208E4B7FC20509C12"
X-Virus-Scanned: by amavisd-new at hub.org
X-Spam-Status: No, hits=0.042 required=5 tests=[AWL=-0.008,
	FORGED_RCVD_HELO=0.05]
X-Spam-Level: 
X-Archive-Number: 200508/304
X-Sequence-Number: 14051

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig6CF0AF8208E4B7FC20509C12
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Ron wrote:
> At 02:53 PM 8/20/2005, Jeremiah Jahn wrote:
> 
>> On Fri, 2005-08-19 at 16:03 -0500, John A Meinel wrote:
>> > Jeremiah Jahn wrote:
>> > > On Fri, 2005-08-19 at 12:18 -0500, John A Meinel wrote:
>> > >
>> <snip>
>> >
>> > > it's cached alright. I'm getting a read rate of about 150MB/sec. I 
>> would
>> > > have thought is would be faster with my raid setup. I think I'm 
>> going to
>> > > scrap the whole thing and get rid of LVM. I'll just do a straight 
>> ext3
>> > > system. Maybe that will help. Still trying to get suggestions for a
>> > > stripe size.
>> > >

Well, since you can get a read of the RAID at 150MB/s, that means that 
it is actual I/O speed. It may not be cached in RAM. Perhaps you could 
try the same test, only using say 1G, which should be cached.

>> >
>> > I don't think 150MB/s is out of the realm for a 14 drive array.
>> > How fast is time dd if=/dev/zero of=testfile bs=8192 count=1000000
>> >
>> time dd if=/dev/zero of=testfile bs=8192 count=1000000
>> 1000000+0 records in
>> 1000000+0 records out
>>
>> real    1m24.248s
>> user    0m0.381s
>> sys     0m33.028s
>>
>>
>> > (That should create a 8GB file, which is too big to cache everything)
>> > And then how fast is:
>> > time dd if=testfile of=/dev/null bs=8192 count=1000000
>>
>> time dd if=testfile of=/dev/null bs=8192 count=1000000
>> 1000000+0 records in
>> 1000000+0 records out
>>
>> real    0m54.139s
>> user    0m0.326s
>> sys     0m8.916s
>>
>>
>> and on a second run:
>>
>> real    0m55.667s
>> user    0m0.341s
>> sys     0m9.013s
>>
>>
>> >
>> > That should give you a semi-decent way of measuring how fast the RAID
>> > system is, since it should be too big to cache in ram.
>>
>> about 150MB/Sec. Is there no better way to make this go faster...?

I'm actually curious about PCI bus saturation at this point. Old 32-bit 
33MHz pci could only push 1Gbit = 100MB/s. Now, I'm guessing that this 
is a higher performance system. But I'm really surprised that your write 
speed is that close to your read speed. (100MB/s write, 150MB/s read).

> 
> Assuming these are U320 15Krpm 147GB HDs, a RAID 10 array of 14 of them 
> doing raw sequential IO like this should be capable of at
>  ~7*75MB/s= 525MB/s using Seagate Cheetah 15K.4's,  ~7*79MB/s= 553MB/s 
> if using Fujitsu MAU's, and ~7*86MB/s= 602MB/s if using Maxtor Atlas 15K 
> II's to devices external to the RAID array.

I know I thought these were SATA drives, over 2 controllers. I could be 
completely wrong, though.

> 
> _IF_ the controller setup is high powered enough to keep that kind of IO 
> rate up.  This will require a controller or controllers providing dual 
> channel U320 bandwidth externally and quad channel U320 bandwidth 
> internally.  IOW, it needs a controller or controllers talking 64b 
> 133MHz PCI-X, reasonably fast DSP/CPU units, and probably a decent sized 
> IO buffer as well.
> 
> AFAICT, the Dell PERC4 controllers use various flavors of the LSI Logic 
> MegaRAID controllers.  What I don't know is which exact one yours is, 
> nor do I know if it (or any of the MegaRAID controllers) are high 
> powered enough.
> 
> Talk to your HW supplier to make sure you have controllers adequate to 
> your HD's.
> 
> ...and yes, your average access time will be in the 5.5ms - 6ms range 
> when doing a physical seek.
> Even with RAID, you want to minimize seeks and maximize sequential IO 
> when accessing them.
> Best to not go to HD at all ;-)

Well, certainly, if you can get more into RAM, you're always better off. 
For writing, a battery-backed write cache, and for reading lots of 
system RAM.

> 
> Hope this helps,
> Ron Peacetree
> 

John
=:->

--------------enig6CF0AF8208E4B7FC20509C12
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (Cygwin)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFDB+ekJdeBCYSNAAMRAlbBAKCPJZ4Ox/4u9nFiD5u0fZT9zpTDpwCferGG
CmQ+DrIWO1Rw86QSK9rrRfw=
=PICd
-----END PGP SIGNATURE-----

--------------enig6CF0AF8208E4B7FC20509C12--