User-Agent: Microsoft-Entourage/11.2.1.051004
Date: Mon, 21 Nov 2005 10:14:29 -0800
Subject: Re: Hardware/OS recommendations for large databases (
From: "Luke Lonergan" <llonergan@greenplum.com>
To: "Tom Lane" <tgl@sss.pgh.pa.us>
Cc: "Alan Stange" <stange@rentec.com>,
	"Greg Stark" <gsstark@mit.edu>, "Dave Cramer" <pg@fastcrypt.com>,
	"Joshua Marsh" <icub3d@gmail.com>, pgsql-performance@postgresql.org
Message-ID: <BFA74E85.142B8%llonergan@greenplum.com>
Thread-Topic: [PERFORM] Hardware/OS recommendations for large databases
 (
Thread-Index: AcXuq9ofs/zrYpYQQESePaNtiZGFLgAG4+iw
In-Reply-To: <1611.1132585006@sss.pgh.pa.us>
MIME-Version: 1.0
Content-Type: text/plain;
 charset=us-ascii
Content-Transfer-Encoding: 7bit

Tom,

On 11/21/05 6:56 AM, "Tom Lane" <tgl@sss.pgh.pa.us> wrote:

> "Luke Lonergan" <llonergan@greenplum.com> writes:
>> OK - slower this time:
> 
>> We've seen between 110MB/s and 120MB/s on a wide variety of fast CPU
>> machines with fast I/O subsystems that can sustain 250MB/s+ using dd, but
>> which all are capped at 120MB/s when doing sequential scans with different
>> versions of Postgres.
> 
> Luke, sometime it would be nice if you would post your raw evidence
> and let other people do their own analysis.  I for one have gotten
> tired of reading sweeping generalizations unbacked by any data.

This has partly been a challenge to get others to post their results.
  
> I find the notion of a magic 120MB/s barrier, independent of either
> CPU or disk speed, to be pretty dubious to say the least.  I would
> like to know exactly what the "wide variety" of data points you
> haven't shown us are.

I'll try to put up some of them, they've occurred over the last 3 years on
various platforms including:
- Dual 3.2GHz Xeon, 2 x Adaptec U320 SCSI attached to 6 x 10K RPM disks,
Linux 2.6.4(?) - 2.6.10 kernel, ext2/3 and Reiser filesystems
120-130MB/s Postgres seq scan rate on 7.4 and 8.0.

- Dual 1.8 GHz Opteron, 2 x LSI U320 SCSI attached to 6 x 10K RPM disks,
Linux 2.6.10 kernel, ext2/3 and Reiser filesystems
110-120MB/s Postgres seq scan rate on 8.0

- Same machine as above running Solaris 10, with UFS filesystem.  When I/O
caching is tuned, we reach the same 110-120MB/s Postgres seq scan rate

- Sam machine as above with 7 x 15K RPM 144GB disks in an external disk
tray, same scan rate

Only when we got these new SATA systems and tried out XFS with large
readahead have we been able to break past the 120-130MB/s.  After Alan's
post, it seems that XFS might be a big part of that.  I think we'll test
ext2/3 against XFS on the same machine to find out.

It may have to wait a week, as many of us are on vacation.

- Luke