From: Luke Lonergan <llonergan@greenplum.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Alan Stange <stange@rentec.com>
Cc: Greg Stark <gsstark@mit.edu>
Cc: Dave Cramer <pg@fastcrypt.com>
Cc: Joshua Marsh <icub3d@gmail.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: Hardware/OS recommendations for large databases (
Date: Mon, 21 Nov 2005 10:14:29 -0800
Message-ID: <BFA74E85.142B8%llonergan@greenplum.com> (raw)
In-Reply-To: <1611.1132585006@sss.pgh.pa.us>
Tom,
On 11/21/05 6:56 AM, "Tom Lane" <tgl@sss.pgh.pa.us> wrote:
> "Luke Lonergan" <llonergan@greenplum.com> writes:
>> OK - slower this time:
>
>> We've seen between 110MB/s and 120MB/s on a wide variety of fast CPU
>> machines with fast I/O subsystems that can sustain 250MB/s+ using dd, but
>> which all are capped at 120MB/s when doing sequential scans with different
>> versions of Postgres.
>
> Luke, sometime it would be nice if you would post your raw evidence
> and let other people do their own analysis. I for one have gotten
> tired of reading sweeping generalizations unbacked by any data.
This has partly been a challenge to get others to post their results.
> I find the notion of a magic 120MB/s barrier, independent of either
> CPU or disk speed, to be pretty dubious to say the least. I would
> like to know exactly what the "wide variety" of data points you
> haven't shown us are.
I'll try to put up some of them, they've occurred over the last 3 years on
various platforms including:
- Dual 3.2GHz Xeon, 2 x Adaptec U320 SCSI attached to 6 x 10K RPM disks,
Linux 2.6.4(?) - 2.6.10 kernel, ext2/3 and Reiser filesystems
120-130MB/s Postgres seq scan rate on 7.4 and 8.0.
- Dual 1.8 GHz Opteron, 2 x LSI U320 SCSI attached to 6 x 10K RPM disks,
Linux 2.6.10 kernel, ext2/3 and Reiser filesystems
110-120MB/s Postgres seq scan rate on 8.0
- Same machine as above running Solaris 10, with UFS filesystem. When I/O
caching is tuned, we reach the same 110-120MB/s Postgres seq scan rate
- Sam machine as above with 7 x 15K RPM 144GB disks in an external disk
tray, same scan rate
Only when we got these new SATA systems and tried out XFS with large
readahead have we been able to break past the 120-130MB/s. After Alan's
post, it seems that XFS might be a big part of that. I think we'll test
ext2/3 against XFS on the same machine to find out.
It may have to wait a week, as many of us are on vacation.
- Luke
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: llonergan@greenplum.com, tgl@sss.pgh.pa.us, stange@rentec.com, gsstark@mit.edu, pg@fastcrypt.com, icub3d@gmail.com
Subject: Re: Hardware/OS recommendations for large databases (
In-Reply-To: <BFA74E85.142B8%llonergan@greenplum.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox