Message-ID: <437FFEF0.9090409@rentec.com>
Date: Sat, 19 Nov 2005 23:43:28 -0500
From: Alan Stange <stange@rentec.com>
User-Agent: Thunderbird 1.5 (X11/20051025)
MIME-Version: 1.0
To: Luke Lonergan <llonergan@greenplum.com>
CC: Greg Stark <gsstark@mit.edu>, Dave Cramer <pg@fastcrypt.com>,
	Joshua Marsh <icub3d@gmail.com>, pgsql-performance@postgresql.org
Subject: Re: Hardware/OS recommendations for large databases (
References: <BFA48F15.14182%llonergan@greenplum.com>
In-Reply-To: <BFA48F15.14182%llonergan@greenplum.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Another data point. 

We had some down time on our system today to complete some maintenance 
work.  It took the opportunity to rebuild the 700GB file system using 
XFS instead of Reiser.

One iostat output for 30 seconds is

avg-cpu:  %user   %nice    %sys %iowait   %idle
           1.58    0.00   19.69   31.94   46.78

Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
sdd             343.73    175035.73       277.55    5251072       8326

while doing a select count(1) on the same large table as before.   
Subsequent iostat output all showed that this data rate was being 
maintained.  The system is otherwise mostly idle during this measurement.

The sequential read rate is 175MB/s.  The system is the same as earlier, 
one cpu is idle and the second is ~40% busy doing the scan and ~60% 
idle.   This is  postgresql 8.1rc1, 32KB block size.  No tuning except 
for using a 1024KB read ahead.

The peak speed of the attached storage is 200MB/s (a 2Gb/s fiber channel 
controller).  I see no reason why this configuration wouldn't generate 
higher IO rates if a faster IO connection were available.

Can you explain again why you think there's an IO ceiling of 120MB/s 
because I really don't understand?

-- Alan