User-Agent: Microsoft-Entourage/11.2.1.051004
Date: Fri, 18 Nov 2005 05:46:58 -0800
Subject: Re: Hardware/OS recommendations for large databases (
From: "Luke Lonergan" <llonergan@greenplum.com>
To: stange@rentec.com
cc: "Dave Cramer" <pg@fastcrypt.com>, "Greg Stark" <gsstark@mit.edu>,
	"Joshua Marsh" <icub3d@gmail.com>, pgsql-performance@postgresql.org
Message-ID: <BFA31B52.14012%llonergan@greenplum.com>
Thread-Topic: [PERFORM] Hardware/OS recommendations for large databases
 (
Thread-Index: AcXsReV45/v76ff9T2yQxgfgf3wdzwAAKXmW
In-Reply-To: <437DDA26.4000603@rentec.com>
MIME-Version: 1.0
Content-Type: text/plain;
 charset=us-ascii
Content-Transfer-Encoding: 7bit

Alan,

On 11/18/05 5:41 AM, "Alan Stange" <stange@rentec.com> wrote:

> 
> That's interesting, as I occasionally see more than 110MB/s of
> postgresql IO on our system.  I'm using a 32KB block size, which has
> been a huge win in performance for our usage patterns.   300GB database
> with a lot of turnover.  A vacuum analyze now takes about 3 hours, which
> is much shorter than before.  Postgresql 8.1, dual opteron, 8GB memory,
> Linux 2.6.11, FC drives.

300GB / 3 hours = 27MB/s.

If you are using the 2.6 linux kernel, you may be fooled into thinking you
burst more than you actually get in net I/O because the I/O stats changed in
tools like iostat and vmstat.

The only meaningful stats are (size of data) / (time to process data).  Do a
sequential scan of one of your large tables that you know the size of, then
divide by the run time and report it.

I'm compiling some new test data to make my point now.

Regards,

- Luke