Date: Mon, 16 May 2011 17:06:36 +0000
From: John Rouillard <rouilj@renesys.com>
To: Jeff <threshar@torgo.dyndns-server.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: Using pgiosim realistically
Message-ID: <20110516170636.GB27218@renesys.com>
References: <20110513210941.GX27218@renesys.com>
	<20110514170702.GC32389@staff-mud-56-27.rice.edu>
	<20110516131730.GA27218@renesys.com>
	<B15C84A0-369F-4F29-B3F3-A85C3107DDBD@torgo.978.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <B15C84A0-369F-4F29-B3F3-A85C3107DDBD@torgo.978.org>
User-Agent: Mutt/1.5.20 (2009-06-14)

On Mon, May 16, 2011 at 12:23:13PM -0400, Jeff wrote:
> On May 16, 2011, at 9:17 AM, John Rouillard wrote:
> >However, in my case I have an 8 disk raid 10 with a read only load (in
> >this testing configuration). Shouldn't I expect more iops than a
> >single disk can provide? Maybe pgiosim is hitting some other boundary
> >than just i/o?
> >
> 
> given your command line you are only running a single thread - use
> the -t argument to add more threads and that'll increase
> concurrency.  a single process can only process so much at once and
> with multiple threads requesting different things the drive will
> actually be able to respond faster since it will have more work to
> do.
> I tend to test various levels - usually a single (-t 1 - the
> default) to get a base line, then -t (drives / 2), -t (#drives) up
> to probably 4x drives (you'll see iops level off).

Ok cool. I'll try that.
 
> >Also it turns out that pgiosim can only handle 64 files. I haven't
> >checked to see if this is a compile time changable item or not.
> 
> that is a #define in pgiosim.c

So which is a better test, modifying the #define to allow specifying
200-300 1GB files, or using 64 files but increasing the size of my
files to 2-3GB for a total bytes in the file two or three times the
memory in my server (96GB)?

> also, are you running the latest pgiosim from pgfoundry?

yup version 0.5 from the foundry.

> the -w param to pgiosim has it rewrite blocks out as it runs. (it is
> a percentage).

Yup, I was running with that and getting low enough numbers, that I
switched to pure read tests. It looks like I just need multiple
threads so I can have multiple reads/writes in flight at the same
time.

-- 
				-- rouilj

John Rouillard       System Administrator
Renesys Corporation  603-244-9084 (cell)  603-643-9300 x 111