From: Scott Carey <scott@richrelevance.com>
To: Josh Berkus <josh@agliodbs.com>, henk de wit <henk53602@hotmail.com>
CC: "jesper@krogh.cc" <jesper@krogh.cc>, "pgsql-performance@postgresql.org"
	<pgsql-performance@postgresql.org>
Date: Fri, 10 Apr 2009 10:11:46 -0700
Subject: Re: Using IOZone to simulate DB access patterns
Thread-Topic: [PERFORM] Using IOZone to simulate DB access patterns
Thread-Index: Acm5nxwt5pLCefJhSHWsqVl7s7jJZgAYFE5Z
Message-ID: <C604CBE2.4856%scott@richrelevance.com>
In-Reply-To: <49DEDC1B.7080303@agliodbs.com>
Accept-Language: en-US
Content-Language: en
acceptlanguage: en-US
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

I've switched to using FIO.

Bonnie in my experience produces poor results and is better suited to
testing desktop/workstation type load.  Most of its tests don't apply to ho=
w
postgres writes/reads anyway.

IOZone is a bit more troublesome to get it to work on the file(s) you want
under concurrency and is also hard to get it to avoid the OS file cache.  O=
n
systems with lots of RAM,  it takes too long as a result.  I personally lik=
e
it better than bonnnie by far, but its not flexible enough for me and is
often used by hardware providers to 'show' theier RAID cards are doing fine
(PERC 6 doing 4GB /sec file access -- see!  Its fine!) but the thing is jus=
t
testing in memory cached reads for most of the test or all if not configure=
d
right...

FIO with profiles such as the below samples are easy to set up, and they ca=
n
be mix/matched to test what happens with mixed read/write seq/rand -- with
surprising and useful tuning results.  Forcing a cache flush or sync before
or after a run is trivial.  Changing to asynchronous I/O, direct I/O, or
other forms is trivial.  The output result formatting is very useful as
well.

I got into using FIO when I needed to test a matrix of about 400 different
tuning combinations.  This would have taken a month with Iozone, but I coul=
d
create my profiles with FIO, force the OS cache to flush, and constrain the
time appropriately for each test, and run the batch overnight.


#----------------
[read-rand]
rw=3Drandread
; this will be total of all individual files per process
size=3D1g
directory=3D/data/test
fadvise_hint=3D0
blocksize=3D8k
direct=3D0
ioengine=3Dsync
iodepth=3D1
numjobs=3D32
; this is number of files total per process
nrfiles=3D1
group_reporting=3D1
runtime=3D1m
exec_prerun=3Decho 3 > /proc/sys/vm/drop_caches
#--------------------
[read]
rw=3Dread
; this will be total of all individual files per process
size=3D512m
directory=3D/data/test
fadvise_hint=3D0
blocksize=3D8k
direct=3D0
ioengine=3Dsync
iodepth=3D1
numjobs=3D8
; this is number of files total per process
nrfiles=3D1
runtime=3D30s
group_reporting=3D1
exec_prerun=3Decho 3 > /proc/sys/vm/drop_caches

#----------------------
[write]
rw=3Dwrite
; this will be total of all individual files per process
size=3D4g
directory=3D/data/test
fadvise_hint=3D0
blocksize=3D8k
direct=3D0
ioengine=3Dsync
iodepth=3D1
numjobs=3D1
;rate=3D10000
; this is number of files total per process
nrfiles=3D1
runtime=3D48s
group_reporting=3D1
end_fsync=3D1
exec_prerun=3Decho 3 >sync;  /proc/sys/vm/drop_caches


On 4/9/09 10:41 PM, "Josh Berkus" <josh@agliodbs.com> wrote:

> All,
>=20
> Wow, am I really the only person here who's used IOZone?
>=20
> --
> Josh Berkus
> PostgreSQL Experts Inc.
> www.pgexperts.com
>=20
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org=
)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
>=20