From: Scott Carey <scott@richrelevance.com>
To: Josh Berkus <josh@agliodbs.com>
To: henk de wit <henk53602@hotmail.com>
Cc: jesper@krogh.cc <jesper@krogh.cc>
Cc: pgsql-performance@postgresql.org <pgsql-performance@postgresql.org>
Subject: Re: Using IOZone to simulate DB access patterns
Date: Fri, 10 Apr 2009 10:11:46 -0700
Message-ID: <C604CBE2.4856%scott@richrelevance.com> (raw)
In-Reply-To: <49DEDC1B.7080303@agliodbs.com>
I've switched to using FIO.
Bonnie in my experience produces poor results and is better suited to
testing desktop/workstation type load. Most of its tests don't apply to how
postgres writes/reads anyway.
IOZone is a bit more troublesome to get it to work on the file(s) you want
under concurrency and is also hard to get it to avoid the OS file cache. On
systems with lots of RAM, it takes too long as a result. I personally like
it better than bonnnie by far, but its not flexible enough for me and is
often used by hardware providers to 'show' theier RAID cards are doing fine
(PERC 6 doing 4GB /sec file access -- see! Its fine!) but the thing is just
testing in memory cached reads for most of the test or all if not configured
right...
FIO with profiles such as the below samples are easy to set up, and they can
be mix/matched to test what happens with mixed read/write seq/rand -- with
surprising and useful tuning results. Forcing a cache flush or sync before
or after a run is trivial. Changing to asynchronous I/O, direct I/O, or
other forms is trivial. The output result formatting is very useful as
well.
I got into using FIO when I needed to test a matrix of about 400 different
tuning combinations. This would have taken a month with Iozone, but I could
create my profiles with FIO, force the OS cache to flush, and constrain the
time appropriately for each test, and run the batch overnight.
#----------------
[read-rand]
rw=randread
; this will be total of all individual files per process
size=1g
directory=/data/test
fadvise_hint=0
blocksize=8k
direct=0
ioengine=sync
iodepth=1
numjobs=32
; this is number of files total per process
nrfiles=1
group_reporting=1
runtime=1m
exec_prerun=echo 3 > /proc/sys/vm/drop_caches
#--------------------
[read]
rw=read
; this will be total of all individual files per process
size=512m
directory=/data/test
fadvise_hint=0
blocksize=8k
direct=0
ioengine=sync
iodepth=1
numjobs=8
; this is number of files total per process
nrfiles=1
runtime=30s
group_reporting=1
exec_prerun=echo 3 > /proc/sys/vm/drop_caches
#----------------------
[write]
rw=write
; this will be total of all individual files per process
size=4g
directory=/data/test
fadvise_hint=0
blocksize=8k
direct=0
ioengine=sync
iodepth=1
numjobs=1
;rate=10000
; this is number of files total per process
nrfiles=1
runtime=48s
group_reporting=1
end_fsync=1
exec_prerun=echo 3 >sync; /proc/sys/vm/drop_caches
On 4/9/09 10:41 PM, "Josh Berkus" <josh@agliodbs.com> wrote:
> All,
>
> Wow, am I really the only person here who's used IOZone?
>
> --
> Josh Berkus
> PostgreSQL Experts Inc.
> www.pgexperts.com
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
>
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: scott@richrelevance.com, josh@agliodbs.com, henk53602@hotmail.com, jesper@krogh.cc
Subject: Re: Using IOZone to simulate DB access patterns
In-Reply-To: <C604CBE2.4856%scott@richrelevance.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox