Date: Tue, 6 May 2008 16:21:08 -0400 (EDT)
From: Greg Smith <gsmith@gregsmith.com>
To: Craig James <craig_james@emolecules.com>
cc: pgsql-performance@postgresql.org
Subject: Re: RAID 10 Benchmark with different I/O schedulers
In-Reply-To: <4820A3C2.4000307@emolecules.com>
Message-ID: <Pine.GSO.4.64.0805061605030.11474@westnet.com>
References: 
 <D960CB61B694CF459DCFB4B0128514C201FA62C8@exadv11.host.magwien.gv.at>
	<C050E63A-2209-4E89-AA99-6A327CCE7380@torgo.978.org>
	<Pine.LNX.4.64.0804111736580.20402@aragorn.flymine.org>
	<47FFA7FF.10801@postnewspapers.com.au>
	<544D07A2-EB3F-4D91-A749-0C4D5D38AB80@gmail.com>
	<481F992D.7070509@emolecules.com>
	<Pine.GSO.4.64.0805052354120.3752@westnet.com>
	<4820A3C2.4000307@emolecules.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed

On Tue, 6 May 2008, Craig James wrote:

> I only did two runs of each, which took about 24 minutes.  Like the first 
> round of tests, the "noise" in the measurements (about 10%) exceeds the 
> difference between scheduler-algorithm performance, except that 
> "anticipatory" seems to be measurably slower.

Those are much better results.  Any test that says anticipatory is 
anything other than useless for database system use with a good controller 
I presume is broken, so that's how I know you're in the right ballpark now 
but weren't before.

In order to actually get some useful data out of the noise that is 
pgbench, you need a lot more measurements of longer runs.  As perspective, 
the last time I did something in this area, in order to get enough data to 
get a clear picture I ran tests for 12 hours.  I'm hoping to repeat that 
soon with some more common hardware that gives useful results I can give 
out.

> So it still looks like cfq, noop and deadline are more or less equivalent 
> when used with a battery-backed RAID.

I think it's fair to say they're within 10% of one another on raw 
throughput.  The thing you're not measuring here is worst-case latency, 
and that's where there might be a more interesting difference.  Most tests 
I've seen suggest deadline is the best in that regard, cfq the worst, and 
where noop fits in depends on the underlying controller.

pgbench produces log files with latency measurements if you pass it "-l". 
Here's a snippet of shell that runs pgbench then looks at the resulting 
latency results for the worst 5 numbers:

pgbench ... -l &
p=$!
wait $p
mv pgbench_log.${p} pgbench.log
echo Worst latency results:
cat pgbench.log | cut -f 3 -d " " | sort -n | tail -n 5

However, that may not give you much useful info either--in most cases 
checkpoint issues kind of swamp the worst-base behavior in PostgreSQL, 
and to quantify I/O schedulers you need to look more complicated 
statistics on latency.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD