Subject: Re: pgbench results interpretation?
From: Joost Kraaijeveld <J.Kraaijeveld@Askesis.nl>
To: Gavin Sherry <swm@alcove.com.au>
Cc: Pgsql-Performance <pgsql-performance@postgresql.org>
In-Reply-To: <Pine.LNX.4.58.0511012011460.6141@linuxworld.com.au>
References: <1130832189.21036.19.camel@Panoramix>
	<Pine.LNX.4.58.0511012011460.6141@linuxworld.com.au>
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
Date: Tue, 01 Nov 2005 11:05:42 +0100
Message-Id: <1130839542.3883.19.camel@Panoramix>
Mime-Version: 1.0

Hi Gavin,

Thanks for answering.

On Tue, 2005-11-01 at 20:16 +1100, Gavin Sherry wrote:
> On Tue, 1 Nov 2005, Joost Kraaijeveld wrote:
> > 1. Is there a repository somewhere that shows results, using and
> > documenting different kinds of hard- and software setups so that I can
> > compare my results with someone elses?
> 
> Other than the archives of this mailing list, no.
OK.

> >
> > 2. Is there a reason for the difference in values from run-to-run of
> > pgbench:
> Well, firstly: pgbench is not a good benchmarking tool. 
Is there a reason why that is the case? I would like to understand why?
Is it because the transaction is to small/large? Or that the queries are
to small/large? Or just experience?

> It is mostly used
> to generate load. Secondly, the numbers are suspicious: do you have fsync
> turned off? 
In the first trials I posted yes, in the second no.

> Do you have write caching enabled? If so, you'd want to make
> sure that cache is battery backed. 
I am aware of that, but for now, I am mostly interested in the effects
of the configuration parameters. I won't do this at home ;-)


> Thirdly, the effects of caching will be
> seen on subsequent runs.
In that case I would expect mostly rising values. I only copied and
pasted 4 trials that were available in my xterm at the time of writing
my email, but I could expand the list ad infinitum: the variance between
the runs is very large. I also expect that if there is no shortage of
memory wrt caching that the effect would be negligible, but I may be
wrong. Part of using pgbench is learning about performance, not
achieving it.

> > 3. It appears that running more transactions with the same amount of
> > clients leads to a drop in the transactions per second. I do not
> > understand why this is (a drop from more clients I do understand). Is
> > this because of the way pgbench works, the way PostgrSQL works or even
> > Linux?
> This degradation seems to suggest effects caused by the disk cache filling
> up (assuming write caching is enabled) and checkpointing.
Which diskcache are your referring to? The onboard harddisk or RAID5
controller caches or the OS cache? The first two I can unstand but if
you refer to the OS cache I do not understand what I am seeing. I have
enough memory giving the size of the database: during these duration (~)
tests fsync was on, and the files could be loaded into memory easily
(effective_cache_size = 32768 which is ~ 265 MB, the complete database
directory 228 MB)

-- 
Groeten,

Joost Kraaijeveld
Askesis B.V.
Molukkenstraat 14
6524NB Nijmegen
tel: 024-3888063 / 06-51855277
fax: 024-3608416
e-mail: J.Kraaijeveld@Askesis.nl
web: www.askesis.nl