DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:cc:content-type:content-transfer-encoding;
	b=WYVcHbGX7HG0zeS0rBNHUPuPHxngEMOjpLBpJZdxVc69IIdYleR/R7mfAphF1+HB8I
	TF5vWilobWHXW/O5aAE0ifpkXdRmQhlAfomo/bNk+eR1MPOnZy8k80qjR4Y2lWqOwHnD
	2wSI6vDHz4UAzpchmSvkL5KqJlc9OcolGTUp0=
MIME-Version: 1.0
In-Reply-To: <alpine.GSO.2.01.0903170012530.5259@westnet.com>
References: <49BE2E84.2040506@bortal.de>
	<alpine.GSO.2.01.0903161052430.9846@westnet.com>
	<87bps132t1.fsf@oxford.xeocode.com>
	<alpine.GSO.2.01.0903161227130.13431@westnet.com>
	<87ljr50zqp.fsf@oxford.xeocode.com>
	<alpine.GSO.2.01.0903170012530.5259@westnet.com>
Date: Mon, 16 Mar 2009 23:33:12 -0600
Message-ID: <dcc563d10903162233ta0b6517m6793d6c74a8418f2@mail.gmail.com>
Subject: Re: Postgres benchmarking with pgbench
From: Scott Marlowe <scott.marlowe@gmail.com>
To: Greg Smith <gsmith@gregsmith.com>
Cc: Gregory Stark <stark@enterprisedb.com>, "ml@bortal.de" <ml@bortal.de>,
	pgsql-performance@postgresql.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Mon, Mar 16, 2009 at 10:17 PM, Greg Smith <gsmith@gregsmith.com> wrote:
> On Tue, 17 Mar 2009, Gregory Stark wrote:
>
>> I think pgbench is just not that great a model for real-world usage
>
> pgbench's default workload isn't a good model for anything. =A0It wasn't =
a
> particularly real-world test when the TPC-B it's based on was created, an=
d
> that was way back in 1990. =A0And pgbench isn't even a good implementatio=
n of
> that spec (the rows are too narrow, comments about that in pgbench.c).
>
> At this point, the only good thing you can say about pgbench is that it's
> been a useful tool for comparing successive releases of PostgreSQL in a
> relatively fair way. =A0Basically, it measures what pgbench measures, and=
 that
> has only a loose relationship with what people want a database to do.

I'd say pgbench is best in negative.  I.e it can't tell you a database
server is gonna be fast, but it can usually tell you when something's
horrifically wrong.  If I just installed a new external storage array
of some kind and I'm getting 6 tps, something is wrong somewhere.

And it's good for exercising your disk farm for a week during burn in.
 It certainly turned up a bad RAID card last fall during acceptance
testing our new servers.  Took 36 hours of pgbench to trip the bug and
cause the card to lock up.  Had one bad disk drive too that pgbench
killed of for me.