pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Thomas F.O'Connell <tfo@sitening.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: PgSQL - Performance <pgsql-performance@postgresql.org>
Subject: Re: pgbench Comparison of 7.4.7 to 8.0.2
Date: Sat, 23 Apr 2005 21:31:13 -0500
Message-ID: <b4a298793e5eee60fb12a725197d5c42@sitening.com> (raw)
In-Reply-To: <6865.1113600214@sss.pgh.pa.us>
References: <2784476415a96bb0912ebca41edb3d84@sitening.com>
	<6865.1113600214@sss.pgh.pa.us>

Okay. I updated my benchmark page with new numbers, which are the 
result of extensive pgbench usage over this past week. In fact, I 
modified pgbench (for both of the latest version of postgres) to be 
able to accept multiple iterations as an argument and report the 
results of each iteration as well as a summary of mean tps at the end. 
The modifications of the source are included on the new page, and I'd 
be happy to submit them as patches if this seems like useful 
functionality to the developers and the community. I find it nicer to 
have pgbench be the authoritative source of iterative results rather 
than a wrapper script, but it'd be nice to have an extra set of eyes 
guarantee that I've included in the loop everything that ought to be 
there.

A couple of notes:

* There was some interesting oscillation behavior in both version of 
postgres that occurred with 25 clients and 1000 transactions at a 
scaling factor of 100. This was repeatable with the distribution 
version of pgbench run iteratively from the command line. I'm not sure 
how to explain this.

* I'm not really sure why the single client run at 1000 transactions 
seemed so much slower than all successive iterations, including single 
client with 10000 transactions at a scaling factor of 100. It's 
possible that I should be concerned about how throughput was so much 
higher for 10000 transactions.

Anyway, the code changes, the configuration details, and the results 
are all posted here:

http://www.sitening.com/pgbench.html

Once again, I'd be curious to get feedback from developers and the 
community about the results, and I'm happy to answer any questions.

-tfo

--
Thomas F. O'Connell
Co-Founder, Information Architect
Sitening, LLC

Strategic Open Source: Open Your i™

http://www.sitening.com/
110 30th Avenue North, Suite 6
Nashville, TN 37203-6320
615-260-0005

On Apr 15, 2005, at 4:23 PM, Tom Lane wrote:

> "Thomas F.O'Connell" <tfo@sitening.com> writes:
>> http://www.sitening.com/pgbench.html
>
> You need to run *many* more transactions than that to get pgbench
> numbers that aren't mostly noise.  In my experience 1000 transactions
> per client is a rock-bottom minimum to get repeatable numbers; 10000 
> per
> is better.
>
> Also, in any run where #clients >= scaling factor, what you're 
> measuring
> is primarily contention to update the "branches" rows.  Which is not
> necessarily a bad thing to check, but it's generally not the most
> interesting performance domain (if your app is like that you need to
> redesign the app...)
>
>> To me, it looks like basic transactional performance is modestly
>> improved at 8.0 across a variety of metrics.
>
> That's what I would expect --- we usually do some performance work in
> every release cycle, but there was not a huge amount of it for 8.0.
>
> However, these numbers don't prove much either way.
>
> 			regards, tom lane




view thread (10+ messages)  latest in thread

Message-ID: <b4a298793e5eee60fb12a725197d5c42@sitening.com>
Permalink:  ../b4a298793e5eee60fb12a725197d5c42@sitening.com/
Also on:    postgresql.org/message-id/b4a298793e5eee60fb12a725197d5c42@sitening.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-performance@postgresql.org
  Cc: tfo@sitening.com, tgl@sss.pgh.pa.us
  Subject: Re: pgbench Comparison of 7.4.7 to 8.0.2
  In-Reply-To: <b4a298793e5eee60fb12a725197d5c42@sitening.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox