X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 4C522536D1 for ; Sat, 23 Apr 2005 23:31:25 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 55114-02 for ; Sun, 24 Apr 2005 02:31:15 +0000 (GMT) Received: from window.monsterlabs.com (window.monsterlabs.com [216.183.105.176]) by svr1.postgresql.org (Postfix) with SMTP id AF3855323A for ; Sat, 23 Apr 2005 23:31:10 -0300 (ADT) Received: (qmail 7454 invoked from network); 24 Apr 2005 02:31:14 -0000 Received: from host-209.149.56.238.nashville.net (HELO ?10.0.1.2?) (209.149.56.238) by 0 with SMTP; 24 Apr 2005 02:31:14 -0000 In-Reply-To: <6865.1113600214@sss.pgh.pa.us> References: <2784476415a96bb0912ebca41edb3d84@sitening.com> <6865.1113600214@sss.pgh.pa.us> Mime-Version: 1.0 (Apple Message framework v622) Content-Type: text/plain; charset=WINDOWS-1252; format=flowed Message-Id: Content-Transfer-Encoding: quoted-printable Cc: PgSQL - Performance From: Thomas F.O'Connell Subject: Re: pgbench Comparison of 7.4.7 to 8.0.2 Date: Sat, 23 Apr 2005 21:31:13 -0500 To: Tom Lane X-Mailer: Apple Mail (2.622) X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.009 tagged_above=0 required=5 tests=AWL X-Spam-Level: X-Archive-Number: 200504/660 X-Sequence-Number: 12057 Okay. I updated my benchmark page with new numbers, which are the=20 result of extensive pgbench usage over this past week. In fact, I=20 modified pgbench (for both of the latest version of postgres) to be=20 able to accept multiple iterations as an argument and report the=20 results of each iteration as well as a summary of mean tps at the end.=20= The modifications of the source are included on the new page, and I'd=20 be happy to submit them as patches if this seems like useful=20 functionality to the developers and the community. I find it nicer to=20 have pgbench be the authoritative source of iterative results rather=20 than a wrapper script, but it'd be nice to have an extra set of eyes=20 guarantee that I've included in the loop everything that ought to be=20 there. A couple of notes: * There was some interesting oscillation behavior in both version of=20 postgres that occurred with 25 clients and 1000 transactions at a=20 scaling factor of 100. This was repeatable with the distribution=20 version of pgbench run iteratively from the command line. I'm not sure=20= how to explain this. * I'm not really sure why the single client run at 1000 transactions=20 seemed so much slower than all successive iterations, including single=20= client with 10000 transactions at a scaling factor of 100. It's=20 possible that I should be concerned about how throughput was so much=20 higher for 10000 transactions. Anyway, the code changes, the configuration details, and the results=20 are all posted here: http://www.sitening.com/pgbench.html Once again, I'd be curious to get feedback from developers and the=20 community about the results, and I'm happy to answer any questions. -tfo -- Thomas F. O'Connell Co-Founder, Information Architect Sitening, LLC Strategic Open Source: Open Your i=99 http://www.sitening.com/ 110 30th Avenue North, Suite 6 Nashville, TN 37203-6320 615-260-0005 On Apr 15, 2005, at 4:23 PM, Tom Lane wrote: > "Thomas F.O'Connell" writes: >> http://www.sitening.com/pgbench.html > > You need to run *many* more transactions than that to get pgbench > numbers that aren't mostly noise. In my experience 1000 transactions > per client is a rock-bottom minimum to get repeatable numbers; 10000=20= > per > is better. > > Also, in any run where #clients >=3D scaling factor, what you're=20 > measuring > is primarily contention to update the "branches" rows. Which is not > necessarily a bad thing to check, but it's generally not the most > interesting performance domain (if your app is like that you need to > redesign the app...) > >> To me, it looks like basic transactional performance is modestly >> improved at 8.0 across a variety of metrics. > > That's what I would expect --- we usually do some performance work in > every release cycle, but there was not a huge amount of it for 8.0. > > However, these numbers don't prove much either way. > > regards, tom lane