X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 4AFA653689 for ; Sat, 23 Apr 2005 23:33:03 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 55114-06 for ; Sun, 24 Apr 2005 02:32:57 +0000 (GMT) Received: from window.monsterlabs.com (window.monsterlabs.com [216.183.105.176]) by svr1.postgresql.org (Postfix) with SMTP id 696C8536C3 for ; Sat, 23 Apr 2005 23:32:56 -0300 (ADT) Received: (qmail 8047 invoked from network); 24 Apr 2005 02:33:00 -0000 Received: from host-209.149.56.238.nashville.net (HELO ?10.0.1.2?) (209.149.56.238) by 0 with SMTP; 24 Apr 2005 02:33:01 -0000 In-Reply-To: <42603127.5070108@sfnet.cc> References: <2784476415a96bb0912ebca41edb3d84@sitening.com> <42603127.5070108@sfnet.cc> Mime-Version: 1.0 (Apple Message framework v622) Content-Type: text/plain; charset=WINDOWS-1252; format=flowed Message-Id: Content-Transfer-Encoding: quoted-printable Cc: PgSQL - Performance From: Thomas F.O'Connell Subject: Re: pgbench Comparison of 7.4.7 to 8.0.2 Date: Sat, 23 Apr 2005 21:33:00 -0500 To: Steve Poe X-Mailer: Apple Mail (2.622) X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.008 tagged_above=0 required=5 tests=AWL X-Spam-Level: X-Archive-Number: 200504/661 X-Sequence-Number: 12058 Steve, Per your and Tom's recommendations, I significantly increased the=20 number of transactions used for testing. See my last post. The database will have pretty heavy mixed use, i.e., both reads and=20 writes. I performed 32 iterations per scenario this go-round. I'll look into OSDB for further benchmarking. Thanks for the tip. Since pgbench is part of the postgres distribution and I had it at hand=20= and it seems to be somewhat widely referenced, I figured I go ahead and=20= post preliminary results from it. -tfo -- Thomas F. O'Connell Co-Founder, Information Architect Sitening, LLC Strategic Open Source: Open Your i=99 http://www.sitening.com/ 110 30th Avenue North, Suite 6 Nashville, TN 37203-6320 615-260-0005 On Apr 15, 2005, at 4:24 PM, Steve Poe wrote: > Tom, > > People's opinions on pgbench may vary, so take what I say with a grain=20= > of salt. Here are my thoughts: > > 1) Test with no less than 200 transactions per client. I've heard with=20= > less than this, your results will vary too much with the direction of=20= > the wind blowing. A high enough value will help rule out some "noise"=20= > factor. If I am wrong, please let me know. > > > 2) How is the database going to be used? What percentage will be=20 > read/write if you had to guess? Pgbench is like a TPC-B with will help=20= > guage the potential throughput of your tps. However, it may not stress=20= > the server enough to help you make key performance changes. However,=20= > benchmarks are like statistics...full of lies . > > 3) Run not just a couple pgbench runs, but *many* (I do between 20-40=20= > runs) so you can rule out noise and guage improvement on median=20 > results. > > 4) Find something that you test OLTP-type transactions. I used OSDB=20 > since it is simple to implement and use. Although OSDL's OLTP testing=20= > will closer to reality. > > Steve Poe