Message-ID: <4DC0853D.4070401@2ndquadrant.com>
Date: Tue, 03 May 2011 18:44:13 -0400
From: Greg Smith <greg@2ndquadrant.com>
User-Agent: Thunderbird 2.0.0.24 (X11/20100317)
MIME-Version: 1.0
To: Greg Spiegelberg <gspiegelberg@gmail.com>
CC: Alan Hodgson <ahodgson@simkin.ca>, pgsql-performance@postgresql.org
Subject: Re: amazon ec2
References: <ADB48657-780F-4465-ABE6-752C2FB3AF48@gmail.com>
	<201105031241.20348.ahodgson@simkin.ca>
	<F3B854F7-53C8-4895-99BF-EB64BD6E5B53@gmail.com>
	<201105031309.52154.ahodgson@simkin.ca>
	<BANLkTi=nTpG=bS+rAONQi_ZDorBMwUzN1Q@mail.gmail.com>
In-Reply-To: <BANLkTi=nTpG=bS+rAONQi_ZDorBMwUzN1Q@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Greg Spiegelberg wrote:
> I ran pgbench tests late last year comparing EC2, GoGrid, a 5 year-old 
> lab server and a new server.  Whether I used a stock postgresql.conf 
> or tweaked, the current 8.4 or 9.0, or varied the EC2 instance size 
> EC2 was always at the bottom ranging from 409.834 to 693.100 tps. 
>  GoGrid's pgbench TPS numbers in similar tests were, on average, 3X 
> that of EC2 (1,399.550 to 1,631.887 tps).  The tests I conducted were 
> small with 10 connections and total 5,000 transactions.  The single 
> variable that helped pgbench tests in EC2 was to select an instance 
> size where the number of cores was equal to or greater than the number 
> of connections I used in the tests however this only improved things 
> slightly (715.931 tps).

The standard pgbench test is extremely sensitive to how fast 
transactions can be committed to disk.  That doesn't reflect what 
performance looks like on most real-world workloads, which tend toward 
more reads.  The fact that GoGrid is much faster at doing commits than 
EC2 is interesting, but that's only one of many parameters that impact 
performance on more normal workloads.

The one parameter that can change how the test runs is turning off 
synchronous_commit, which pulls the commit time out of the results to 
some extent.  And if you'd switched pgbench to a more read-oriented 
test, you'd discover it becomes extremely sensitive to the size of the 
database, as set by pgbench's scale parameter during setup.

-- 
Greg Smith   2ndQuadrant US    greg@2ndQuadrant.com   Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support  www.2ndQuadrant.us
"PostgreSQL 9.0 High Performance": http://www.2ndQuadrant.com/books