Message-ID: <4DC08420.7050602@2ndquadrant.com>
Date: Tue, 03 May 2011 18:39:28 -0400
From: Greg Smith <greg@2ndquadrant.com>
User-Agent: Thunderbird 2.0.0.24 (X11/20100317)
MIME-Version: 1.0
To: Mark Rostron <mrostron@ql2.com>
CC: Alan Hodgson <ahodgson@simkin.ca>, pgsql-performance@postgresql.org
Subject: Re: amazon ec2
References: <ADB48657-780F-4465-ABE6-752C2FB3AF48@gmail.com>
	<201105031241.20348.ahodgson@simkin.ca>
	<F3B854F7-53C8-4895-99BF-EB64BD6E5B53@gmail.com>
	<201105031309.52154.ahodgson@simkin.ca>
	<BANLkTinKbgzSGwg1GWTpwUCkwSpaCaebsA@mail.gmail.com>
In-Reply-To: <BANLkTinKbgzSGwg1GWTpwUCkwSpaCaebsA@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Mark Rostron wrote:
> the success/failure of it depends on your typical query activity, the 
> size of your critical result set, and whether you are able to get 
> enough RAM to make this work.

Basically, it all comes down to "does the working set of data I access 
frequently fit in RAM?"  If it does, it's possible to get reasonable 
performance out of an EC2 instance.  The EBS disks are so slow, both on 
average and particularly in cases where you have contention with other 
users slowing you down, that any situation where you have to use them is 
never going to work well.  If most of the data fits in RAM, and the CPU 
resources available to your instance are sufficient to service your 
queries, you might see acceptable performance.

> greenplum is also a better parallel machine than postgres, so 
> combining the cache technique above with greenplum compression and 
> parallel query, i have been able to get 20:1 reduction in response 
> times for some of our queries.

I've also seen over a 20:1 speedup over PostgreSQL by using Greenplum's 
free Community Edition server, in situations where its column store + 
compression features work well on the data set.  That's easiest with an 
append-only workload, and the data set needs to fit within the 
constraints where indexes on compressed data are useful.  But if you fit 
the use profile it's good at, you end up with considerable ability to 
trade-off using more CPU resources to speed up queries.  It effectively 
increases the amount of data that can be cached in RAM by a large 
multiple, and in the EC2 context (where any access to disk is very slow) 
it can be quite valuable.  My colleague Gabrielle wrote something about 
setting this up on an earlier version of Greenplum's software at 
http://blog.2ndquadrant.com/en/2010/03/installing-greenplum-sne-ec2.html 
that gives an idea how that was setup.

-- 
Greg Smith   2ndQuadrant US    greg@2ndQuadrant.com   Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support  www.2ndQuadrant.us
"PostgreSQL 9.0 High Performance": http://www.2ndQuadrant.com/books