Date: Sun, 12 Feb 2012 14:01:59 -0600 (CST)
From: Joshua Berkus <josh@agliodbs.com>
To: Jeff Janes <jeff.janes@gmail.com>
Cc: Peter van Hardenberg <pvh@pvh.ca>, pgsql-performance@postgresql.org,
	Scott Marlowe <scott.marlowe@gmail.com>
Message-ID: <873667728.6754.1329076919645.JavaMail.root@mail-1.01.com>
In-Reply-To: 
 <CAMkU=1zzx96FQssKixaBubYCWw6Msf1E-2C18HYPDuDfSBTqwA@mail.gmail.com>
Subject: Re: random_page_cost = 2.0 on Heroku Postgres
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


> Is there an easy and unintrusive way to get such a metric as the
> aggregated query times?  And to normalize it for how much work
> happens
> to have been doing on at the time?

You'd pretty much need to do large-scale log harvesting combined with samples of query concurrency taken several times per minute.  Even that won't "normalize" things the way you want, though, since all queries are not equal in terms of the amount of data they hit.

Given that, I'd personally take a statistical approach.  Sample query execution times across a large population of servers and over a moderate amount of time.  Then apply common tests of statistical significance.  This is why Heroku has the opportunity to do this in a way that smaller sites could not; they have enough servers to (probably) cancel out any random activity effects.

--Josh Berkus