Message-ID: <4E6E31DF.6020407@peak6.com>
Date: Mon, 12 Sep 2011 11:22:55 -0500
From: Shaun Thomas <sthomas@peak6.com>
Reply-To: <sthomas@peak6.com>
Organization: Peak6
User-Agent: Mozilla/5.0 (X11; Linux i686;
	rv:5.0) Gecko/20110624 Thunderbird/5.0
MIME-Version: 1.0
To: Marti Raudsepp <marti@juffo.org>
CC: Andy Colson <andy@squeakycode.net>, Igor Chudov <ichudov@gmail.com>,
	<pgsql-performance@postgresql.org>
Subject: Re: Postgres for a "data warehouse", 5-10 TB
References: 
 <CAMhtkAbK7saRO_Tayb1nS3YLWN=Wns=Ci47qs-3YbX0T2+hkpA@mail.gmail.com>
	<CAOR=d=0zmw-EGRJyYmwCTxR5g6osbFLS5i-3kzjG05j7S=1JaA@mail.gmail.com>
	<CAMhtkAah2c4XfSec=OtgL1V51wpD=jygbZnBVBRYCV02MscebQ@mail.gmail.com>
	<4E6CC458.4010206@squeakycode.net>
	<CABRT9RB-etP-K-0u0y-Wv+BysSd9ToZm71BqcY7hFQZC2tmjYQ@mail.gmail.com>
In-Reply-To: 
 <CABRT9RB-etP-K-0u0y-Wv+BysSd9ToZm71BqcY7hFQZC2tmjYQ@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
content-transfer-encoding: 7bit

On 09/11/2011 12:02 PM, Marti Raudsepp wrote:

> Which brings me to another important point: don't do lots of small
> write transactions, SAVEPOINTs or PL/pgSQL subtransactions. Besides
> being inefficient, they introduce a big maintenance burden.

I'd like to second this. Before a notable application overhaul, we were 
handling about 300-million transactions per day (250M of that was over a 
6-hour period). To avoid the risk of mid-day vacuum-freeze, we disabled 
autovacuum and run a nightly vacuum over the entire database. And that 
was *after* bumping  autovacuum_freeze_max_age to 600-million.

You do *not* want to screw with that if you don't have to, and a setting 
of 600M is about 1/3 of the reasonable boundary there. If not for the 
forced autovacuums, a database with this much traffic would be corrupt 
in less than a week. We've managed to cut that transaction traffic by 
60%, and it greatly improved the database's overall health.

-- 
Shaun Thomas
OptionsHouse | 141 W. Jackson Blvd. | Suite 800 | Chicago IL, 60604
312-676-8870
sthomas@peak6.com

______________________________________________

See http://www.peak6.com/email-disclaimer/ for terms and conditions related to this email