Date: Thu, 24 Jan 2008 13:49:34 -0500 (EST)
From: Greg Smith <gsmith@gregsmith.com>
To: Tory M Blue <tmblue@gmail.com>
cc: pgsql-performance@postgresql.org
Subject: Re: Postgres 8.2 memory weirdness
In-Reply-To: <8a547c840801240946n37c7fad6ycf62187f82b2027c@mail.gmail.com>
Message-ID: <Pine.GSO.4.64.0801241335180.7217@westnet.com>
References: <8a547c840801231023q55796128nb2e88b8c8cfe86e@mail.gmail.com>
	<Pine.GSO.4.64.0801231923420.12679@westnet.com>
	<8a547c840801240946n37c7fad6ycf62187f82b2027c@mail.gmail.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed

On Thu, 24 Jan 2008, Tory M Blue wrote:

> Postg: 8.2.1fc6

8.2.1 has a nasty bug related to statistics collection that causes 
performance issues exactly in the kind of heavy update situation you're 
in.  That's actually why i asked for the exact 8.2 version.  You should 
plan an upgrade as soon as feasible to the current release just to 
eliminate this as a possible influence on your problems.  No need to dump 
the database or do anything fancy, just get the new version going and 
point it at the existing database.

To do a quick check on whether this is impacting things, run top, press 
"c" to show the full process lines, and note whether the statistics 
collector process is taking up a significant amount of CPU time.  If it 
is, you're being nailed by the bug, and you really need that ugprade.

> 8 Gigs of Ram
> shared_buffers = 75000   <--- Believe these need tuning (based on the
> reading last night)

Probably, but if you're having checkpoint problems now making 
shared_buffers bigger will likely make them worse.  Some people with 
update-heavy workloads end up reducing this to a very small value (<250MB) 
even with large amounts of RAM because that makes less information to dump 
at checkpoint time.

> checkpoint_segments = 50
> checkpoint_timeout = 300
> checkpoint_warning = 3600s              <--- set this last night and
> already see instances of
>
> "2008-01-24 03:54:39 PST    LOG:  checkpoints are occurring too
> frequently (89 seconds apart)
> 2008-01-24 03:54:39 PST    HINT:  Consider increasing the
> configuration parameter "checkpoint_segments"."

If you're getting checkpoints every 89 seconds it's no wonder your system 
is dying.  You may need to consider a large increase to 
checkpoint_segments to get the interval between checkpoints to increase. 
It should at least be a few minutes between them if you want any 
reasonable performance level.

> effective_cache_size = 330000  <-- This appears totally wrong and
> something I noticed last night.  left over from previous versions of
> postgres on different hardware. (thinking to set this to 6-7G)

Right, that's where it should be.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD