To: Craig James <craig_james@emolecules.com>
cc: pgsql-performance@postgresql.org
Subject: Re: Occasional giant spikes in CPU load 
In-reply-to: <4BBCFB12.1010306@emolecules.com> 
References: <k2pda0294261004061437p9cd3810dqe9ce528b6ddb0873@mail.gmail.com>
	<m2m603c8f071004071413w1f2d2497hbbe07d9c6491848f@mail.gmail.com>
	<4BBCFB12.1010306@emolecules.com>
Comments: In-reply-to Craig James <craig_james@emolecules.com>
	message dated "Wed, 07 Apr 2010 14:37:22 -0700"
Date: Wed, 07 Apr 2010 17:59:08 -0400
Message-ID: <8127.1270677548@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Craig James <craig_james@emolecules.com> writes:
> Most of the time Postgres runs nicely, but two or three times a day we get a huge spike in the CPU load that lasts just a short time -- it jumps to 10-20 CPU loads.  Today it hit 100 CPU loads.  Sometimes days go by with no spike events.  During these spikes, the system is completely unresponsive (you can't even login via ssh).
> I managed to capture one such event using top(1) with the "batch" option as a background process.  See output below - it shows 19 active postgress processes, but I think it missed the bulk of the spike.

> Any ideas where I can look to find what's triggering this?

> Postgres 8.3.0
           ^^^^^

If it's really 8.3.0, try updating to 8.3.something-recent.  We've fixed
a whole lot of bugs since then.

I have a suspicion that this might be an sinval overrun scenario, in
which case you'd need to update to 8.4 to get a real fix.  But updating
in the 8.3 branch would be cheap and easy.

If it is sinval overrun, it would presumably be triggered by a whole lot
of catalog changes being made at approximately the same time.  Can you
correlate the spikes with anything like that?

			regards, tom lane