From: Joshua D. Drake <jd@commandprompt.com>
To: Craig James <craig_james@emolecules.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: Occasional giant spikes in CPU load
Date: Wed, 07 Apr 2010 15:36:48 -0700
Message-ID: <1270679808.19131.28.camel@jd-desktop.unknown.charter.com> (raw)
In-Reply-To: <4BBCFCDE.5020405@emolecules.com>
References: <k2pda0294261004061437p9cd3810dqe9ce528b6ddb0873@mail.gmail.com>
<m2m603c8f071004071413w1f2d2497hbbe07d9c6491848f@mail.gmail.com>
<4BBCFB12.1010306@emolecules.com>
<1270676410.19131.21.camel@jd-desktop.unknown.charter.com>
<4BBCFCDE.5020405@emolecules.com>
On Wed, 2010-04-07 at 14:45 -0700, Craig James wrote:
> On 4/7/10 2:40 PM, Joshua D. Drake wrote:
> > On Wed, 2010-04-07 at 14:37 -0700, Craig James wrote:
> >> Most of the time Postgres runs nicely, but two or three times a day we get a huge spike in the CPU load that lasts just a short time -- it jumps to 10-20 CPU loads. Today it hit 100 CPU loads. Sometimes days go by with no spike events. During these spikes, the system is completely unresponsive (you can't even login via ssh).
> >>
> >> I managed to capture one such event using top(1) with the "batch" option as a background process. See output below - it shows 19 active postgress processes, but I think it missed the bulk of the spike.
> >
> > What does iostat 5 say during the jump?
>
> It's very hard to say ... I'll have to start a background job to watch for a day or so. While it's happening, you can't login, and any open windows become unresponsive. I'll probably have to run it at high priority using nice(1) to get any data at all during the event.
Do you have sar runing? Say a sar -A ?
>
> Would vmstat be informative?
Yes.
My guess is that it is not CPU, it is IO and your CPU usage is all WAIT
on IO.
To have your CPUs so flooded that they are the cause of an inability to
log in is pretty suspect.
Joshua D. Drake
>
> Thanks,
> Craig
>
--
PostgreSQL.org Major Contributor
Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564
Consulting, Training, Support, Custom Development, Engineering
Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: jd@commandprompt.com, craig_james@emolecules.com
Subject: Re: Occasional giant spikes in CPU load
In-Reply-To: <1270679808.19131.28.camel@jd-desktop.unknown.charter.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox