From: Scott Marlowe <scott.marlowe@gmail.com>
To: Joe Uhl <joeuhl@gmail.com>
Cc: Greg Smith <gsmith@gregsmith.com>
Cc: Gregory Stark <stark@enterprisedb.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: High CPU Utilization
Date: Fri, 20 Mar 2009 14:29:33 -0600
Message-ID: <dcc563d10903201329l7ea173bcg76a06f5cce082bdb@mail.gmail.com> (raw)
In-Reply-To: <9719B46B-6CF8-4710-BE2A-1DE8E1C0C05B@gmail.com>
References: <8609D1C0-3ECD-4A4F-B9C6-1FC8496EDC7A@gmail.com>
<200903161252.22202@hal.medialogik.com>
<5F9840B1-D080-4F3A-81D4-57E8F1DB0955@gmail.com>
<alpine.GSO.2.01.0903161621230.3428@westnet.com>
<87d4ch0z4z.fsf@oxford.xeocode.com>
<alpine.GSO.2.01.0903170017410.5259@westnet.com>
<9719B46B-6CF8-4710-BE2A-1DE8E1C0C05B@gmail.com>
On Fri, Mar 20, 2009 at 2:26 PM, Joe Uhl <joeuhl@gmail.com> wrote:
> On Mar 17, 2009, at 12:19 AM, Greg Smith wrote:
>
>> On Tue, 17 Mar 2009, Gregory Stark wrote:
>>
>>> Hm, well the tests I ran for posix_fadvise were actually on a Perc5 --
>>> though
>>> who knows if it was the same under the hood -- and I saw better
>>> performance
>>> than this. I saw about 4MB/s for a single drive and up to about 35MB/s
>>> for 15
>>> drives. However this was using linux md raid-0, not hardware raid.
>>
>> Right, it's the hardware RAID on the Perc5 I think people mainly complain
>> about. If you use it in JBOD mode and let the higher performance CPU in
>> your main system drive the RAID functions it's not so bad.
>>
>> --
>> * Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD
>
> I have not yet had a chance to try software raid on the standby server
> (still planning to) but wanted to follow up to see if there was any good way
> to figure out what the postgresql processes are spending their CPU time on.
>
> We are under peak load right now, and I have Zabbix plotting CPU utilization
> and CPU wait (from vmstat output) along with all sorts of other vitals on
> charts. CPU utilization is a sustained 90% - 95% and CPU Wait is hanging
> below 10%. Since being pointed at vmstat by this list I have been watching
> CPU Wait and it does get high at times (hence still wanting to try Perc5 in
> JBOD) but then there are sustained periods, right now included, where our
> CPUs are just getting crushed while wait and IO (only doing about 1.5 MB/sec
> right now) are very low.
>
> This high CPU utilization only occurs when under peak load and when our JDBC
> pools are fully loaded. We are moving more things into our cache and
> constantly tuning indexes/tables but just want to see if there is some
> underlying cause that is killing us.
>
> Any recommendations for figuring out what our database is spending its CPU
> time on?
What does the cs entry on vmstat say at this time? If you're cs is
skyrocketing then you're getting a context switch storm, which is
usually a sign that there are just too many things going on at once /
you've got an old kernel things like that.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: scott.marlowe@gmail.com, joeuhl@gmail.com, gsmith@gregsmith.com, stark@enterprisedb.com
Subject: Re: High CPU Utilization
In-Reply-To: <dcc563d10903201329l7ea173bcg76a06f5cce082bdb@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox