pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
High %SYS CPU usage
4+ messages / 2 participants
[nested] [flat]

* High %SYS CPU usage
@ 2012-10-30 12:54  Cesar Martin <cmartinp@gmail.com>
  0 siblings, 1 reply; 4+ messages in thread

From: Cesar Martin @ 2012-10-30 12:54 UTC (permalink / raw)
  To: pgsql-performance

Hello there,

I have PostgreSQL 8.3.18 server running on Centos 6.2 (2.6.32-220.7.1) with
this specs:

2x CPU AMD Opteron 6282
128GB RAM
Raid 10 (12HD 15k rpm 1GB cache) with data
Raid 10 (4HD 15k rpm 1GB cache) with xlog
Raid 1 (15k rpm 1GB cache shared with xlog) with system

On this server I have only one database with 312GB of data. The database
had run fine during 4 months, but from two months ago, during high work
load periods, the server is collapsed by "%sys" type load.

For example "dstat -ar --socket --tcp" during %sys load problem:
http://pastebin.com/7zfDNvPh

Reboot the server mitigates the problem during few days, but always
reappear.
Server not is swapping, don't have excessive I/O, don't have %IRQ load.

I don't have any ideas...

Thank you very much for your help.

My sysctl and postgres.conf:

sysclt -a:
http://pastebin.com/EEVnNxsZ

My Postgres.conf:
max_connections = 500 # (change requires restart)
unix_socket_directory = '/var/run/postgres' # (change requires restart)
shared_buffers = 18GB # min 128kB or max_connections*16kB
work_mem = 30MB # min 64kB
maintenance_work_mem = 1GB # min 1MB
max_fsm_pages = 8553600 # min max_fsm_relations*16, 6 bytes each
max_fsm_relations = 409000 # min 100, ~70 bytes each
fsync = on # turns forced synchronization on or off
synchronous_commit = off # immediate fsync at commit
wal_buffers = 8MB # min 32kB
checkpoint_segments = 64 # in logfile segments, min 1, 16MB each
checkpoint_completion_target = 0.9 # checkpoint target duration, 0.0 - 1.0
archive_mode = on # allows archiving to be done
archive_command = 'exit 0'
effective_cache_size = 100GB
constraint_exclusion = on
default_text_search_config = 'pg_catalog.spanish'
max_locks_per_transaction = 100

-- 
César Martín Pérez
cmartinp@gmail.com

^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* Re: High %SYS CPU usage
@ 2012-10-30 19:07  Josh Berkus <josh@agliodbs.com>
  parent: Cesar Martin <cmartinp@gmail.com>
  0 siblings, 1 reply; 4+ messages in thread

From: Josh Berkus @ 2012-10-30 19:07 UTC (permalink / raw)
  To: pgsql-performance

Cesar,

> On this server I have only one database with 312GB of data. The database
> had run fine during 4 months, but from two months ago, during high work
> load periods, the server is collapsed by "%sys" type load.

Hmmm.  Have you updated Linux any time recently?  I'm wondering if this
is a PostgreSQL problem at all.  It sounds like an OS issue.

Can you give us the results of mpstat -P ALL 3 ?  The dstat output
doesn't tell me much.

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com




^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* Re: High %SYS CPU usage
@ 2012-11-02 07:47  Cesar Martin <cmartinp@gmail.com>
  parent: Josh Berkus <josh@agliodbs.com>
  0 siblings, 1 reply; 4+ messages in thread

From: Cesar Martin @ 2012-11-02 07:47 UTC (permalink / raw)
  To: Josh Berkus <josh@agliodbs.com>; +Cc: pgsql-performance

Hi Josh,

Today is not the worse day for this issue, because Fridays DB have little
workload... But I have this output:

http://pastebin.com/bKg8tfKC

I don't have updated OS server from the install.
I also begin to think that the problem is OS misconfiguration (Kernel
parameters??) or Hardware problem...

Thanks for your help.


2012/10/30 Josh Berkus <josh@agliodbs.com>

> Cesar,
>
> > On this server I have only one database with 312GB of data. The database
> > had run fine during 4 months, but from two months ago, during high work
> > load periods, the server is collapsed by "%sys" type load.
>
> Hmmm.  Have you updated Linux any time recently?  I'm wondering if this
> is a PostgreSQL problem at all.  It sounds like an OS issue.
>
> Can you give us the results of mpstat -P ALL 3 ?  The dstat output
> doesn't tell me much.
>
> --
> Josh Berkus
> PostgreSQL Experts Inc.
> http://pgexperts.com
>
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
>



-- 
César Martín Pérez
cmartinp@gmail.com

^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* Re: High %SYS CPU usage
@ 2012-12-31 13:08  Cesar Martin <cmartinp@gmail.com>
  parent: Cesar Martin <cmartinp@gmail.com>
  0 siblings, 0 replies; 4+ messages in thread

From: Cesar Martin @ 2012-12-31 13:08 UTC (permalink / raw)
  To: Josh Berkus <josh@agliodbs.com>; +Cc: pgsql-performance

Finally I resolv the problem
setting /sys/kernel/mm/redhat_transparent_hugepage/enabled to "never"


2012/11/2 Cesar Martin <cmartinp@gmail.com>

> Hi Josh,
>
> Today is not the worse day for this issue, because Fridays DB have little
> workload... But I have this output:
>
> http://pastebin.com/bKg8tfKC
>
> I don't have updated OS server from the install.
> I also begin to think that the problem is OS misconfiguration (Kernel
> parameters??) or Hardware problem...
>
> Thanks for your help.
>
>
> 2012/10/30 Josh Berkus <josh@agliodbs.com>
>
>> Cesar,
>>
>> > On this server I have only one database with 312GB of data. The database
>> > had run fine during 4 months, but from two months ago, during high work
>> > load periods, the server is collapsed by "%sys" type load.
>>
>> Hmmm.  Have you updated Linux any time recently?  I'm wondering if this
>> is a PostgreSQL problem at all.  It sounds like an OS issue.
>>
>> Can you give us the results of mpstat -P ALL 3 ?  The dstat output
>> doesn't tell me much.
>>
>> --
>> Josh Berkus
>> PostgreSQL Experts Inc.
>> http://pgexperts.com
>>
>>
>> --
>> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org
>> )
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgsql-performance
>>
>
>
>
> --
> César Martín Pérez
> cmartinp@gmail.com
>
>


-- 
César Martín Pérez
cmartinp@gmail.com

^ permalink  raw  reply  [nested|flat] 4+ messages in thread


end of thread, other threads:[~2012-12-31 13:08 UTC | newest]

Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2012-10-30 12:54 High %SYS CPU usage Cesar Martin <cmartinp@gmail.com>
2012-10-30 19:07 ` Josh Berkus <josh@agliodbs.com>
2012-11-02 07:47   ` Cesar Martin <cmartinp@gmail.com>
2012-12-31 13:08     ` Cesar Martin <cmartinp@gmail.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox