To: pgsql-performance@postgresql.org
From: Ivan Voras <ivoras@freebsd.org>
Subject: Re: performance config help
Date: Mon, 11 Jan 2010 18:12:59 +0100
Lines: 68
Message-ID: <hifm6q$dp8$1@ger.gmane.org>
References: <61039b861001110544o2c352763q13dcf9d78a156714@mail.gmail.com>	
	<dcc563d11001110550s2af6d559l5db495d5e0fc6a57@mail.gmail.com>
	<61039b861001110842g15bb1f26jc50b6e6762e01427@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
User-Agent: Thunderbird 2.0.0.23 (X11/20091210)
In-Reply-To: <61039b861001110842g15bb1f26jc50b6e6762e01427@mail.gmail.com>

Bob Dusek wrote:
> On Mon, Jan 11, 2010 at 8:50 AM, Scott Marlowe <scott.marlowe@gmail.com 
> <mailto:scott.marlowe@gmail.com>> wrote:
> 
>     On Mon, Jan 11, 2010 at 6:44 AM, Bob Dusek <redusek@gmail.com
>     <mailto:redusek@gmail.com>> wrote:
>      > Hello,
>      >
>      > We're running Postgres 8.4.2 on Red Hat 5, on pretty hefty
>     hardware...
>      >
>      > 4X E7420 Xeon, Four cores (for a total of 16 cores)
>      > 2.13 GHz, 8M Cache, 1066 Mhz FSB
>      > 32 Gigs of RAM
>      > 15 K RPM drives in striped raid
> 
>     What method of striped RAID?  RAID-5?  RAID-10?  RAID-4?  RAID-0?
> 
> 
> RAID-0

And how many drives?

> 
>      > Things run fine, but when we get a lot of concurrent queries
>     running, we see
>      > a pretty good slow down.
> 
>     Definte "a lot".
> 
> 
> We have an application server that is processing requests.  Each request 
> consists of a combination of selects, inserts, and deletes.  We actually 
> see degredation when we get more than 40 concurrent requests.  The exact 
> number of queries executed by each request isn't known.  It varies per 
> request.  But, an example request would be about 16 inserts and 113 
> selects.  Any given request can't execute more than a single query at a 
> time.  

So, you are concurrently trying to achieve more than around 640 writes 
and 4520 reads (worst case figures...). This should be interesting. For 
40 concurrent requests you will probably need at least 4 drives in 
RAID-0 to sustain the write rates (and I'll guess 5-6 to be sure to 
cover read requests also, together with plenty of RAM).

>  avg-cpu:  %user   %nice %system %iowait  %steal   %idle
>           34.29    0.00    7.09    0.03    0.00   58.58
> 
> Device:         rrqm/s   wrqm/s   r/s   w/s   rsec/s   wsec/s avgrq-sz 
> avgqu-sz   await  svctm  %util
> sda               0.00   112.20  0.00 133.40     0.00  1964.80    
> 14.73     0.42    3.17   0.04   0.48
> 
> The iowait seems pretty low, doesn't it?

Yes, but you are issuing 133 write operations per seconds per drive(s) - 
this is nearly the limit of what you can get with 15k RPM drives 
(actually, the limit should be somewhere around 200..250 IOPS but 133 
isn't that far).

> top - 10:11:43 up 12 days, 20:48,  6 users,  load average: 10.48, 4.16, 2.83
> Tasks: 798 total,   8 running, 790 sleeping,   0 stopped,   0 zombie
> Cpu0  : 33.3%us,  7.6%sy,  0.0%ni, 59.1%id,  0.0%wa,  0.0%hi,  0.0%si,  
> 0.0%st

There is one other possibility - since the CPUs are not very loaded, are 
you sure the client application with which you are testing is fast 
enough to issue enough request to saturate the database?