Date: Fri, 20 Jul 2007 14:18:05 -0400
From: "Jignesh K. Shah" <J.K.Shah@Sun.COM>
Subject: Re: User concurrency thresholding: where do I look?
In-reply-to: <46A0FA31.2080208@sun.com>
To: "Jignesh K. Shah" <J.K.Shah@Sun.COM>
Cc: Josh Berkus <josh@agliodbs.com>, Tom Lane <tgl@sss.pgh.pa.us>,
	pgsql-performance@postgresql.org,
	Alvaro Herrera <alvherre@commandprompt.com>
Message-id: <46A0FC5D.40001@sun.com>
Organization: Sun Microsystems
MIME-version: 1.0
Content-type: text/plain; format=flowed; charset=UTF-8
Content-transfer-encoding: 7BIT
References: <200707190828.59088.josh@agliodbs.com>
	<200707191819.13899.josh@agliodbs.com> <15685.1184895926@sss.pgh.pa.us>
	<200707192138.39679.josh@agliodbs.com> <46A0FA31.2080208@sun.com>
User-Agent: Thunderbird 1.5.0.8 (X11/20061110)

I forgot to add one more piece of information.. I also tried the same 
test with 64-bit postgresql with 6GB shared_buffers and results are the 
same it drops around the same point which to me sounds like a bottleneck..

More later

-Jignesh


Jignesh K. Shah wrote:
> Awww Josh,
>
> I was just enjoying the chat on the picket fence! :-)
>
> Anyway the workload is mixed (reads,writes) with simple to medium 
> queries. The workload is known to scale well. But inorder to provide 
> substantial input I am still trying to eliminate things that can 
> bottleneck.  Currently I have eliminated CPU (plenty free) , RAM 
> (memory  is 48GB RAM in this server for  a 32-bit postgresql 
> instance), IO Storage (used the free ram to do /tmp database to 
> eliminate IO)  and am still trying to eliminate any network 
> bottlenecks to say for sure we have a problem in PostgreSQL. But yes 
> till that final thing is confirmed (network which can very well be the 
> case) it could be a problem somewhere else. However the thing that 
> worries me is more of the big drop instead of remaining constant out 
> there..
>
> Anyway more on this within a day or two once I add more network nics 
> between the systems to eliminate network problems (even though stats 
> dont show them as problems right now) and also reduce malloc lock 
> penalties if any.
>
> As for other questions:
>
> max_locks_per_transactions is set to default (10 I believe) increasing 
> it still seems to degrade overall throughput number.
>
> max_connections is set to 1500 for now till I get decent scaling till 
> 1400-1500 users.
>
> There are no hard failures reported anywhere. Log min durations does 
> show that queries are now slowing down and taking longer.
>
> OS is not swapping and also eliminated IO  by putting the whole 
> database on /tmp
>
> So while I finish adding more network connections between the two 
> systems (need to get cards) do enjoy the following URL :-)
>
> http://www.spec.org/jAppServer2004/results/res2007q3/jAppServer2004-20070703-00073.html 
>
>
> Of course all postgresql.conf still remains from the old test so no 
> flames on that one again :-)
>
> Regards,
> Jignesh
>
>
>
>
> Josh Berkus wrote:
>> Tom,
>>
>>  
>>> Well, if the load is a lot of short writing transactions then you'd
>>> expect the throughput to depend on how fast stuff can be pushed down to
>>> WAL.  What have you got wal_buffers set to?  Are you using a commit
>>> delay?  What's the I/O system anyway (any BB write cache on the WAL
>>> disk?) and what wal sync method are you using?
>>>     
>>
>> You know, I think Jignesh needs to me on this list so I can stop 
>> relaying questions on a workload I didn't design.  Let me get him.
>>
>>   
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
>               http://archives.postgresql.org