From: Jignesh K. Shah <J.K.Shah@Sun.COM>
To: Jignesh K. Shah <J.K.Shah@Sun.COM>
Cc: Josh Berkus <josh@agliodbs.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-performance@postgresql.org, Alvaro Herrera <alvherre@commandprompt.com>
Subject: Re: User concurrency thresholding: where do I look?
Date: Fri, 20 Jul 2007 14:18:05 -0400
Message-ID: <46A0FC5D.40001@sun.com> (raw)
In-Reply-To: <46A0FA31.2080208@sun.com>
References: <200707190828.59088.josh@agliodbs.com>
<200707191819.13899.josh@agliodbs.com>
<15685.1184895926@sss.pgh.pa.us>
<200707192138.39679.josh@agliodbs.com>
<46A0FA31.2080208@sun.com>
I forgot to add one more piece of information.. I also tried the same
test with 64-bit postgresql with 6GB shared_buffers and results are the
same it drops around the same point which to me sounds like a bottleneck..
More later
-Jignesh
Jignesh K. Shah wrote:
> Awww Josh,
>
> I was just enjoying the chat on the picket fence! :-)
>
> Anyway the workload is mixed (reads,writes) with simple to medium
> queries. The workload is known to scale well. But inorder to provide
> substantial input I am still trying to eliminate things that can
> bottleneck. Currently I have eliminated CPU (plenty free) , RAM
> (memory is 48GB RAM in this server for a 32-bit postgresql
> instance), IO Storage (used the free ram to do /tmp database to
> eliminate IO) and am still trying to eliminate any network
> bottlenecks to say for sure we have a problem in PostgreSQL. But yes
> till that final thing is confirmed (network which can very well be the
> case) it could be a problem somewhere else. However the thing that
> worries me is more of the big drop instead of remaining constant out
> there..
>
> Anyway more on this within a day or two once I add more network nics
> between the systems to eliminate network problems (even though stats
> dont show them as problems right now) and also reduce malloc lock
> penalties if any.
>
> As for other questions:
>
> max_locks_per_transactions is set to default (10 I believe) increasing
> it still seems to degrade overall throughput number.
>
> max_connections is set to 1500 for now till I get decent scaling till
> 1400-1500 users.
>
> There are no hard failures reported anywhere. Log min durations does
> show that queries are now slowing down and taking longer.
>
> OS is not swapping and also eliminated IO by putting the whole
> database on /tmp
>
> So while I finish adding more network connections between the two
> systems (need to get cards) do enjoy the following URL :-)
>
> http://www.spec.org/jAppServer2004/results/res2007q3/jAppServer2004-20070703-00073.html
>
>
> Of course all postgresql.conf still remains from the old test so no
> flames on that one again :-)
>
> Regards,
> Jignesh
>
>
>
>
> Josh Berkus wrote:
>> Tom,
>>
>>
>>> Well, if the load is a lot of short writing transactions then you'd
>>> expect the throughput to depend on how fast stuff can be pushed down to
>>> WAL. What have you got wal_buffers set to? Are you using a commit
>>> delay? What's the I/O system anyway (any BB write cache on the WAL
>>> disk?) and what wal sync method are you using?
>>>
>>
>> You know, I think Jignesh needs to me on this list so I can stop
>> relaying questions on a workload I didn't design. Let me get him.
>>
>>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
> http://archives.postgresql.org
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: J.K.Shah@Sun.COM, josh@agliodbs.com, tgl@sss.pgh.pa.us, alvherre@commandprompt.com
Subject: Re: User concurrency thresholding: where do I look?
In-Reply-To: <46A0FC5D.40001@sun.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox