From: Jignesh K. Shah <J.K.Shah@Sun.COM>
To: Josh Berkus <josh@agliodbs.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-performance@postgresql.org, Alvaro Herrera <alvherre@commandprompt.com>
Subject: Re: User concurrency thresholding: where do I look?
Date: Fri, 20 Jul 2007 14:08:49 -0400
Message-ID: <46A0FA31.2080208@sun.com> (raw)
In-Reply-To: <200707192138.39679.josh@agliodbs.com>
References: <200707190828.59088.josh@agliodbs.com>
<200707191819.13899.josh@agliodbs.com>
<15685.1184895926@sss.pgh.pa.us>
<200707192138.39679.josh@agliodbs.com>
Awww Josh,
I was just enjoying the chat on the picket fence! :-)
Anyway the workload is mixed (reads,writes) with simple to medium
queries. The workload is known to scale well. But inorder to provide
substantial input I am still trying to eliminate things that can
bottleneck. Currently I have eliminated CPU (plenty free) , RAM
(memory is 48GB RAM in this server for a 32-bit postgresql instance),
IO Storage (used the free ram to do /tmp database to eliminate IO) and
am still trying to eliminate any network bottlenecks to say for sure we
have a problem in PostgreSQL. But yes till that final thing is confirmed
(network which can very well be the case) it could be a problem
somewhere else. However the thing that worries me is more of the big
drop instead of remaining constant out there..
Anyway more on this within a day or two once I add more network nics
between the systems to eliminate network problems (even though stats
dont show them as problems right now) and also reduce malloc lock
penalties if any.
As for other questions:
max_locks_per_transactions is set to default (10 I believe) increasing
it still seems to degrade overall throughput number.
max_connections is set to 1500 for now till I get decent scaling till
1400-1500 users.
There are no hard failures reported anywhere. Log min durations does
show that queries are now slowing down and taking longer.
OS is not swapping and also eliminated IO by putting the whole database
on /tmp
So while I finish adding more network connections between the two
systems (need to get cards) do enjoy the following URL :-)
http://www.spec.org/jAppServer2004/results/res2007q3/jAppServer2004-20070703-00073.html
Of course all postgresql.conf still remains from the old test so no
flames on that one again :-)
Regards,
Jignesh
Josh Berkus wrote:
> Tom,
>
>
>> Well, if the load is a lot of short writing transactions then you'd
>> expect the throughput to depend on how fast stuff can be pushed down to
>> WAL. What have you got wal_buffers set to? Are you using a commit
>> delay? What's the I/O system anyway (any BB write cache on the WAL
>> disk?) and what wal sync method are you using?
>>
>
> You know, I think Jignesh needs to me on this list so I can stop relaying
> questions on a workload I didn't design. Let me get him.
>
>
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: J.K.Shah@Sun.COM, josh@agliodbs.com, tgl@sss.pgh.pa.us, alvherre@commandprompt.com
Subject: Re: User concurrency thresholding: where do I look?
In-Reply-To: <46A0FA31.2080208@sun.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox