From: Jignesh K. Shah <J.K.Shah@Sun.COM>
To: Simon Riggs <simon@2ndquadrant.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Josh Berkus <josh@agliodbs.com>
Cc: pgsql-performance@postgresql.org, Alvaro Herrera <alvherre@commandprompt.com>
Subject: Re: User concurrency thresholding: where do I look?
Date: Thu, 26 Jul 2007 17:17:55 -0400
Message-ID: <46A90F83.8050401@sun.com> (raw)
In-Reply-To: <1185480291.5510.22.camel@ebony.site>
References: <200707190828.59088.josh@agliodbs.com>
<200707191819.13899.josh@agliodbs.com>
<15685.1184895926@sss.pgh.pa.us>
<200707192138.39679.josh@agliodbs.com>
<46A0FA31.2080208@sun.com>
<12002.1184955831@sss.pgh.pa.us>
<46A1055F.9010304@sun.com>
<12445.1184958793@sss.pgh.pa.us>
<46A11116.5030905@sun.com>
<14909.1184962561@sss.pgh.pa.us>
<46A1205B.8010205@sun.com>
<15343.1184965054@sss.pgh.pa.us>
<1185180668.4284.149.camel@ebony.site>
<23848.1185199901@sss.pgh.pa.us>
<1185201556.4284.243.camel@ebony.site>
<25095.1185202498@sss.pgh.pa.us>
<1185205663.4284.249.camel@ebony.site>
<1185212261.4284.270.camel@ebony.site>
<1545.1185214760@sss.pgh.pa.us>
<46A7D5AD.9000706@sun.com>
<13162.1185405865@sss.pgh.pa.us>
<46A8AFDF.3050804@sun.com>
<26825.1185460816@sss.pgh.pa.us>
<46A8BD62.2080904@sun.com>
<1185465076.4190.87.camel@ebony.site>
<46A8F9AA.4040507@sun.com>
<1185480291.5510.22.camel@ebony.site>
Will try 16 and 32 CLOGBUFFER tomorrow:
But here is locks data again with about increased time profiling (about
2 minutes) for the connection with about 2000 users:
bash-3.00# time ./4_lwlock_waits.d 13583
^C
Lock Id Mode Count
ProcArrayLock Shared 5
XidGenLock Exclusive 13
CLogControlLock Shared 14
XidGenLock Shared 15
CLogControlLock Exclusive 21
WALInsertLock Exclusive 77
WALWriteLock Exclusive 175
ProcArrayLock Exclusive 275
Lock Id Combined Time (ns)
XidGenLock 194966200
WALInsertLock 517955000
CLogControlLock 679665100
WALWriteLock 2838716200
ProcArrayLock 44181002600
Top Wait time seems to come from the following code path for
ProcArrayLock:
Lock Id Mode Count
ProcArrayLock Exclusive 21
Lock Id Combined Time (ns)
ProcArrayLock 5255937500
Lock Id Combined Time (ns)
postgres`LWLockAcquire+0x1f0
postgres`CommitTransaction+0x104
postgres`CommitTransactionCommand+0xbc
postgres`finish_xact_command+0x78
postgres`exec_execute_message+0x42c
postgres`PostgresMain+0x1838
postgres`BackendRun+0x2f8
postgres`ServerLoop+0x680
postgres`PostmasterMain+0xda8
postgres`main+0x3d0
postgres`_start+0x17c
5255937500
Regards,
Jignesh
Simon Riggs wrote:
> On Thu, 2007-07-26 at 15:44 -0400, Jignesh K. Shah wrote:
>
>> BEAUTIFUL!!!
>>
>> Using the Patch I can now go upto 1300 users without dropping.. But now
>> it still repeats at 1300-1350 users..
>>
>
> OK, can you try again with 16 and 32 buffers please? We need to know
> how many is enough and whether this number needs to be variable via a
> parameter, or just slightly larger by default. Thanks.
>
>
>> I corrected the Lock Descriptions based on what I got from lwlock.h and
>> retried the whole scalability again with profiling again.. This time it
>> looks like the ProcArrayLock
>>
>
> That's what I would expect with that many users.
>
>
>> Lock Id Mode Count
>> XidGenLock Exclusive 1
>> CLogControlLock Shared 2
>> XidGenLock Shared 2
>> WALWriteLock Exclusive 4
>> WALInsertLock Exclusive 8
>> CLogControlLock Exclusive 9
>> ProcArrayLock Exclusive 9
>>
>
> ...but as Tom mentioned, we need to do longer runs now so these counts
> get to somewhere in the hundreds so we have some statistical validity.
>
>
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: J.K.Shah@Sun.COM, simon@2ndquadrant.com, tgl@sss.pgh.pa.us, josh@agliodbs.com, alvherre@commandprompt.com
Subject: Re: User concurrency thresholding: where do I look?
In-Reply-To: <46A90F83.8050401@sun.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox