Received: from localhost (unknown [200.46.208.211]) by mail.postgresql.org (Postfix) with ESMTP id 56DE663295F for ; Thu, 12 Mar 2009 19:57:20 -0300 (ADT) Received: from mail.postgresql.org ([200.46.204.86]) by localhost (mx1.hub.org [200.46.208.211]) (amavisd-maia, port 10024) with ESMTP id 65936-02 for ; Thu, 12 Mar 2009 19:57:02 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from EXHUB018-3.exch018.msoutlookonline.net (exhub018-3.exch018.msoutlookonline.net [64.78.17.18]) by mail.postgresql.org (Postfix) with ESMTP id 6DC1C62F435 for ; Thu, 12 Mar 2009 19:57:08 -0300 (ADT) Received: from EXVMBX018-1.exch018.msoutlookonline.net ([64.78.17.47]) by EXHUB018-3.exch018.msoutlookonline.net ([64.78.17.18]) with mapi; Thu, 12 Mar 2009 15:57:06 -0700 From: Scott Carey To: "Jignesh K. Shah" CC: Tom Lane , Kevin Grittner , "pgsql-performance@postgresql.org" Date: Thu, 12 Mar 2009 15:57:05 -0700 Subject: Re: Proposal of tunable fix for scalability of 8.4 Thread-Topic: [PERFORM] Proposal of tunable fix for scalability of 8.4 Thread-Index: AcmjQip4cjYw6yTmR+at0IpwQNJjngAI7Jz7 Message-ID: In-Reply-To: <49B9566C.3010708@sun.com> Accept-Language: en-US Content-Language: en X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_C5DEE15133BAscottrichrelevancecom_" MIME-Version: 1.0 X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=0.001 tagged_above=0 required=5 tests=AWL=-0.000, HTML_MESSAGE=0.001 X-Spam-Level: X-Archive-Number: 200903/145 X-Sequence-Number: 33067 --_000_C5DEE15133BAscottrichrelevancecom_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable On 3/12/09 11:37 AM, "Jignesh K. Shah" wrote: And again this is the third time I am saying.. the test users also have som= e latency build up in them which is what generally is exploited to get more= users than number of CPUS on the system but that's the point we want to ex= ploit.. Otherwise if all new users begin to do their job with no latency th= en we would need 6+ billion cpus to handle all possible users. Typically as= an administrator (System and database) I can only tweak/control latencies = within my domain, that is network, disk, cpu's etc and those are what I am = tweaking and coming to a *Configured* environment and now trying to improve= lock contentions/waits in PostgreSQL so that we have an optimized setup. In general, I suggest that it is useful to run tests with a few different t= ypes of pacing. Zero delay pacing will not have realistic number of connec= tions, but will expose bottlenecks that are universal, and less controversi= al. Small latency (100ms to 1s) tests are easy to make from the zero dela= y ones, and help expose problems with connection count or other forms of 'n= on-active' concurrency. End-user realistic delays are app specific, and us= eful with larger holistic load tests (say, through the application interfac= e). Generally, running them in this order helps because at each stage you = are adding complexity. Based on your explanations, you've probably done mu= ch of this so far and your approach sounds solid to me. If the first case fails (zero delay, smaller user count), there is no way t= he others will pass. I am trying another run where I limit the waked up threads to a pre-configu= red number to see how various numbers pans out in terms of throughput on th= is server. Regards, Jignesh This would be good, as would waking up only the shared locks, but refining = the test somewhat to be maximally convincing would help. The first thing t= o show is either a test with very small or no sleep delay, or with a connec= tion pooler in between. I prefer the former since it is the most simple. = This will be a test that is less entangled with the connection count and s= hould peak at a lot closer to the CPU core count and be more convincing to = some. I'm positive it won't change the basic trend (ramp up and plateau, = with a higher plateau with the changed lock code) but others seem unconvinc= ed and I'm a nobody anyway. --_000_C5DEE15133BAscottrichrelevancecom_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Re: [PERFORM] Proposal of tunable fix for scalability of 8.4 On 3/12/09 11:37 AM, "Jignesh K. Shah" <J.K.Shah@Sun.COM> wrote:


And again this is the third time I am saying.. the test users also have som= e latency build up in them which is what generally is exploited to get more= users than number of CPUS on the system but that's the point we want to ex= ploit.. Otherwise if all new users begin to do their job with no latency th= en we would need 6+ billion cpus to handle all possible users. Typically as= an administrator (System and database) I can only tweak/control latencies = within my domain, that is network, disk, cpu's etc and those are what I am = tweaking and coming to a *Configured* environment and now trying to improve= lock contentions/waits in PostgreSQL so that we have an optimized setup.
In general, I suggest that it is useful to= run tests with a few different types of pacing.  Zero delay pacing wi= ll not have realistic number of connections, but will expose bottlenecks th= at are universal, and less controversial.   Small latency (100ms = to 1s) tests are easy to make from the zero delay ones, and help expose pro= blems with connection count or other forms of ‘non-active’ conc= urrency.  End-user realistic delays are app specific, and useful with = larger holistic load tests (say, through the application interface).  = Generally, running them in this order helps because at each stage you are a= dding complexity.  Based on your explanations, you’ve probably d= one much of this so far and your approach sounds solid to me.  
If the first case fails (zero delay, smaller user count), there is no way t= he others will pass.

I am trying another run where I limit the waked up threads to a pre-configu= red number to see how various numbers pans out in terms of throughput on th= is server.

Regards,
Jignesh