Date: Wed, 8 Sep 2010 14:20:56 -0700
From: David Kerr <dmk@mr-paradox.net>
To: Kevin Grittner <Kevin.Grittner@wicourts.gov>
Cc: Greg Smith <greg@2ndquadrant.com>, pgsql-performance@postgresql.org,
	Tom Lane <tgl@sss.pgh.pa.us>
Subject: Re: pgbench could not send data to client: Broken pipe
Message-ID: <20100908212056.GI49941@mr-paradox.net>
References: <20100908174629.GD40339@mr-paradox.net>
	<8037.1283973026@sss.pgh.pa.us> <4C87E3A6.8060800@2ndquadrant.com>
	<20100908195654.GB49941@mr-paradox.net>
	<9771.1283978128@sss.pgh.pa.us>
	<20100908204443.GG49941@mr-paradox.net>
	<4C87B22802000025000353B7@gw.wicourts.gov>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <4C87B22802000025000353B7@gw.wicourts.gov>
User-Agent: Mutt/1.4.2.3i

On Wed, Sep 08, 2010 at 03:56:24PM -0500, Kevin Grittner wrote:
- David Kerr <dmk@mr-paradox.net> wrote:
-  
- > Actually, this is real.. that's 2000 connections - connection
- > pooled out to 20k or so. (although i'm pushing for closer to 1000
- > connections).
- > 
- > I know that's not the ideal way to go, but it's what i've got to
- > work with.
- > 
- > It IS a huge box though...
-  
- FWIW, my benchmarks (and I've had a couple people tell me this is
- consistent with what they've seen) show best throughput and best
- response time when the connection pool is sized such that the number
- of active PostgreSQL connections is limited to about twice the
- number of CPU cores plus the number of effective spindles.  Either
- you've got one heck of a machine, or your "sweet spot" for the
- connection pool will be well under 1000 connections.
-  
- It is important that your connection pool queues requests when
- things are maxed out, and quickly submit a new request when
- completion brings the number of busy connections below the maximum.
-  
- -Kevin

Hmm, i'm not following you. I've got 48 cores. that means my sweet-spot
active connections would be 96. (i.e., less than the default max_connections
shipped with PG) and this is a very very expensive machine.

Now if i were to connection pool that out to 15 people per connection, 
that's 1440 users "total" able to use my app at one time. (with only 
96 actually doing anything). not really great for a web-based app that 
will have millions of users accessing it when we're fully ramped up.

I've got a few plans to spread the load out across multiple machines
but at 1440 users per machine this wouldn't be sustanable.. 

I know that other people are hosting more than that on larger machines
so i hope i'm ok.

Thanks

Dave