From: David Kerr <dmk@mr-paradox.net>
To: Greg Smith <greg@2ndquadrant.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-performance@postgresql.org
Subject: Re: pgbench could not send data to client: Broken pipe
Date: Wed, 8 Sep 2010 12:56:54 -0700
Message-ID: <20100908195654.GB49941@mr-paradox.net> (raw)
In-Reply-To: <4C87E3A6.8060800@2ndquadrant.com>
References: <20100908174629.GD40339@mr-paradox.net>
<8037.1283973026@sss.pgh.pa.us>
<4C87E3A6.8060800@2ndquadrant.com>
On Wed, Sep 08, 2010 at 03:27:34PM -0400, Greg Smith wrote:
- Tom Lane wrote:
- >As of the 9.0 release, it's possible to run pgbench in a "multi thread"
- >mode, and if you forced the subprocess rather than thread model it looks
- >like the select() limit would be per subprocess rather than global.
- >So I think you could get above the FD_SETSIZE limit with a bit of
- >hacking if you were using 9.0's pgbench. No chance with 8.3 though.
- >
-
- I believe David can do this easily enough by compiling a 9.0 source code
- tree with the "--disable-thread-safety" option. That's the simplest way
- to force the pgbench client to build itself using the multi-process
- model, rather than the multi-threaded one.
-
- It's kind of futile to run pgbench simulating much more than a hundred
- or two clients before 9.0 anyway. Without multiple workers, you're
- likely to just run into the process switching limitations within pgbench
- itself rather than testing server performance usefully. I've watched
- the older pgbench program fail to come close to saturating an 8 core
- server without running into its own limitations first.
-
- You might run a 9.0 pgbench client against an 8.3 server though, if you
- did the whole thing starting from pgbench database initialization over
- again--the built-in tables like "accounts" changed to "pgbench_accounts"
- in 8.4. That might work, can't recall any changes that would prevent
- it; but as I haven't tested it yet I can't say for sure.
Thanks, I compiled the 9.0 RC1 branch with the --disable-thread-safety option
and ran PG bench on my 8.3 DB it seemed to work fine,
However, MAXCLIENTS is still 1024, if i hack it to switch it up to 2048 i
get this:
starting vacuum...end.
select failed: Bad file descriptor <---------------
transaction type: TPC-B (sort of)
scaling factor: 1
query mode: simple
number of clients: 1900
number of threads: 1
number of transactions per client: 10
number of transactions actually processed: 3723/19000
tps = 52.007642 (including connections establishing)
tps = 82.579077 (excluding connections establishing)
I'm not sure what Tom is referring to with the select(2) limitation, maybe I'm running
into it (where do i find that? /usr/include/sys/select.h? )
should i be running pgbench differently? I tried increasing the # of threads
but that didn't increase the number of backend's and i'm trying to simulate
2000 physical backend processes.
thanks
Dave
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: dmk@mr-paradox.net, greg@2ndquadrant.com, tgl@sss.pgh.pa.us
Subject: Re: pgbench could not send data to client: Broken pipe
In-Reply-To: <20100908195654.GB49941@mr-paradox.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox