pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Fabien COELHO <coelho@cri.ensmp.fr>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Andres Freund <andres@anarazel.de>
Cc: PostgreSQL Bugs <pgsql-bugs@lists.postgresql.org>
Cc: David Rowley <dgrowleyml@gmail.com>
Cc: Jawarilal, Manish <Manish.Jawarilal@dell.com>
Subject: Re: pgbench bug / limitation
Date: Tue, 9 Jun 2020 09:45:58 +0200 (CEST)
Message-ID: <alpine.DEB.2.22.394.2006090941480.274691@pseudo> (raw)
In-Reply-To: <alpine.DEB.2.22.394.2006061819450.32228@pseudo>
References: <BL0PR1901MB198583EAC6C8A3B770D20F1AED8A0@BL0PR1901MB1985.namprd19.prod.outlook.com>
	<384920.1591027137@sss.pgh.pa.us>
	<CAApHDvrjWtJa9oQ98oH+9PCvTgOkGZP+gcmottNX95sjn6o9fQ@mail.gmail.com>
	<512846.1591057287@sss.pgh.pa.us>
	<CAApHDvpMEAaZMxtzixbnb7ySo=Kb1eUfshEB-y4YOm8SgGrpiw@mail.gmail.com>
	<alpine.DEB.2.22.394.2006021258170.87735@pseudo>
	<CAApHDvpDBkjj6a+Ld-9YGW=_Bcxss8Yyz_QYjtjus3WUsTb+1g@mail.gmail.com>
	<1507639.1591375547@sss.pgh.pa.us>
	<D973F4AF-8949-42C1-ABF1-E87F7C63E746@anarazel.de>
	<1509305.1591378355@sss.pgh.pa.us>
	<20200605174418.zbb4egfe5wxhv2yd@alap3.anarazel.de>
	<1510961.1591380652@sss.pgh.pa.us>
	<alpine.DEB.2.22.394.2006060851480.32228@pseudo>
	<1673150.1591455433@sss.pgh.pa.us>
	<alpine.DEB.2.22.394.2006061819450.32228@pseudo>


> I cannot foresee a setting where over 60 clients per thread (ISTM there is at 
> most one wait per client) would make much sense, so let's not help the user 
> run ineffective benchmarks. Maybe we should even issue a warning when 
> #client/#thread is high. Even if there is a bench where a thread could really 
> handle 1000 clients (implying 1000 pg processes server side), probably the 
> user could suffer to run that bench with 16 threads.

Here is an attempt at given advice in the doc & a warning when launching.

-- 
Fabien.

Attachments:

pgbench-doc-threads-1.patch (text/x-diff, 2.0K) download diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml index 8e728dc094..e9e7b6b5a7 100644 --- a/doc/src/sgml/ref/pgbench.sgml +++ b/doc/src/sgml/ref/pgbench.sgml @@ -2309,9 +2309,12 @@ statement latencies in milliseconds: <para> A limitation of <application>pgbench</application> is that it can itself become the bottleneck when trying to test a large number of client sessions. - This can be alleviated by running <application>pgbench</application> on a different - machine from the database server, although low network latency will be - essential. It might even be useful to run several <application>pgbench</application> + First, <application>pgbench</application> should be given enough threads to + process its clients, depending on the client-side load induced by benchmarking scripts. + A typical setting would be that a thread manages a few clients, say between 2 and 10. + Second, it might be advisable to run <application>pgbench</application> on a different + machine from the database server, although low network latency will be essential. + It might even be useful to run several <application>pgbench</application> instances concurrently, on several client machines, against the same database server. </para> diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c index 08a5947a9e..3d6080fc83 100644 --- a/src/bin/pgbench/pgbench.c +++ b/src/bin/pgbench/pgbench.c @@ -5841,6 +5841,15 @@ main(int argc, char **argv) if (nthreads > nclients) nthreads = nclients; + /* + * The best #client/#thread ratio depends on the induced client-side load, which + * depends on --rate, on the executed scripts, on the server capabilities... + * + * Just warn if the ratio looks high. + */ + if (nclients > 16 * nthreads) + pg_log_warning("over 16 clients per thread, consider using more threads"); + /* * Convert throttle_delay to a per-thread delay time. Note that this * might be a fractional number of usec, but that's OK, since it's just

view thread (26+ messages)  latest in thread

Message-ID: <alpine.DEB.2.22.394.2006090941480.274691@pseudo>
Permalink:  ../alpine.DEB.2.22.394.2006090941480.274691@pseudo/
Also on:    postgresql.org/message-id/alpine.DEB.2.22.394.2006090941480.274691@pseudo

 ·  · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-bugs@postgresql.org
  Cc: coelho@cri.ensmp.fr, tgl@sss.pgh.pa.us, andres@anarazel.de, pgsql-bugs@lists.postgresql.org, dgrowleyml@gmail.com, Manish.Jawarilal@dell.com
  Subject: Re: pgbench bug / limitation
  In-Reply-To: <alpine.DEB.2.22.394.2006090941480.274691@pseudo>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox