From: Fabien COELHO <coelho@cri.ensmp.fr>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Andres Freund <andres@anarazel.de>
Cc: PostgreSQL Bugs <pgsql-bugs@lists.postgresql.org>
Cc: David Rowley <dgrowleyml@gmail.com>
Cc: Jawarilal, Manish <Manish.Jawarilal@dell.com>
Subject: Re: pgbench bug / limitation
Date: Tue, 9 Jun 2020 09:45:58 +0200 (CEST)
Message-ID: <alpine.DEB.2.22.394.2006090941480.274691@pseudo> (raw)
In-Reply-To: <alpine.DEB.2.22.394.2006061819450.32228@pseudo>
References: <BL0PR1901MB198583EAC6C8A3B770D20F1AED8A0@BL0PR1901MB1985.namprd19.prod.outlook.com>
<384920.1591027137@sss.pgh.pa.us>
<CAApHDvrjWtJa9oQ98oH+9PCvTgOkGZP+gcmottNX95sjn6o9fQ@mail.gmail.com>
<512846.1591057287@sss.pgh.pa.us>
<CAApHDvpMEAaZMxtzixbnb7ySo=Kb1eUfshEB-y4YOm8SgGrpiw@mail.gmail.com>
<alpine.DEB.2.22.394.2006021258170.87735@pseudo>
<CAApHDvpDBkjj6a+Ld-9YGW=_Bcxss8Yyz_QYjtjus3WUsTb+1g@mail.gmail.com>
<1507639.1591375547@sss.pgh.pa.us>
<D973F4AF-8949-42C1-ABF1-E87F7C63E746@anarazel.de>
<1509305.1591378355@sss.pgh.pa.us>
<20200605174418.zbb4egfe5wxhv2yd@alap3.anarazel.de>
<1510961.1591380652@sss.pgh.pa.us>
<alpine.DEB.2.22.394.2006060851480.32228@pseudo>
<1673150.1591455433@sss.pgh.pa.us>
<alpine.DEB.2.22.394.2006061819450.32228@pseudo>
> I cannot foresee a setting where over 60 clients per thread (ISTM there is at
> most one wait per client) would make much sense, so let's not help the user
> run ineffective benchmarks. Maybe we should even issue a warning when
> #client/#thread is high. Even if there is a bench where a thread could really
> handle 1000 clients (implying 1000 pg processes server side), probably the
> user could suffer to run that bench with 16 threads.
Here is an attempt at given advice in the doc & a warning when launching.
--
Fabien.
Attachments:pgbench-doc-threads-1.patch (text/x-diff, 2.0K) downloaddiff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgmlindex 8e728dc094..e9e7b6b5a7 100644--- a/doc/src/sgml/ref/pgbench.sgml+++ b/doc/src/sgml/ref/pgbench.sgml@@ -2309,9 +2309,12 @@ statement latencies in milliseconds:
<para>
A limitation of <application>pgbench</application> is that it can itself become
the bottleneck when trying to test a large number of client sessions.
- This can be alleviated by running <application>pgbench</application> on a different- machine from the database server, although low network latency will be- essential. It might even be useful to run several <application>pgbench</application>+ First, <application>pgbench</application> should be given enough threads to+ process its clients, depending on the client-side load induced by benchmarking scripts.+ A typical setting would be that a thread manages a few clients, say between 2 and 10.+ Second, it might be advisable to run <application>pgbench</application> on a different+ machine from the database server, although low network latency will be essential.+ It might even be useful to run several <application>pgbench</application>
instances concurrently, on several client machines, against the same
database server.
</para>
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.cindex 08a5947a9e..3d6080fc83 100644--- a/src/bin/pgbench/pgbench.c+++ b/src/bin/pgbench/pgbench.c@@ -5841,6 +5841,15 @@ main(int argc, char **argv)
if (nthreads > nclients)
nthreads = nclients;
+ /*+ * The best #client/#thread ratio depends on the induced client-side load, which+ * depends on --rate, on the executed scripts, on the server capabilities...+ *+ * Just warn if the ratio looks high.+ */+ if (nclients > 16 * nthreads)+ pg_log_warning("over 16 clients per thread, consider using more threads");+
/*
* Convert throttle_delay to a per-thread delay time. Note that this
* might be a fractional number of usec, but that's OK, since it's just
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-bugs@postgresql.org
Cc: coelho@cri.ensmp.fr, tgl@sss.pgh.pa.us, andres@anarazel.de, pgsql-bugs@lists.postgresql.org, dgrowleyml@gmail.com, Manish.Jawarilal@dell.com
Subject: Re: pgbench bug / limitation
In-Reply-To: <alpine.DEB.2.22.394.2006090941480.274691@pseudo>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox