Content-Type: text/plain;
	charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3864.600.51.1.1\))
Subject: Re: pgbench --continue-on-error: clarify TPS and failure reporting
From: Chao Li <li.evan.chao@gmail.com>
In-Reply-To: 
 <CAHGQGwFX6Wqdp=ZEjs6WSu3GDSpgM9qZTixBHXKB9xaVzQ+0-A@mail.gmail.com>
Date: Thu, 18 Jun 2026 20:45:29 +0800
Cc: Yugo Nagata <nagata@sraoss.co.jp>,
 PostgreSQL-development <pgsql-hackers@postgresql.org>,
 Rintaro Ikeda <ikedarintarof@oss.nttdata.com>
Content-Transfer-Encoding: quoted-printable
Message-Id: <09361838-9213-4961-95A5-A0CD4AABF2DD@gmail.com>
References: <597C9755-4386-488D-A289-16D50AD81FBF@gmail.com>
 <20260618143631.25c2ed34b1d7527ee2d92a17@sraoss.co.jp>
 <958C5B66-50B0-47C8-BBA5-7D32F7A09566@gmail.com>
 <CAHGQGwFX6Wqdp=ZEjs6WSu3GDSpgM9qZTixBHXKB9xaVzQ+0-A@mail.gmail.com>
To: Fujii Masao <masao.fujii@gmail.com>
Archived-At: 
 <https://www.postgresql.org/message-id/09361838-9213-4961-95A5-A0CD4AABF2DD%40gmail.com>
Precedence: bulk


> On Jun 18, 2026, at 17:40, Fujii Masao <masao.fujii@gmail.com> wrote:
>=20
> On Thu, Jun 18, 2026 at 3:25=E2=80=AFPM Chao Li =
<li.evan.chao@gmail.com> wrote:
>> Thanks for confirming the doc issue. The here comes a small doc =
patch.
>>=20
>> I also noticed there is a later paragraph about =E2=80=9Clatency=E2=80=9D=
 that also needs an update, it=E2=80=99s included in the patch as well.
>=20
> Thanks for the patch! The TPS-related changes look good to me.
>=20
>    The latency of a successful transaction includes the entire time of
> -   transaction execution with rollbacks and retries. The latency is =
measured
> -   only for successful transactions and commands but not for failed
> transactions
> -   or commands.
> +   transaction execution with rollbacks and retries.  The latency is =
measured
> +   only for successful transactions and commands, except that failed
> +   transactions are included in the simple average latency when the
> +   <option>--continue-on-error</option> option is used.
>=20
> It doesn't seem that the behavior of the latency average in the main
> report depends on --continue-on-error itself. Instead, when neither
> --progress, --rate, nor --latency-limit is specified, the main
> report shows a simple latency average computed as the total benchmark
> duration divided by the number of successful and failed transactions,
> so failed transactions are included.
>=20
> On the other hand, when any of those options is specified, the
> latency average and latency stddev in the main report are based
> only on successful transactions. Likewise, the per-script and
> per-command latency statistics are measured only for successful
> transactions and commands.
>=20
> So how about something like this instead?
>=20
> -------------------------------
> <para>
>    The latency of a successful transaction includes the entire time of
> -   transaction execution with rollbacks and retries. The simple
> -   <literal>latency average</literal> shown in the main report is =
computed from
> -   the total benchmark duration divided by both successful and failed
> -   transactions, and therefore includes failed transactions. In =
contrast,
> -   detailed latency statistics, including the per-script and =
per-command
> -   reports, are measured only for successful transactions and =
commands, not
> -   for failed transactions or commands.
> +   transaction execution with rollbacks and retries. In the main =
report, when
> +   neither <option>--progress</option>, <option>--rate</option>, nor
> +   <option>--latency-limit</option> is specified, the
> +   <literal>latency average</literal> is computed from the total =
benchmark
> +   duration divided by both successful and failed transactions, and =
therefore
> +   includes failed transactions. Otherwise, the <literal>latency
> average</literal>
> +   and <literal>latency stddev</literal> shown in the main report are =
measured
> +   only for successful transactions. Detailed latency statistics, =
including
> +   the per-script and per-command reports, are also measured only for
> +   successful transactions and commands, not for failed transactions =
or
> +   commands.
>   </para>
> -------------------------------
>=20

I feel that might be too detailed. Listing these three option names here =
could make the doc harder to read and maintain. When failures are =
included, the output line already says so explicitly:
```
latency average =3D 0.387 ms (including failures)
```

Can we simply say something like this?
```
  <para>
   The latency of a successful transaction includes the entire time of
   transaction execution with rollbacks and retries.  The latency is =
measured
   only for successful transactions and commands, not for failed =
transactions
   or commands, unless the report explicitly says that it includes =
failures.
  </para>
```

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/