From: Chao Li <li.evan.chao@gmail.com>
Content-Type: text/plain;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3864.600.51.1.1\))
Subject: pgbench --continue-on-error: clarify TPS and failure reporting
Message-Id: <597C9755-4386-488D-A289-16D50AD81FBF@gmail.com>
Date: Thu, 18 Jun 2026 11:24:04 +0800
Cc: Fujii Masao <masao.fujii@gmail.com>,
 Rintaro Ikeda <ikedarintarof@oss.nttdata.com>,
 Yugo Nagata <nagata@sraoss.co.jp>
To: PostgreSQL-development <pgsql-hackers@postgresql.org>
Archived-At: 
 <https://www.postgresql.org/message-id/597C9755-4386-488D-A289-16D50AD81FBF%40gmail.com>
Precedence: bulk

Hi

While testing =E2=80=9C[0ab208fa5] pgbench: Add --continue-on-error =
option=E2=80=9D, I got a question about the doc:
```
       <para>
        This option is useful when your custom script may raise errors
        such as unique constraint violations, but you want the benchmark
        to continue and measure performance including those failures.
       </para>
```

The phrase =E2=80=9Cmeasure performance including those failures=E2=80=9D =
gives the impression that failed transactions might be counted in TPS, =
but they are not.

In my test, all transactions failed, and TPS was reported as 0:
```
% pgbench  -d evantest -n -t 5 --continue-on-error --failures-detailed =
-f pgbench-coe.sql
pgbench (19beta1)
transaction type: pgbench-coe.sql
scaling factor: 1
query mode: simple
number of clients: 1
number of threads: 1
maximum number of tries: 1
number of transactions per client: 5
number of transactions actually processed: 0/5
number of failed transactions: 5 (100.000%)
number of serialization failures: 0 (0.000%)
number of deadlock failures: 0 (0.000%)
number of other failures: 5 (100.000%)
latency average =3D 0.389 ms (including failures)
initial connection time =3D 4.172 ms
tps =3D 0.000000 (without initial connection time)
```

If this behavior is intended, I wonder if we should clarify the TPS =
counting logic more explicitly. For example:
```
       <para>
        This option is useful when your custom script may raise errors
        such as unique constraint violations, but you want the benchmark
        to continue despite individual statement failures. Failed =
transactions
        are reported separately and included in latency statistics, but =
are not
        counted as transactions actually processed for TPS.
       </para>
```

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/