public inbox for [email protected]
help / color / mirror / Atom feedFrom: Yugo Nagata <[email protected]>
To: Chao Li <[email protected]>
Cc: PostgreSQL-development <[email protected]>
Cc: Fujii Masao <[email protected]>
Cc: Rintaro Ikeda <[email protected]>
Subject: Re: pgbench --continue-on-error: clarify TPS and failure reporting
Date: Thu, 18 Jun 2026 14:36:31 +0900
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
On Thu, 18 Jun 2026 11:24:04 +0800
Chao Li <[email protected]> wrote:
> Hi
>
> While testing “[0ab208fa5] pgbench: Add --continue-on-error option”, I got a question about the doc:
> ```
> <para>
> This option is useful when your custom script may raise errors
> such as unique constraint violations, but you want the benchmark
> to continue and measure performance including those failures.
> </para>
> ```
>
> The phrase “measure performance including those failures” gives the impression that failed transactions might be counted in TPS, but they are not.
I agree with your point. The original phrasing "including those failures" is
indeed ambiguous and could mislead users into thinking failed transactions are
included in the TPS calculation.
> In my test, all transactions failed, and TPS was reported as 0:
> ```
> % pgbench -d evantest -n -t 5 --continue-on-error --failures-detailed -f pgbench-coe.sql
> pgbench (19beta1)
> transaction type: pgbench-coe.sql
> scaling factor: 1
> query mode: simple
> number of clients: 1
> number of threads: 1
> maximum number of tries: 1
> number of transactions per client: 5
> number of transactions actually processed: 0/5
> number of failed transactions: 5 (100.000%)
> number of serialization failures: 0 (0.000%)
> number of deadlock failures: 0 (0.000%)
> number of other failures: 5 (100.000%)
> latency average = 0.389 ms (including failures)
> initial connection time = 4.172 ms
> tps = 0.000000 (without initial connection time)
> ```
>
> If this behavior is intended, I wonder if we should clarify the TPS counting logic more explicitly. For example:
> ```
> <para>
> This option is useful when your custom script may raise errors
> such as unique constraint violations, but you want the benchmark
> to continue despite individual statement failures. Failed transactions
> are reported separately and included in latency statistics, but are not
> counted as transactions actually processed for TPS.
> </para>
> ```
Your proposed documentation change is much clearer and accurately reflects
the actual behavior. I prefer this version as well.
Regards,
Yugo Nagata
--
Yugo Nagata <[email protected]>
view thread (6+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected]
Subject: Re: pgbench --continue-on-error: clarify TPS and failure reporting
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox