From: Noah Misch <noah@leadboat.com>
To: Andrew Dunstan <andrew@dunslane.net>
Cc: Fabien COELHO <coelho@cri.ensmp.fr>
Cc: Gregory Smith <gregsmithpgsql@gmail.com>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Cc: david.christensen@crunchydata.com
Subject: Re: pgbench logging broken by time logic changes
Date: Thu, 17 Jun 2021 21:05:28 -0700
Message-ID: <20210618040528.GA1066073@rfd.leadboat.com> (raw)
In-Reply-To: <e05abaab-7bd2-93ed-eeca-3f3ceb542506@dunslane.net>
References: <CAHLJuCW_8Vpcr0=t6O_gozrg3wXXWXZXDioYJd3NhvKriqgpfQ@mail.gmail.com>
<alpine.DEB.2.22.394.2106162032110.2407635@pseudo>
<e05abaab-7bd2-93ed-eeca-3f3ceb542506@dunslane.net>
On Wed, Jun 16, 2021 at 03:13:30PM -0400, Andrew Dunstan wrote:
> On 6/16/21 2:59 PM, Fabien COELHO wrote:
> > The key feedback for me is the usual one: what is not tested does not
> > work. Wow:-)
>
> Agreed.
>
> > I'm unhappy because I already added tap tests for time-sensitive
> > features (-T and others, maybe logging aggregates, cannot remember),
> > which have been removed because they could fail under some
> > circonstances (eg very very very very slow hosts), or required some
> > special handling (a few lines of code) in pgbench, and the net result
> > of this is there is not a single test in place for some features:-(
>
> I'm not familiar with exactly what happened in this case, but tests need
> to be resilient over a wide range of performance characteristics. One
> way around this issue might be to have a way of detecting that it's on a
> slow platform and if so either skipping tests (Test::More provides
> plenty of support for this) or expecting different results.
Detection would need the host to be consistently slow, like running under
Valgrind or a 20-year-old CPU. We also test on systems having highly-variable
performance due to other processes competing for the same hardware. I'd
perhaps add a "./configure --enable-realtime-tests" option that enables
affected tests. Testers should use the option whenever the execution
environment has sufficient reserved CPU.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: noah@leadboat.com, andrew@dunslane.net, coelho@cri.ensmp.fr, gregsmithpgsql@gmail.com, david.christensen@crunchydata.com
Subject: Re: pgbench logging broken by time logic changes
In-Reply-To: <20210618040528.GA1066073@rfd.leadboat.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox