From: Noah Misch <noah@leadboat.com>
To: Andres Freund <andres@anarazel.de>
Cc: Petr Jelinek <petr.jelinek@2ndquadrant.com>
Cc: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Cc: Erik Rijkers <er@xs4all.nl>
Cc: David Steele <david@pgmasters.net>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Craig Ringer <craig@2ndquadrant.com>
Cc: pgsql-hackers-owner@postgresql.org
Subject: Re: snapbuild woes
Date: Sat, 29 Apr 2017 14:42:01 -0700
Message-ID: <20170429214201.GB799290@rfd.leadboat.com> (raw)
In-Reply-To: <20170422053458.z7kmzopqdyi2tai4@alap3.anarazel.de>
References: <20170408143059.nrzuwyvv5apnhhta@alap3.anarazel.de>
<20170412063142.GC2870454@tornado.leadboat.com>
<d6f4a15c-b95d-54b1-0b06-f24bcd0b6fcb@2ndquadrant.com>
<20170412172151.nauhiagitlpfxief@alap3.anarazel.de>
<20170413045812.GA2917375@tornado.leadboat.com>
<20170417020404.GB2988768@tornado.leadboat.com>
<20170418041657.hvwss6hz73ghozp5@alap3.anarazel.de>
<20170420000906.75t6uo7bof5pazrq@alap3.anarazel.de>
<ad637c84-534d-b291-9b6c-458f1ff7f54b@2ndquadrant.com>
<20170422053458.z7kmzopqdyi2tai4@alap3.anarazel.de>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
On Fri, Apr 21, 2017 at 10:34:58PM -0700, Andres Freund wrote:
> I've a bunch of tests, but I don't quite know whether we can expose all
> of them via classical tests. There are several easy ones that I
> definitely want to add (import "empty" snapshot; import snapshot with
> running xacts; create snapshot, perform some ddl, import snapshot,
> perform some ddl, check things work reasonably crazy), but there's
> enough others that are just probabilistic. I was wondering about adding
> a loop that simply runs for like 30s and then quits or such, but who
> knows.
If the probabilistic test catches the bug even 5% of the time in typical
configurations, the buildfarm will rapidly identify any regression. I'd
choose a 7s test that detects the bug 5% of the time over a 30s test that
detects it 99% of the time. (When I wrote src/bin/pgbench/t/001_pgbench.pl
for a probabilistic bug, I sized that test to finish in 1s and catch its bug
half the time. In its case, only two buildfarm members were able to
demonstrate the original bug, so 5% detection would have been too low.)
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: noah@leadboat.com, andres@anarazel.de, petr.jelinek@2ndquadrant.com, peter.eisentraut@2ndquadrant.com, er@xs4all.nl, david@pgmasters.net, robertmhaas@gmail.com, craig@2ndquadrant.com, pgsql-hackers-owner@postgresql.org
Subject: Re: snapbuild woes
In-Reply-To: <20170429214201.GB799290@rfd.leadboat.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox