From: Tom Lane <tgl@sss.pgh.pa.us>
To: Fabien COELHO <coelho@cri.ensmp.fr>
Cc: PostgreSQL Developers <pgsql-hackers@lists.postgresql.org>
Subject: Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD
Date: Tue, 22 Jan 2019 12:12:29 -0500
Message-ID: <11101.1548177149@sss.pgh.pa.us> (raw)
In-Reply-To: <alpine.DEB.2.21.1901221654400.16643@lancre>
References: <9427.1547701450@sss.pgh.pa.us>
<alpine.DEB.2.21.1901170659210.7416@lancre>
<21538.1547770868@sss.pgh.pa.us>
<4615.1547792324@sss.pgh.pa.us>
<alpine.DEB.2.21.1901180926580.26418@lancre>
<29225.1547834182@sss.pgh.pa.us>
<alpine.DEB.2.21.1901182257480.20734@lancre>
<alpine.DEB.2.21.1901201051270.14950@lancre>
<7879.1548015981@sss.pgh.pa.us>
<alpine.DEB.2.21.1901202225310.28945@lancre>
<alpine.DEB.2.21.1901221054310.15365@lancre>
<6661.1548171989@sss.pgh.pa.us>
<alpine.DEB.2.21.1901221654400.16643@lancre>
Fabien COELHO <coelho@cri.ensmp.fr> writes:
>> I'm not following this argument. The test case is basically useless
>> for its intended purpose with that parameter, because it's highly
>> likely that the failure mode it's supposedly checking for will be
>> masked by the "random" function's tendency to spit out the same
>> value all the time.
> The first value is taken about 75% of the time for N=1000 and s=2.5, which
> means that a non deterministic implementation would succeed about 0.75² ~
> 56% of the time on that one.
Right, that's about what we've been seeing on OpenBSD.
> Also, the drawing procedure is less efficient when the parameter is close
> to 1 because it is more likely to loop,
That might be something to fix, but I agree it's a reason not to go
overboard trying to flatten the test case's distribution right now.
> If you want something more drastic, using 1.5 instead of 2.5 would reduce
> the probability of accidentaly passing the test by chance to about 20%, so
> it would fail 80% of the time.
I think your math is off; 1.5 works quite well here. I saw one failure
to produce distinct values in 20 attempts. It's not demonstrably slower
than 2.5 either. (1.1 is measurably slower; probably not by enough for
anyone to care, but 1.5 is good enough for me.)
regards, tom lane
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tgl@sss.pgh.pa.us, coelho@cri.ensmp.fr, pgsql-hackers@lists.postgresql.org
Subject: Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD
In-Reply-To: <11101.1548177149@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox