Date: Fri, 18 Feb 2022 18:41:36 -0800
From: Noah Misch <noah@leadboat.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Andres Freund <andres@anarazel.de>, pgsql-hackers@postgresql.org
Subject: Re: Timeout control within tests
Message-ID: <20220219024136.GA3670392@rfd.leadboat.com>
References: <20220218052842.GA3627003@rfd.leadboat.com>
 <20220218054825.vhoiv2f7vgwdsnnb@alap3.anarazel.de>
 <20220218071911.GB3506226@rfd.leadboat.com>
 <3949563.1645198012@sss.pgh.pa.us>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <3949563.1645198012@sss.pgh.pa.us>
User-Agent: Mutt/1.5.24 (2015-08-30)
Archived-At: <https://www.postgresql.org/message-id/20220219024136.GA3670392%40rfd.leadboat.com>
Precedence: bulk

On Fri, Feb 18, 2022 at 10:26:52AM -0500, Tom Lane wrote:
> Noah Misch <noah@leadboat.com> writes:
> > On Thu, Feb 17, 2022 at 09:48:25PM -0800, Andres Freund wrote:
> >> Meson's test runner has the concept of a "timeout multiplier" for ways of
> >> running tests. Meson's stuff is about entire tests (i.e. one tap test), so
> >> doesn't apply here, but I wonder if we shouldn't do something similar?
> 
> > Hmmm.  It is good if the user can express an intent that continues to make
> > sense if we change the default timeout.  For the buildfarm use case, a
> > multiplier is moderately better on that axis (PG_TEST_TIMEOUT_MULTIPLIER=100
> > beats PG_TEST_TIMEOUT_DEFAULT=18000).  For the hacker use case, an absolute
> > value is substantially better on that axis (PG_TEST_TIMEOUT_DEFAULT=3 beats
> > PG_TEST_TIMEOUT_MULTIPLIER=.016666).
> 
> FWIW, I'm fairly sure that PGISOLATIONTIMEOUT=300 was selected after
> finding that smaller values didn't work reliably in the buildfarm.
> Now maybe 741d7f1 fixed that, but I wouldn't count on it.  So while I
> approve of the idea to remove PGISOLATIONTIMEOUT in favor of using this
> centralized setting, I think that we might need to have a multiplier
> there, or else we'll end up with PG_TEST_TIMEOUT_DEFAULT set to 300
> across the board.  Perhaps the latter is fine, but a multiplier seems a
> bit more flexible.

The PGISOLATIONTIMEOUT replacement was 2*timeout_default, so isolation suites
would get 2*180s=360s.  (I don't want to lower any default timeouts, but I
don't mind raising them.)  In a sense, PG_TEST_TIMEOUT_DEFAULT is a multiplier
with as many sites as possible multiplying it by 1.  The patch has multiples
at two code sites.

> On the other hand, I also support your point that an absolute setting
> is easier to think about / adjust for special uses.  So maybe we should
> just KISS and use a single absolute setting until we find a hard reason
> why that doesn't work well.