pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
To: Andres Freund <andres@anarazel.de>
Cc: pgsql-hackers@lists.postgresql.org
Subject: Re: test/modules/test_oat_hooks vs. debug_discard_caches=1
Date: Sat, 19 Nov 2022 11:34:07 +0100
Message-ID: <9ce14134-5725-5b9f-e27e-f4002ff13a4e@enterprisedb.com> (raw)
In-Reply-To: <2316565.1668827411@sss.pgh.pa.us>
References: <2267406.1668804934@sss.pgh.pa.us>
	<20221119003349.uzpopxq2skgrmwc4@awork3.anarazel.de>
	<2316565.1668827411@sss.pgh.pa.us>



On 11/19/22 04:10, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
>> On 2022-11-18 15:55:34 -0500, Tom Lane wrote:
>>> We realized today [1] that it's been some time since the buildfarm
>>> had any debug_discard_caches (nee CLOBBER_CACHE_ALWAYS) coverage.
> 
>> Do we know when it was covered last? I assume it's before the addition of
>> test_oat_hooks in 90efa2f5565?
> 
> As far as that goes: some digging in the buildfarm DB says that avocet
> last did a CCA run on 2021-10-22 and trilobite on 2021-10-24.  They
> were then offline completely until 2022-02-10, and when they restarted
> the runtimes were way too short to be CCA tests.
> 

Yeah. I'll try setting up a better monitoring / alerting to notice
issues like this more promptly ... it's a bit tough, because IIRC the
gap 2021-10-22 - 2022-02-10 was due to the tests running, but getting
stuck for some reason. So it's not like the machine was off.

I wonder if it'd make sense to have some simple & optional alerting
based on how long ago the machine reported the last result. Send e-mail
if there was no report for a month or so would be enough.

> Seems like maybe we need a little more redundancy in this bunch of
> buildfarm animals.
> 

It's actually a bit worse than that, because both animals are on the
same machine. So avocet gets "stuck" -> trilobite is stuck too.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





view thread (9+ messages)  latest in thread

Message-ID: <9ce14134-5725-5b9f-e27e-f4002ff13a4e@enterprisedb.com>
Permalink:  ../9ce14134-5725-5b9f-e27e-f4002ff13a4e@enterprisedb.com/
Also on:    postgresql.org/message-id/9ce14134-5725-5b9f-e27e-f4002ff13a4e@enterprisedb.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-hackers@postgresql.org
  Cc: tomas.vondra@enterprisedb.com, tgl@sss.pgh.pa.us, andres@anarazel.de
  Subject: Re: test/modules/test_oat_hooks vs. debug_discard_caches=1
  In-Reply-To: <9ce14134-5725-5b9f-e27e-f4002ff13a4e@enterprisedb.com>
  References: <2267406.1668804934@sss.pgh.pa.us> <20221119003349.uzpopxq2skgrmwc4@awork3.anarazel.de> <2316565.1668827411@sss.pgh.pa.us> <9ce14134-5725-5b9f-e27e-f4002ff13a4e@enterprisedb.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox