Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1owLAx-0001ou-Gt for pgsql-hackers@arkaria.postgresql.org; Sat, 19 Nov 2022 10:34:19 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1owLAu-0002ax-Uz for pgsql-hackers@arkaria.postgresql.org; Sat, 19 Nov 2022 10:34:16 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1owLAu-0002ao-7z for pgsql-hackers@lists.postgresql.org; Sat, 19 Nov 2022 10:34:16 +0000 Received: from mail-ed1-x530.google.com ([2a00:1450:4864:20::530]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1owLAp-0001Wv-DJ for pgsql-hackers@lists.postgresql.org; Sat, 19 Nov 2022 10:34:15 +0000 Received: by mail-ed1-x530.google.com with SMTP id x102so10387533ede.0 for ; Sat, 19 Nov 2022 02:34:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enterprisedb.com; s=google; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=GhKgHliX8K1v4ITGssquB3rsodST+VphynWrqK0ttZM=; b=B0i2mfgRHgFPGRDMfr9UZ3IAPFVaEGlXcD4J4eW2rCNQy4/4CimBZBrOb4FtEjyoqs /kmpTQLSHAH9bLyxnYHVSpxhcp1GJx+hGcyMvJt0w0oLbeulFlnBivkUMLEurIc5r+vf DchQGSPHGMpGMx14G7rvBhk/YE5+4ofzNv/kX0uWorndIocnqHBLdHsvrbZQhmZmJ8uQ AmxfCseq1rYyPs/H1a1fug3WdhRFinRNA95TwSA7InLKsZxh5g7+aQfzhYcetDpjvua6 o10YqRWOjzO75R7jku9O7jwVDwdE+pex0TSdGNq/VjJWp/Q2Eqbln7ee3LK5nyonDtYB hcSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=GhKgHliX8K1v4ITGssquB3rsodST+VphynWrqK0ttZM=; b=DqpNwVUr8WpYuDP+pvWrpErJLeB0wa/3aQ28mZq05TetUDizzsufJH03uVwBhhBudN a2bpc5WDfsSZw5fnz1AjrwgO3hI8U4JSy6t9aKDLMU+zDuz0kVyQWCszfj2Q370o1LX/ Mtqymcx2GhaOSxxzWsahMsN68g0p1glKxp7KNv0hGDU7HODCUO8P7kwDztF3hnGek9LT A87u4NPMwoM0ntEif3E7njulF8JImvOsCBiHeKitSavDB7Zeova9LGJOKgxQt9ndCVzO M+K8QPrMBDyAbNCG23GVxeWAlXmJvNgJ1TdBo4TsLCpoMj44qHTTGrrOjN866cEZ5FoZ QvOQ== X-Gm-Message-State: ANoB5pnorC10nV3gi7PfNpxz4/q34ExgMoa7uHIWm12s4nCfl6QRLo79 omBxLhLokWb3DFZVLL+9vtlkzqnSPjZG9PjybZHXuUFI+XS1pQgEfu7j+7dP+hYAo24fpNhJNaj LTyJFssNnuawF0Ts7H21xBwTEbmtXaX33PqxnHGLjKHqXFewbdYGCGdGotOzMrdbFdopBFiGzr+ RQxLCoEbwFTFfQyQUfKtdoKRMIjjxf9mxCoRuZk7HGABDRRtbkybI/ogwc6NyKHdRGudw0Bg4Zo BZOALuRzjVVW+BWS8uK2cbcJGskp8aG2HmZz39mGlh0mPBqUwW7+QsEXJbw8gcuW0Bn X-Google-Smtp-Source: AA0mqf4elJgzHG73NPE/8FJsDSrhYdkw2QZ0h53eNqTxARf4g1/CPpY8LnT6ehwKQJxq7gg02tGTfA== X-Received: by 2002:a05:6402:4d6:b0:458:789b:c1b0 with SMTP id n22-20020a05640204d600b00458789bc1b0mr9158077edw.89.1668854049484; Sat, 19 Nov 2022 02:34:09 -0800 (PST) Received: from [10.137.0.17] (ip-86-49-228-162.bb.vodafone.cz. [86.49.228.162]) by smtp.gmail.com with ESMTPSA id z21-20020a17090665d500b007aed2057ea1sm2705292ejn.167.2022.11.19.02.34.08 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 19 Nov 2022 02:34:09 -0800 (PST) Message-ID: <9ce14134-5725-5b9f-e27e-f4002ff13a4e@enterprisedb.com> Date: Sat, 19 Nov 2022 11:34:07 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.5.0 Subject: Re: test/modules/test_oat_hooks vs. debug_discard_caches=1 To: Tom Lane , Andres Freund Cc: pgsql-hackers@lists.postgresql.org References: <2267406.1668804934@sss.pgh.pa.us> <20221119003349.uzpopxq2skgrmwc4@awork3.anarazel.de> <2316565.1668827411@sss.pgh.pa.us> Content-Language: en-US From: Tomas Vondra In-Reply-To: <2316565.1668827411@sss.pgh.pa.us> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-CLOUD-SEC-AV-Info: enterprisedb,google_mail,monitor X-CLOUD-SEC-AV-Sent: true X-Gm-Spam: 0 X-Gm-Phishy: 0 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On 11/19/22 04:10, Tom Lane wrote: > Andres Freund writes: >> On 2022-11-18 15:55:34 -0500, Tom Lane wrote: >>> We realized today [1] that it's been some time since the buildfarm >>> had any debug_discard_caches (nee CLOBBER_CACHE_ALWAYS) coverage. > >> Do we know when it was covered last? I assume it's before the addition of >> test_oat_hooks in 90efa2f5565? > > As far as that goes: some digging in the buildfarm DB says that avocet > last did a CCA run on 2021-10-22 and trilobite on 2021-10-24. They > were then offline completely until 2022-02-10, and when they restarted > the runtimes were way too short to be CCA tests. > Yeah. I'll try setting up a better monitoring / alerting to notice issues like this more promptly ... it's a bit tough, because IIRC the gap 2021-10-22 - 2022-02-10 was due to the tests running, but getting stuck for some reason. So it's not like the machine was off. I wonder if it'd make sense to have some simple & optional alerting based on how long ago the machine reported the last result. Send e-mail if there was no report for a month or so would be enough. > Seems like maybe we need a little more redundancy in this bunch of > buildfarm animals. > It's actually a bit worse than that, because both animals are on the same machine. So avocet gets "stuck" -> trilobite is stuck too. regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company