Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pMuQY-00042Y-J3 for pgsql-hackers@arkaria.postgresql.org; Tue, 31 Jan 2023 17:28:14 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1pMuQX-00053U-En for pgsql-hackers@arkaria.postgresql.org; Tue, 31 Jan 2023 17:28:13 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pMuQX-00053J-1X for pgsql-hackers@lists.postgresql.org; Tue, 31 Jan 2023 17:28:13 +0000 Received: from mail-il1-x12b.google.com ([2607:f8b0:4864:20::12b]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1pMuQT-00017P-St for pgsql-hackers@lists.postgresql.org; Tue, 31 Jan 2023 17:28:12 +0000 Received: by mail-il1-x12b.google.com with SMTP id h18so4020385ilj.6 for ; Tue, 31 Jan 2023 09:28:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telsasoft-com.20210112.gappssmtp.com; s=20210112; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=g11gDLPV54m2P3gJfIa0tw3XuYh2wQvJnBXs2p92ZQ8=; b=xmNVs3M0oXWXn1aNp3vUd7eTuUYoSrtF5qxsgzEsL63i8I8hmczivC4KfzsVSPon44 nrfS0pb2P/E21xYxVBul8tQl7NSHzHqJIWV0p2cVGdDPYkzAtBfz2x0LFPJD3582lenF v1GNVt2lnbWzLYQCQuHbxgZoMmtsrM9+5Ry0mllxCmzSGu/UG0dmmP1AIaV/A+tULH+l FAGMuPqPhp41FFxkaqbSmtTPzu6sTEV1/3yzyygEOF1tc5/uTPPANctI9WkYOTOwyyOx 6xCxOI7SOuzsQxMbvuDCqlYYre0yKy1MTdadUmrFQS5RZbMUqrp5xcF7Q86eO9cOSfbZ YzWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=g11gDLPV54m2P3gJfIa0tw3XuYh2wQvJnBXs2p92ZQ8=; b=URyUkPFvUV+cqWsMrqC58TBcZc7TdSkuPNLtZGCIrGLR1RnqMFh7dzwqkthdC3u+8j RV7ShMFSc9hpV0J4cFOnHGbi6v5GQV+NU5+nNmKpgkGBJExEDs0EyHEijC0oQXfr+3Ob R2gwRNLnP5BCiCeVuUvZS7NvHS7xgi1/acv6DlBh4crYJ21Psb0SOgCWE2B21MPlF61p Io6Bj18w6RLPNyV+a/7eJDdYGDqCEL7jMaWRhVjbWU/9LnduoHLaXFiGlyKm4VrgW96b K+fTS1n1DauVv+bUhTBV9HkidrCOBLS8XvUgdF666BvF8Di0sqfpyTWbkZvb3A/lkXi7 qbgA== X-Gm-Message-State: AO0yUKWxKz2HMvUir8AUCtJzb0a/hs634ETAy2rUI3X4iclPsMKqvj3I LxxDkfFvfpKaPsFLda1MeywHWQ== X-Google-Smtp-Source: AK7set8NN67s15bI4uz615Wa8mTCBVn4+EseNHS6mahoYiW+vS6ppJhWuDVlswFdF/aipFJoS3QPBg== X-Received: by 2002:a05:6e02:1a0d:b0:311:7e6:f59 with SMTP id s13-20020a056e021a0d00b0031107e60f59mr3699359ild.1.1675186088163; Tue, 31 Jan 2023 09:28:08 -0800 (PST) Received: from pryzbyj.telsasoft (charmander.telsasoft.com. [50.244.222.1]) by smtp.gmail.com with ESMTPSA id h17-20020a92d091000000b0030f8a95b60asm5002155ilh.30.2023.01.31.09.28.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 31 Jan 2023 09:28:07 -0800 (PST) Received: by pryzbyj.telsasoft (Postfix, from userid 1000) id 5E40F800983; Tue, 31 Jan 2023 11:28:06 -0600 (CST) Date: Tue, 31 Jan 2023 11:28:06 -0600 From: Justin Pryzby To: Thomas Munro Cc: Andres Freund , Michael Paquier , pgsql-hackers@lists.postgresql.org, Nazir Bilal Yavuz Subject: Re: pg_upgrade test failure Message-ID: <20230131172806.GM22427@telsasoft.com> References: <20221018043144.GF7745@telsasoft.com> <20221018162908.cb3n7tahtekypbkf@awork3.anarazel.de> <20221206181533.pmcz75zbjgvnnzum@awork3.anarazel.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Tue, Jan 31, 2023 at 02:00:05PM +1300, Thomas Munro wrote: > On Thu, Jan 5, 2023 at 4:11 PM Thomas Munro wrote: > > On Wed, Dec 7, 2022 at 7:15 AM Andres Freund wrote: > > > On 2022-11-08 01:16:09 +1300, Thomas Munro wrote: > > > > So [1] on its own didn't fix this. My next guess is that the attached > > > > might help. > > > > > What is our plan here? This afaict is the most common "false positive" for > > > cfbot in the last weeks. > > I pushed the rmtree() change. Let's see if that helps, or tells us > something new. I found a few failures since then: https://api.cirrus-ci.com/v1/artifact/task/6696942420361216/testrun/build/testrun/pg_upgrade/002_pg_upgrade/log/regress_log_002_pg_upgrade pg_upgrade: warning: could not remove directory "C:/cirrus/build/testrun/pg_upgrade/002_pg_upgrade/data/t_002_pg_upgrade_new_node_data/pgdata/pg_upgrade_output.d/20230131T134931.720/log": Directory not empty pg_upgrade: warning: could not remove directory "C:/cirrus/build/testrun/pg_upgrade/002_pg_upgrade/data/t_002_pg_upgrade_new_node_data/pgdata/pg_upgrade_output.d/20230131T134931.720": Directory not empty https://api.cirrus-ci.com/v1/artifact/task/5119776607961088/testrun/build/testrun/pg_upgrade/002_pg_upgrade/log/regress_log_002_pg_upgrade same I verified that those both include your 54e72b66e, which is pretty strange, since the patch passed tests 10s of times on CI until it was merged, when it started/kept failing. -- Justin