Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mCEHi-0000bn-Nt for pgsql-committers@arkaria.postgresql.org; Sat, 07 Aug 2021 04:50:11 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1mCEHg-0006lT-0u for pgsql-committers@arkaria.postgresql.org; Sat, 07 Aug 2021 04:50:08 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mCEHe-0006lL-JT for pgsql-committers@lists.postgresql.org; Sat, 07 Aug 2021 04:50:07 +0000 Received: from wout1-smtp.messagingengine.com ([64.147.123.24]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mCEHW-0003G3-NP for pgsql-committers@lists.postgresql.org; Sat, 07 Aug 2021 04:50:04 +0000 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id 29D9B320091E; Sat, 7 Aug 2021 00:49:55 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Sat, 07 Aug 2021 00:49:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=anarazel.de; h= date:from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm2; bh=ISrhC7Niaa9emndxewkX3ZE5G4Z JAUpn9t9mRdxxn6o=; b=sfydqpIAVTqeXp7Aeo+iFOtzXN1vZjUZYz1GnGf9NHT mnVBIWjtolCAn2ZYKJg+bt7BE376MRXgT6DLRP0gFD3WuFFwQoEk8hTmVmLT+RoF +DuxBH3/2AIwfaLrRkXLkyOcXiLpGw/QvCTZJ1oSwucBIapV4ZVVmJAUG+iT/vXn ai4vGkqlmGDZbSM6TW6iFFzJp2EdMDxWP5xaBViK2TUCvulRm/PYvw5rJxfB3JG2 ejnGiOisTFPzstAnuNZ2fobZ+5Ae9/BNRxlbT8GA4iNfBSEDgn45tAVaO26vJMhM Ku8vzSpZBZB2hC4aKTgPtluRtl45lNnzbAGWcTPLFjA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=ISrhC7 Niaa9emndxewkX3ZE5G4ZJAUpn9t9mRdxxn6o=; b=AXs5TCTc4Ggu6uX1SHNnb5 5MF+GWaBSdxpNtLsfNb/zZqxqpSbo1l+EUy8X1MtU+tm7Y4KP61tSbaw9bXIKAF9 eareMDgUg5pNm+Iz0fI43LydpZeNbIIZ/Ame5pqfPzT0im/e9x9c5p06x5wT4d76 YxOCnumjieSj19LXfop+0T1dnjdUlF0/Ekat5tm6Hr6O/hf0NIXpOJNP61JJy8ql fM+4pcy33zZb9GTcBL5gmDlILR5K8G3mSxf3TnT1oKeN5lgkI6Y8aWNoMn/w2CWi tWXwLjI03WhZebWer8c8nW2r7Ne+znXa8XaFsbZOlNmE7B3ZRBMR/bPWgF+R+Krg == X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrjedvgdekgecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehttdertd dttddvnecuhfhrohhmpeetnhgurhgvshcuhfhrvghunhguuceorghnughrvghssegrnhgr rhgriigvlhdruggvqeenucggtffrrghtthgvrhhnpeevhfejveehvddvgfdtueefuefhhf etkefgfedtkefhieeuudefveduffdugfdvgeenucffohhmrghinheptghirhhruhhsqdgt ihdrtghomhenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhroh hmpegrnhgurhgvshesrghnrghrrgiivghlrdguvg X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 7 Aug 2021 00:49:54 -0400 (EDT) Date: Fri, 6 Aug 2021 21:49:52 -0700 From: Andres Freund To: Tom Lane Cc: pgsql-committers@lists.postgresql.org Subject: Re: pgsql: pgstat: Bring up pgstat in BaseInit() to fix uninitialized use o Message-ID: <20210807044952.orwuf7hua2wytx4g@alap3.anarazel.de> References: <2638683.1628304247@sss.pgh.pa.us> <20210807030652.qf5wsbhxk7dpsmtn@alap3.anarazel.de> <2640064.1628306008@sss.pgh.pa.us> <2640490.1628306556@sss.pgh.pa.us> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2640490.1628306556@sss.pgh.pa.us> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk Hi, On 2021-08-06 23:22:36 -0400, Tom Lane wrote: > I wrote: > > Guessing the common factor is "macOS", but that's just a guess. > > I can poke into it tomorrow. > > I did try it real quick on my Mac laptop, and that fails too. > Here's a more accurate backtrace, in case that helps. Thanks! I now managed to reproduce it on a CI system (after some initial failed attempts due to pg_upgrade tests failing due to path length issues): https://api.cirrus-ci.com/v1/artifact/task/4863568911794176/log/src/bin/pg_basebackup/tmp_check/log/010_pg_basebackup_main.log 2021-08-06 21:18:40.096 PDT [22031] 010_pg_basebackup.pl LOG: received replication command: BASE_BACKUP LABEL 'pg_basebackup base backup' PROGRESS NOWAIT MANIFEST 'yes' 2021-08-06 21:18:40.096 PDT [22031] 010_pg_basebackup.pl STATEMENT: BASE_BACKUP LABEL 'pg_basebackup base backup' PROGRESS NOWAIT MANIFEST 'yes' 2021-08-06 21:18:42.210 PDT [22031] 010_pg_basebackup.pl LOG: could not send data to client: Broken pipe 2021-08-06 21:18:42.210 PDT [22031] 010_pg_basebackup.pl STATEMENT: BASE_BACKUP LABEL 'pg_basebackup base backup' PROGRESS NOWAIT MANIFEST 'yes' 2021-08-06 21:18:42.210 PDT [22031] 010_pg_basebackup.pl FATAL: connection to client lost 2021-08-06 21:18:42.210 PDT [22031] 010_pg_basebackup.pl STATEMENT: BASE_BACKUP LABEL 'pg_basebackup base backup' PROGRESS NOWAIT MANIFEST 'yes' TRAP: FailedAssertion("pgstat_is_initialized && !pgstat_is_shutdown", File: "pgstat.c", Line: 4810, PID: 22031) vs what I locally get: 2021-08-06 20:52:06.163 PDT [1397252] 010_pg_basebackup.pl LOG: received replication command: BASE_BACKUP LABEL 'pg_basebackup base backup' PROGRESS NOWAIT MANIFEST 'yes' 2021-08-06 20:52:06.163 PDT [1397252] 010_pg_basebackup.pl STATEMENT: BASE_BACKUP LABEL 'pg_basebackup base backup' PROGRESS NOWAIT MANIFEST 'yes' 2021-08-06 20:52:08.189 PDT [1397252] 010_pg_basebackup.pl LOG: could not send data to client: Broken pipe 2021-08-06 20:52:08.189 PDT [1397252] 010_pg_basebackup.pl STATEMENT: BASE_BACKUP LABEL 'pg_basebackup base backup' PROGRESS NOWAIT MANIFEST 'yes' 2021-08-06 20:52:08.189 PDT [1397252] 010_pg_basebackup.pl ERROR: base backup could not send data, aborting backup 2021-08-06 20:52:08.189 PDT [1397252] 010_pg_basebackup.pl STATEMENT: BASE_BACKUP LABEL 'pg_basebackup base backup' PROGRESS NOWAIT MANIFEST 'yes' 2021-08-06 20:52:08.189 PDT [1397252] 010_pg_basebackup.pl FATAL: connection to client lost Note that OSX version doesn't have the "base backup could not send data, aborting backup" message. I guess that is what leads to the difference in behaviour: The temp file is created by InitializeBackupManifest(). In the !OSX case, we first abort via an ERROR, which triggers the cleanup via WalSndResourceCleanup(). On OSX however, we immediately error out with FATAL for some reason (timing? network buffering differences?), which will never reach WalSndErrorCleanup(). Therefore the temp file only gets deleted during proc_exit(), which triggers the issue... Not yet really sure what the best way to deal with this is. Presumably this issue would be fixed if AtProcExit_Files()/CleanupTempFiles() were scheduled via before_shmem_exit(). And perhaps it's not too off to schedule CleanupTempFiles() there - but it doesn't quite seem entirely right either. I'd kinda like to avoid having to overhaul the process exit infrastructure as a prerequisite to getting the shared memory stats patch in :(. Greetings, Andres Freund