Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1upTrU-005zsD-Ru for pgsql-general@arkaria.postgresql.org; Fri, 22 Aug 2025 15:39:30 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1upTrU-00783d-8C for pgsql-general@arkaria.postgresql.org; Fri, 22 Aug 2025 15:39:28 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1upTrT-00783S-DH for pgsql-general@lists.postgresql.org; Fri, 22 Aug 2025 15:39:28 +0000 Received: from fhigh-b3-smtp.messagingengine.com ([202.12.124.154]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1upTrR-001IVG-0E for pgsql-general@lists.postgresql.org; Fri, 22 Aug 2025 15:39:27 +0000 Received: from phl-compute-04.internal (phl-compute-04.internal [10.202.2.44]) by mailfhigh.stl.internal (Postfix) with ESMTP id 6DC757A0227; Fri, 22 Aug 2025 11:39:23 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-04.internal (MEProxy); Fri, 22 Aug 2025 11:39:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aklaver.com; h= cc:cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1755877163; x=1755963563; bh=RQvknz/xvSXX9FLx3U9JOLlbFXtzXNNAeag7JqA4TlQ=; b= jdm5/pyrP/+VJKjrdjA9T919e9VUTn7hwMkc7+YChzjt420TbI6AI8mCdU4Z6V/r HujDtR3umx5nAA4CA1fNF5nt2BboVogiWnyYXPxR/NjdNHBXw6rCze9lZQ6k0NDA /BEwJZHqm0W1bYq1gghvGyIhs60vHHlx5G8+DFkOX2QrblT7G2syjgJfG3YdhXx7 o2w3oEqUa8uaxcJ6mtBe0VzVfSAwMfKxeMnjHODtDON3NLYH5SArhfpAyVp0Cqfh +vkarerGnB58tn720p+2zsVP1K4trHIP9VZsoQIvxMPBPXyvOnKerUgn3WfB4L8z 2e7HC9B6A3r7+OPuB4u6sg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1755877163; x= 1755963563; bh=RQvknz/xvSXX9FLx3U9JOLlbFXtzXNNAeag7JqA4TlQ=; b=j S4ZnBMfYKOKPn81mBvAfZttMV4NUPZgk9xp1b05k4InLSbWrGsGYbEx3Cg1zsU+W NTpkQBaZ0AU01lDbNSCTJxcjarBp1bdJNfSEZar3sGsGt6bINu/tOSdbXhibFeGj Sx+hk50n7UysH239R/dmF1jMmeMxSB1yAs/PsXcXk5BJNFAnYZCKg1Nr7p9caCsZ aTkPPwlf4L0iAJr+4tbnGl4vOPgTQtH6u3AmISD1etnansdtVuOLtwXSgnK36FyP FErWO3M2Ci+1sJhyptdI+Zg1wkudU/3Pz7zaTK/QQkz/UErS9MXdI1qBM3sVmNUg 38T0r0LXIl6o/CuCQbFXg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtdefgdduieegudegucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfurfetoffkrfgpnffqhgenuceu rghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmnecujf gurhepkfffgggfuffvvehfhfgjtgfgsehtjeertddtvdejnecuhfhrohhmpeetughrihgr nhcumfhlrghvvghruceorggurhhirghnrdhklhgrvhgvrhesrghklhgrvhgvrhdrtghomh eqnecuggftrfgrthhtvghrnhephfeviefhveelffeftdehudekveefhfeftdegieefveet fffgfeehtdfftedutedtnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrg hilhhfrhhomheprggurhhirghnrdhklhgrvhgvrhesrghklhgrvhgvrhdrtghomhdpnhgs pghrtghpthhtohepgedpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepuggvphgvsh iiseguvghpvghsiidrtghomhdprhgtphhtthhopehtghhlsehsshhsrdhpghhhrdhprgdr uhhspdhrtghpthhtohepphhgshhqlhdqghgvnhgvrhgrlheslhhishhtshdrphhoshhtgh hrvghsqhhlrdhorhhgpdhrtghpthhtoheptghhrhhishdoghhoohhglhgvsehqfihirhig rdgtohhm X-ME-Proxy: Feedback-ID: i76984098:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 22 Aug 2025 11:39:22 -0400 (EDT) Message-ID: <99abba30-1c14-4cc6-aef4-2f8a8f4bfac2@aklaver.com> Date: Fri, 22 Aug 2025 08:39:21 -0700 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Streaming replica hangs periodically for ~ 1 second - how to diagnose/debug To: depesz@depesz.com, Tom Lane Cc: PostgreSQL General , Chris Wilson References: <25334887-f1c3-40a1-94b0-753c7d67ae2b@aklaver.com> <2a3e4a8d-e8c2-46d6-ad7d-9e631ce6725e@aklaver.com> <1882312.1755876082@sss.pgh.pa.us> Content-Language: en-US From: Adrian Klaver In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On 8/22/25 08:30, hubert depesz lubaczewski wrote: > On Fri, Aug 22, 2025 at 11:21:22AM -0400, Tom Lane wrote: >> hubert depesz lubaczewski writes: >>> I got repeatable case today. Is is breaking on its own everyy >>> ~ 5 minutes. >> >> Interesting. That futex call is presumably caused by interaction >> with some other process within the standby server, and the only >> plausible candidate really is the startup process (which is replaying >> WAL received from the primary). There are cases where WAL replay >> will take locks that can block queries on the standby. Can you >> correlate the delays on the standby server with any DDL events >> occurring on the primary? > > Nope. Plus there is certain repetition of these cases, so even if I'd > miss *some* create table/alter, it just isn't going to be happening > every 4-5 minutes. > > So, while there are outliers, I'd say that most of the problems happens every > 3-5 minutes. Are you using the Postgres community version or the AWS variant? > > depesz > -- Adrian Klaver adrian.klaver@aklaver.com