Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1uomdr-00A7CS-9h for pgsql-general@arkaria.postgresql.org; Wed, 20 Aug 2025 17:30:32 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1uomdq-00AQcw-Oe for pgsql-general@arkaria.postgresql.org; Wed, 20 Aug 2025 17:30:31 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1uomdq-00AQcn-EY for pgsql-general@lists.postgresql.org; Wed, 20 Aug 2025 17:30:31 +0000 Received: from mail-wr1-f50.google.com ([209.85.221.50]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1uomdp-000wwG-04 for pgsql-general@lists.postgresql.org; Wed, 20 Aug 2025 17:30:30 +0000 Received: by mail-wr1-f50.google.com with SMTP id ffacd0b85a97d-3c44dfa7739so49249f8f.1 for ; Wed, 20 Aug 2025 10:30:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755711028; x=1756315828; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=rBep0SuZVNGodZ62QAa4qQJoytNOxUnxXN5gvZO1orw=; b=JxxGYg7Y9EhlBJhuyxV5Sbc2XD7KToOqvdbggiGMA9IZTuSZMrUkGUqOZNbO4ZawA4 eyzyrpMzwR5+iZckSkM/RTFCA+pzO8SPSq745T2VLJRK/NBXlJYdBbzN+TdJ/bca+hdc pV24tvktjzEF3crV0JWFS+P9zZq7fNwRdI3N43pN09GeWIc2M25se4DWXkCpjBQOcp6Q lnv7NNU62UcvWDgGz1ne8g4771m1tu6ZOIl3WwaIalLf5hVSQ9dM/B0hxgRWn9nLWXiS A7U7oRl76zzhpidcl1FeZgufQyv4wXiAaR8FRkLz3l1WCQ3cpi5yEOI5Pq8ChCMVbIUA 4K4Q== X-Forwarded-Encrypted: i=1; AJvYcCV41rRbQvGR4lEO+FinnNThcaeM39Ltb8Mht4MnTdh8QXZJaD0tfSBxH05sFwv1/BJsFCCzP9BFvo75/rbS@lists.postgresql.org X-Gm-Message-State: AOJu0YwgHV8XvIz6N/8g3oh3ml5wvckezAgd70KCKxzD4Q0cOQS14eDV kahEASDPtvg9GNG5u1YgVd9FyO/PaowWtRgrJ5ricjNPmrmAFo2Mkk9q2OVZGlr5T6Zeu5tVfkM gMq6CKsY+XUoq75CHEwu9mwI7gk/Rr9183lYiilU= X-Gm-Gg: ASbGncvzXhUCMmpq5AepdAsHpPdE+bQ4wkLG1wi5NaGq2pvYi8vmtQ8boFYYPaw5gwI riyCsDWF/X5vyLFVJAUH8QUWqwk9wQmJCBfn91A/lqzbcjWcmmM/VZCnOFOsmssx2tSmYEWNfSv fjJcadXdOTNmHskZhwRPCRDT9uwN3lDIa3VkIk5ElirUk6ezowLjZVaH/9Ul//2Q/Bp19SgyfyS /nn5ky1WSi/fNc6V2U= X-Google-Smtp-Source: AGHT+IEMfRYNU0LxAmMhxNpjImnIH+iA8x1T5AjR9lhUeFWxKfuzURdTrFRPvc6Qm9XYULjhO5U2xp8xc/mExUwtLKo= X-Received: by 2002:a05:6000:178d:b0:3b6:5d0d:9826 with SMTP id ffacd0b85a97d-3c46092ff5bmr294412f8f.21.1755711027675; Wed, 20 Aug 2025 10:30:27 -0700 (PDT) MIME-Version: 1.0 References: <05969854-0d19-4726-ae1b-586659dd443b@aklaver.com> In-Reply-To: From: Thom Brown Date: Wed, 20 Aug 2025 18:30:00 +0100 X-Gm-Features: Ac12FXyosVO1H8hv3ytE6szP2pLUvDm0BsCYiF9efxLpwB-g3YzxWS7665yCsKE Message-ID: Subject: Re: Streaming replica hangs periodically for ~ 1 second - how to diagnose/debug To: depesz@depesz.com Cc: Adrian Klaver , PostgreSQL General Content-Type: text/plain; charset="UTF-8" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Wed, 20 Aug 2025 at 17:08, hubert depesz lubaczewski wrote: > > On Wed, Aug 20, 2025 at 08:14:47AM -0700, Adrian Klaver wrote: > > Hmm. > > > > From initial post: > > > > "For ~ 1 second there are no logs going to log (we usually have at 5-20 > > messages logged per second), no connection, nothing. And then we get > > bunch (30+) messages with the same milisecond time." > > > > Are the 30+ messages all coming in on one connection or multiple > > connections? > > Multiple connections. > > > Also to be clear these are statements that are being run on the replica > > locally, correct? > > What do you mean locally? > > Application servers are *not* on the same server as the db. So no, they > are not local. > > > Does the AWS monitoring indicate any issues? > > Nope. All looks "fine". > > Same for our monitoring - CPU usage, iops, load. > > Best regards, > > depesz Do you have THP enabled? Can you use mpstat and see what %steal shows as? Thom