Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1q9bO4-0007p8-TM for pgsql-bugs@arkaria.postgresql.org; Thu, 15 Jun 2023 01:02:57 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1q9bO3-0004Qb-Il for pgsql-bugs@arkaria.postgresql.org; Thu, 15 Jun 2023 01:02:55 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1q9bO2-0004QR-Ke for pgsql-bugs@lists.postgresql.org; Thu, 15 Jun 2023 01:02:55 +0000 Received: from out1-smtp.messagingengine.com ([66.111.4.25]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1q9bNy-002Qxc-T6 for pgsql-bugs@lists.postgresql.org; Thu, 15 Jun 2023 01:02:54 +0000 Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id 3344E5C006D; Wed, 14 Jun 2023 21:02:47 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Wed, 14 Jun 2023 21:02:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=paquier.xyz; h= cc:cc:content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm2; t=1686790967; x=1686877367; bh=pP DiczvIYW6zmLX31EF9q0h9F51qpGJogiOrhiuCZik=; b=AmSCtxyeVlhe9i9nXO iuM9THrhrwedOBF6Sjpd7m5ZkKwF4uw0fuhNelKq0sgKBvqWpRyNLUOhmANUH6Mz cbgEgALoATidE+wqThg1/OqaipVuQvduPIOdTgREnlaOIFYT2Va1lzxKGCq0fL94 5wJYEW7YtMgbgWyrpqZ6esmYgrBaaJ7OI0nB6XNOGiPGXvRCVZy9vGNbRssPK0x0 8BbRtJgpOoflWBvabh4vNNW9MOnJaiVBmJ/2kVb2Frn8rQl8UfXu3kMP66NvRgt/ M6ydKAuXn0SV3UsFZ6ixGGC2olV9YhgFMAlSkG7QzLsLvsiyE5CLR6xQLQHkf14O EBvw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; t=1686790967; x=1686877367; bh=pPDiczvIYW6zm LX31EF9q0h9F51qpGJogiOrhiuCZik=; b=DAaaryWerFmSzs2pvx9hFM88t8ydP ktVNLBJfTktfYtOGuWsNY/P5e5VM7puq9igRGV2FUAe6f8T3a41YgvhARrieFInN Zcu7ld0a0L9/VWAx7eZxF51H7putkO/BS/rDBfQTDYJceV7ob1XFsm9YSlJ6+d98 /AW7G0ODBz0Gxwx4iv8RPyFoRiuWBhtazxlibyAq8p45S5IQzxHs2CH37P0OKPSA 62DTb53Szmvbc1cziswC3AsVvhdV3r4ghzPtY+PQXq8y7+ZpMK9EMpft52b9Hrdd smDuYU2J7+BnxbjZBu0JdFhtFwsrFsor0Rx6WNxZ8TLsv9uzFnyuTTNag== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrgedvuddggedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne gfrhhlucfvnfffucdlvdefmdenucfjughrpeffhffvvefukfhfgggtuggjsehgtderredt tdejnecuhfhrohhmpefoihgthhgrvghlucfrrghquhhivghruceomhhitghhrggvlhesph grqhhuihgvrhdrgiihiieqnecuggftrfgrthhtvghrnhepjeeigeehveejvdegkedtgeev jeeugeehleejgeegffdvhfejfeffiefgjeejvdefnecuffhomhgrihhnpehpohhsthhgrh gvshhqlhdrohhrghenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhl fhhrohhmpehmihgthhgrvghlsehprghquhhivghrrdighiii X-ME-Proxy: Feedback-ID: i0fe9450f:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 14 Jun 2023 21:02:44 -0400 (EDT) Date: Thu, 15 Jun 2023 10:02:39 +0900 From: Michael Paquier To: Michael Guissine Cc: Andres Freund , pgsql-bugs@lists.postgresql.org Subject: Re: BUG #17974: Walsenders memory usage suddenly spike to 80G+ causing OOM and server reboot Message-ID: References: <17974-f8c9d353a62f414d@postgresql.org> <20230614221503.ux62wn25bhydnzjm@awork3.anarazel.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="Z8mxvj6g6TFYvoHd" Content-Disposition: inline In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --Z8mxvj6g6TFYvoHd Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jun 14, 2023 at 08:04:53PM -0400, Michael Guissine wrote: > On Wed, Jun 14, 2023 at 6:15=E2=80=AFPM Andres Freund wrote: >>>> Any known issues in pg 14.8 that would trigger this behaviour? >>> >>> Yes, there are known issues with memory handling in logical >>> replication setups. See for example this thread: >>> >>> https://www.postgresql.org/message-id/CAMnUB3oYugXCBLSkih+qNsWQPciEwos6= g_AMbnz_peNoxfHwyw@mail.gmail.com >> >> Why would 14.8 have made that problem worse? Hmm? I don't necessarily imply that 14.8 made it worse, but that there are known issues in this area as the other thread mention (which is mainly about v15, actually, where the situation could be a bit better). > This is an assumption obviously, based on the fact that we have been > running this database for several years already and haven't seen similar > memory issues until the recent upgrade to 14.8. Was that all the time with 14 as base? What are the minor versions you used in the past? Perhaps there is a correlation here that could help a bit. There have been a few things in this area lately, like 9f2cc1a, 4fbe609, 68dcce2, fd270b7 or 8b9cbd4 (this one made the decoding cheaper), so it could be possible that a defect has been introduced. Being able to know the contexts where the memory is piling up would help a lot, for sure. -- Michael --Z8mxvj6g6TFYvoHd Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAmSKYy8ACgkQnvQgOdby QH0gHw//ac4Q7aItPMcEqIER+tUNIN6WbaPoJ0LPpS3QMNIu8pZPPwm4kKR2iMZQ ZNWHFDj1GZbVyqHd7hAhcELXTxM/7AlhOR+KgDbjnV/JLFsU7KgHembOfuFQ787S h1gNZlIoLOhqANXQWnoqQYzeSUAOJebEMYw4PIszkUpV4a92CzSeKk+iFltyUhkb s/RZpk/w5yqsVTVPkVcfVOgn/5QGdlB74FDwEvTJ39SJukOi/dr81yEC2Jd+RQ5N 2lM9b/+moc94FyaSa9fz3Gv1czGhBKZ+oqXGtpPHELrLidns5OHgLAPpQ+U2RJWz qTLGDCEkn4uYQFwO/tbxfmL7rnHt510xOYonWmxOtidlUvruDU/gZfXa8vJHlHd4 1+c9P1JFFIiTmYpolN80OAv2/la4sjTnGukMABSSvxHHlv4QJcDO6vPnRMRz3h+x bee7SVT+koIGfwNOWiY1paNtA/AST8rz4XYPIQFC6XBdok1H7cvFWTzFHqjMRQKb ehJvvUH3xbz3v08IjradroM203FQsNFhgZ0d0/M3P/EK5PuL/YfSbGX4yGEOps4W PR+4CfDe4LC7tDcnfi0IVAUYuyTBJ9DgSQAtr4q/b1DaWkW17YR0z90awK9DKHIV pqox5cRQe3iWVTHmtDrq8dMIn8rN/pXFiwnAcwpbNINzc+ZzCwo= =/rA1 -----END PGP SIGNATURE----- --Z8mxvj6g6TFYvoHd--