Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jQkl0-0001d8-Jt for pgsql-bugs@arkaria.postgresql.org; Tue, 21 Apr 2020 04:43:39 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1jQkkz-0001TN-FD for pgsql-bugs@arkaria.postgresql.org; Tue, 21 Apr 2020 04:43:37 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jQkkH-00085P-72 for pgsql-bugs@lists.postgresql.org; Tue, 21 Apr 2020 04:42:54 +0000 Received: from out3-smtp.messagingengine.com ([66.111.4.27]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jQkkC-0006Zj-Gn for pgsql-bugs@postgresql.org; Tue, 21 Apr 2020 04:42:52 +0000 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id C6A395C01A6; Tue, 21 Apr 2020 00:42:45 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Tue, 21 Apr 2020 00:42:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=paquier.xyz; h= date:from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm1; bh=Xqhcry4Gw6iaUCFUZB77UXd2aST XE2ueHS7LpaA/8nY=; b=GvwlrdSEFCk5hoD6xHmGl5w8yOl5xGFlnGzLhDCOjC3 D1q3pV0QIyHMluk/J1NhLr3nseOuChEd0zUheccZGMnsVGzlkA2zFC6o5bQ98TU3 Mr2DOx2AsoaiWF2tRUDjENQB4rAK0FL95aMC+M4hR4Iyxd/B497RnZ+fzTwPUEnw UOrRz4eDX6lJW0PW9+87qYWGxXnqZJfokY7wq7Ib6waKM9ioMfQPzhLHa43CrHyi N38CGS0AvzxyDkr3F9q30hwa+0XP988fUB6Xlu1ig17UQilUN6yeXRQ2wWLv/ijr iqL5iJtPhs2xrAL+5rmgK0nEa6ao2KQ1H7Zg6r/o6EQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=Xqhcry 4Gw6iaUCFUZB77UXd2aSTXE2ueHS7LpaA/8nY=; b=cQ5C2zp0JqwFNLSAjKrowq l5HfUOC7JkfHhUHzA5KbMJ57c70HLEppW1jgG/7R046Fn/bSsygJtnkJ4NNvFIf6 Z/CbsS7HsfmkJ82g47NnjhrbudpY340ZzUh9h330IiOEvMSh1gI2d9RqdPl4BpR0 R4PcxlVW49tEkEzvCGAd0eKrtagrYmcPB4kfXaqplfp0JvVvFDV/SVNI358cagCq RAKBQhlEB1mLZfAMuf02+ZucwO9l/0c4KG3dEjkSU25xafmG09EpW1lHVpkohPcp lXOnS8PtpwU7YfpaRz/Cu0ZnvPxjoxQ3qpOV5XNfBLs9cLJrKz74vq+1Uy29QzFg == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrgeeggdekgecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenfg hrlhcuvffnffculddvfedmnecujfgurhepfffhvffukfhfgggtuggjsehgtderredttddv necuhfhrohhmpefoihgthhgrvghlucfrrghquhhivghruceomhhitghhrggvlhesphgrqh huihgvrhdrgiihiieqnecuffhomhgrihhnpehgihhthhhusgdrtghomhdpohhpvghnshhs lhdrohhrghenucfkphepudduuddruddtvddrudektddrudekheenucevlhhushhtvghruf hiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmihgthhgrvghlsehprghquhhi vghrrdighiii X-ME-Proxy: Received: from paquier.xyz (ee0822lan1.rev.em-net.ne.jp [111.102.180.185]) by mail.messagingengine.com (Postfix) with ESMTPA id 0E0103065C54; Tue, 21 Apr 2020 00:42:43 -0400 (EDT) Date: Tue, 21 Apr 2020 13:42:35 +0900 From: Michael Paquier To: Roman Peshkurov Cc: pgsql-bugs@postgresql.org Subject: Re: Bug with memory leak on cert validation in libpq Message-ID: <20200421044235.GA6436@paquier.xyz> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="VbJkn9YxBvnuCH5J" Content-Disposition: inline In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --VbJkn9YxBvnuCH5J Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Apr 20, 2020 at 08:40:06PM +0300, Roman Peshkurov wrote: > The problem is incorrect usage of openssl stack "destructor". We noticed = it > on a graphs for the few months. It looked lika an increasing line of res > memory usage. >=20 > Sorry, I've already made pull request (wasn't familiar with your flow). T= he > all information is described there: > https://github.com/postgres/postgres/pull/52 Thanks for taking the time to send an email here. I suspect that few people around here look at the cloned repo on github (I don't FWIW). If this reference goes away, then then community archives would lose a part of its history. So, first, here is the patch for reference: --- a/src/interfaces/libpq/fe-secure-openssl.c +++ b/src/interfaces/libpq/fe-secure-openssl.c @@ -552,7 +552,7 @@ pgtls_verify_peer_name_matches_certificate_guts(PGconn = *conn, if (rc !=3D 0) break; } - sk_GENERAL_NAME_free(peer_san); + sk_GENERAL_NAME_pop_free(peer_san, GENERAL_NAME_free); } Looking at the samples of the OpenSSL wiki, it looks that your guess is right: https://wiki.openssl.org/index.php/Hostname_validation After extracting the SAN data from the certificate using X509_get_ext_d2i(), we get to use sk_GENERAL_NAME_pop_free() for a correct in-depth free(). Now, we have in core a set of TAP tests in src/test/ssl with certificates that include some SAN configuration for sslmode=3Dverify-full, able to trigger the code path you are mentioning for pgtls_verify_peer_name_matches_certificate_guts() within fe-secure-openssl.c. In order to see the leak, (I have been lazy with the method and did not use valgrind with a custom C program or just psql, but the result is the same), I have put a custom hack in the code path involved here by adding a tight loop repeating X509_get_ext_d2i() + sk_GENERAL_NAME_free(), then ran the TAP tests of src/test/ssl/. After that, the leak becomes really clear after 5k loops or such in psql, with sk_GENERAL_NAME_pop_free() preventing the issue any rise of memory usage. sk_GENERAL_NAME_pop_free() exists down to OpenSSL 0.9.6, which is what I recall we officially support down to Postgres 9.5, so we are safe. And finally, this needs to be backpatched all the way down to 9.5, so I'll take care of it if there are no objections. Thanks for the report, Roman! -- Michael --VbJkn9YxBvnuCH5J Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAl6eebsACgkQnvQgOdby QH3iUA//aBk/Nehk9D/BsEsZ3ySrlEULxhY8akv2GoPlDmXjlcJdOtxbMb2zbDMV W+fl4arqk9ArCjuuLYP1FZGgrofTFJvqdQ1BNZIUheGhM77I57YM7lEccOVitaGe gmj3SinlHMeguxMHUPr/tYByI+bY0QQ5HoFuC2HjhqgKhQse5NLAg3MEBLJxeEv2 30tqQ4XEXMBTngvktPjDv9fOrtyy1v6KRpA+GxMKMRzarenwBvilp4bQpsY2gG2o 7d4rCUn1up6ChvHzsuPNI1qI+/jz3yvI78Mci4Ff7ZA5ZOEkfY0sL9P65NZnlVSR 4ds8RalzMqMa1+bznj3SUtLkgpWtgTWoaGWkLqX1boggDdE2E2PFcMXk+8y592DG 0KneuKDHjjSLq14Xvhn0l16TsDqCL/XzId+60fufZ8wGcy9HXXgDWN3BmoymySyc JwtupMMrN9n3g1p5Z/1ZyRmF52K+h99pmB9IXAFG1Zc3CPYiYPAxECnz84frtAtF 3HEDhrcUDC7h19ukVexL+/Rkkcmmarpm0Eh+wBUd/8SXLR68d1/ZaCu47wlUV75J YFx/KrqfSUCIWlwo6eaNnHoiWVWWse0aBpBnujwOPczRQs6tX6M+IBFIfag3LBya Xwq68vZnnGAY4MJ1wLq6nLbuIpqFMLZMo4kEujz3Oo451LkQgdM= =AYtA -----END PGP SIGNATURE----- --VbJkn9YxBvnuCH5J--