Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lMiw9-0000VG-25 for pgsql-hackers@arkaria.postgresql.org; Thu, 18 Mar 2021 03:03:01 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1lMiw8-00066o-1D for pgsql-hackers@arkaria.postgresql.org; Thu, 18 Mar 2021 03:03:00 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lMiw7-00066h-1x for pgsql-hackers@lists.postgresql.org; Thu, 18 Mar 2021 03:02:59 +0000 Received: from wout1-smtp.messagingengine.com ([64.147.123.24]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lMiw2-0002tA-UZ for pgsql-hackers@lists.postgresql.org; Thu, 18 Mar 2021 03:02:58 +0000 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id E96271E72; Wed, 17 Mar 2021 23:02:51 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Wed, 17 Mar 2021 23:02:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=anarazel.de; h= date:from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm1; bh=fSY1GrApZXtqU3amf5qbwh7Bn0D 6tVzXoXOrGGvCksY=; b=iMpNEgMhofdwi74kaTg0UJ+TSSd6UirhxZH3d6c8rFh HtD7DkLycLu+E+zhm5gjzUUKkaxRu//gBKtx7tpNsLZv4BkSQRRn9jXdL5xkCLXw DaWbLe5YdnaycfLCdxihvpJH5UmOHgGUsphbz9d+gPdy5YZ9+TOCRjTR4nrOHfHZ wTLTq7WTN6Rj9s5pPhgaJ0PTJe0RF/nKt11SQ5YzfvlKmiKk+kDGeGE9B/qYiPtl ALXlO+OoSm6czlmpdrv5Du7e+Zxr8F3r8JYljlLl/kSPu4+m9vA+/czI6IqS+ftk lo4R1rcuV3BpKG++g/4BKraeMOOBhXi+GwlrEHJymKw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=fSY1Gr ApZXtqU3amf5qbwh7Bn0D6tVzXoXOrGGvCksY=; b=KAWFQaKqysV+qDOqjB/Qsz Bv/YbZQ7Tc71LOt63TR6k1eSgKHwxaufrhb6fjBWVOHALxPgOsk9DfdIKX4ZBgyn IXDqkPYZQwavt7cYc4eYSfjC4M7zOE3tuP8X7M71seYkqoPJe3Ql03f8lY5hQPpY VzwKs1zLv71GDByBkrrG59wdrmZPuVjHZYe19lamV9M+gtyEGDC3AGOBTBKzHKYL 0wvMLp2TK8BccV2iBQFSXJNmgD0TB4VuFo6m8g0MVOfsR+DkJnoMlMc7rSShvvTt KdESkwldo0V/2nBMnYnJdzNt9KP0N2qSrr5dYr+ZNgPFvrMQZsrhIR4Wxh/AbIOw == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudefhedghedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkfhggtggujgesthdtre dttddtvdenucfhrhhomheptehnughrvghsucfhrhgvuhhnugcuoegrnhgurhgvshesrghn rghrrgiivghlrdguvgeqnecuggftrfgrthhtvghrnhepudekhfekleeugeevteehleffff ejgeelueduleeffeeutdelffeujeffhfeuffdunecukfhppeeijedrudeitddrvddujedr vdehtdenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe grnhgurhgvshesrghnrghrrgiivghlrdguvg X-ME-Proxy: Received: from intern.anarazel.de (c-67-160-217-250.hsd1.ca.comcast.net [67.160.217.250]) by mail.messagingengine.com (Postfix) with ESMTPA id 3C256240057; Wed, 17 Mar 2021 23:02:51 -0400 (EDT) Date: Wed, 17 Mar 2021 20:02:50 -0700 From: Andres Freund To: Tom Lane Cc: pgsql-hackers@lists.postgresql.org Subject: Re: Getting better results from valgrind leak tracking Message-ID: <20210318030250.xlqxcppknzrakcs5@alap3.anarazel.de> References: <3471359.1615937770@sss.pgh.pa.us> <20210317023101.anvejcfotwka6gaa@alap3.anarazel.de> <3479700.1615950107@sss.pgh.pa.us> <3482163.1615953715@sss.pgh.pa.us> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3482163.1615953715@sss.pgh.pa.us> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk Hi, On 2021-03-17 00:01:55 -0400, Tom Lane wrote: > As for the particular point about ParallelBlockTableScanWorkerData, > I agree with your question to David about why that's in TableScanDesc > not HeapScanDesc, but I can't get excited about it not being freed in > heap_endscan. That's mainly because I do not believe that anything as > complex as a heap or indexscan should be counted on to be zero-leakage. > The right answer is to not do such operations in long-lived contexts. > So if we're running such a thing while switched into CacheContext, > *that* is the bug, not that heap_endscan didn't free this particular > allocation. I agree that it's a bad idea to do scans in non-transient contexts. It does however seems like there's a number of places that do... I added the following hacky definition of "permanent" contexts /* * NB: Only for assertion use. * * TopMemoryContext itself obviously is permanent. Treat CacheMemoryContext * and all its children as permanent too. * * XXX: Might be worth adding this as an explicit flag on the context? */ bool MemoryContextIsPermanent(MemoryContext c) { if (c == TopMemoryContext) return true; while (c) { if (c == CacheMemoryContext) return true; c = c->parent; } return false; } and checked that the CurrentMemoryContext is not permanent in SearchCatCacheInternal() and systable_beginscan(). Hit a number of times. The most glaring case is the RelationInitTableAccessMethod() call in RelationBuildLocalRelation(). Seems like the best fix is to just move the MemoryContextSwitchTo() to just before the RelationInitTableAccessMethod(). Although I wonder if we shouldn't go further, and move it to much earlier, somewhere after the rd_rel allocation. There's plenty other hits, but I think I should get back to working on making the shared memory stats patch committable. I really wouldn't want it to slip yet another year. But I think it might make sense to add a flag indicating contexts that shouldn't be used for non-transient data. Seems like we fairly regularly have "bugs" around this? Greetings, Andres Freund