Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1iV1uL-0002e3-QL for pgsql-bugs@arkaria.postgresql.org; Wed, 13 Nov 2019 23:18:42 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1iV1uJ-00012x-Jh for pgsql-bugs@arkaria.postgresql.org; Wed, 13 Nov 2019 23:18:39 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1iV1uJ-00012q-7p for pgsql-bugs@lists.postgresql.org; Wed, 13 Nov 2019 23:18:39 +0000 Received: from mail-wm1-x343.google.com ([2a00:1450:4864:20::343]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1iV1uB-0003gt-9d for pgsql-bugs@lists.postgresql.org; Wed, 13 Nov 2019 23:18:38 +0000 Received: by mail-wm1-x343.google.com with SMTP id b11so3750013wmb.5 for ; Wed, 13 Nov 2019 15:18:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=2ndquadrant-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=5wuWJPyGooeADOJ8KkSw2BxXs/i/1zlo4qEFSqkjIv0=; b=WiieTVON78WE8WA8FwFkbs2HVIaS/xBdNoJ5AXaOc1/+PC77C1vM3vllk6pWKb52/r 2fVe5OaBJ7gYR7RdoQBWDTALFWOWSeARrzaf8xJTbWh9IQk5X3sdxJwDwVmh7c37RlSA rBES5uDLhSKdCtrCEta/czGzNxolZITb4brJXDLaCvHXK1A9wC6owVThoOtjEdCzwRtZ WYY7GamMyMvsp4s88odthjiColncyzMizlWXDn9dW3I2zKcu3D/THqT3+fjzfsgliR4B E0/BVqxSleOIohmho1e/Fbnf2fehiOUhi3H52f0vkjZQRYdZIVwVVzIwj57fX5Us9zBM YWeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=5wuWJPyGooeADOJ8KkSw2BxXs/i/1zlo4qEFSqkjIv0=; b=ifkdsFVEkomF+y+dpdLgbknEH3eHcwWx1YcLtAeXYPzrXAQdd8nGmq3UMEFHHD2jZg LIMzNmYkFRVfTikAm8ICI3oitD4wmcYG0dRqgNQPRkVmHa99l1HAM1uKGqb+btIdfa5c aJsVgExtoTlBOO+n9mD3uCH57D9rT0u8zfAsHGEfVJDLt1hW1t9d5pxJxAkyFjb/Ms6r gf7JuIXoP6af2/IDNDX704WYMQVK77j6kahEUE+FtCD7NgqZXgllN1UZLoemhQdSQPIO uEKXBBMGH3i7fU5MiySj/+RUwsWpXm3/D/2UGXcoK6dG7AZJ5OfRr3DDSCBhWNH2X1uA dTAg== X-Gm-Message-State: APjAAAVtoplRFK75tcHp0JjYLKsN3TTXQav6jblQyQicCmNqZzJvfYdz GRLhNxvDG57d8b7P62ueCRHp4w== X-Google-Smtp-Source: APXvYqxIQWIpC4YIGqdOO73IMOAi22Udrr4JSF5R/oXb3Xu91dwGNY8N8wrsdNBUJUQ/fJJTmioJBg== X-Received: by 2002:a05:600c:214a:: with SMTP id v10mr4922408wml.102.1573687109825; Wed, 13 Nov 2019 15:18:29 -0800 (PST) Received: from localhost (ip-86-49-253-92.net.upcbroadband.cz. [86.49.253.92]) by smtp.gmail.com with ESMTPSA id y17sm4939859wrs.58.2019.11.13.15.18.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Nov 2019 15:18:29 -0800 (PST) Date: Thu, 14 Nov 2019 00:18:27 +0100 From: Tomas Vondra To: Jeff Janes Cc: ben@lantern.is, pgsql-bugs Subject: Re: BUG #16112: large, unexpected memory consumption Message-ID: <20191113231827.dljqkbuf22v2gftf@development> References: <16112-4448bbf55a404189@postgresql.org> <20191113145004.opxx4vuajyoadbcq@development> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: User-Agent: NeoMutt/20180716 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On Wed, Nov 13, 2019 at 05:53:28PM -0500, Jeff Janes wrote: >On Wed, Nov 13, 2019 at 9:50 AM Tomas Vondra >wrote: > >> >> Yeah, I can reproduce this pretty easily. It seems like a memory leak in >> ExecMakeTableFunctionResult. a9c35cf85ca reworked FunctionCallInfo to be >> variable-length, but it gets allocated in ExecutorState context directly >> and so until the end of the query. >> > >I find the leak was introduced much earlier than that, in: > >commit 763f2edd92095b1ca2f4476da073a28505c13820 >Author: Andres Freund >Date: Thu Nov 15 14:26:14 2018 -0800 > > Rejigger materializing and fetching a HeapTuple from a slot. > >I have no idea if this info is useful to informing the best solution, >though. > Ah, I've only done a simple 'git blame' and assumed it's the last commit that touched the palloc line (and it seemed somewhat plausible). I don't think it changes the reasoning too much, though. >You patch applied to REL_12_STABLE does fix it for me. > > >> The attached trivial patch fixes that by adding a pfree() at the end of >> the function. I wonder if we have the same issue elsewhere ... >> >> >Is there an easy way to assess if the "make check" regression tests are >taking more memory than they used to? > Probably not. The regression tests use fairly small number of rows in general, and we don't have a way to track/inspect high watermaks or anything like that anyway. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services