pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Andres Freund <andres@anarazel.de>
Cc: ben@lantern.is
Cc: pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #16112: large, unexpected memory consumption
Date: Wed, 13 Nov 2019 18:28:03 +0100
Message-ID: <20191113172803.iwyjaikvhdkvftee@development> (raw)
In-Reply-To: <20191113170528.caywphoaq45ggqbx@alap3.anarazel.de>
References: <16112-4448bbf55a404189@postgresql.org>
	<20191113145004.opxx4vuajyoadbcq@development>
	<20191113170528.caywphoaq45ggqbx@alap3.anarazel.de>

On Wed, Nov 13, 2019 at 09:05:28AM -0800, Andres Freund wrote:
>Hi,
>
>On 2019-11-13 15:50:04 +0100, Tomas Vondra wrote:
>> Yeah, I can reproduce this pretty easily. It seems like a memory leak in
>> ExecMakeTableFunctionResult. a9c35cf85ca reworked FunctionCallInfo to be
>> variable-length, but it gets allocated in ExecutorState context directly
>> and so until the end of the query.
>
>Damn. Too bad this got discovered just after the point release was
>wrapped :(
>
>
>> The attached trivial patch fixes that by adding a pfree() at the end of
>> the function.
>
>Hm. That's clearly an improvement. But I'm not quite sure it's really
>the right direction. It seems like a bad idea to rely on
>ExecMakeTableFunctionResult() otherwise never leaking any memory.
>
>It seems to we should be using memory contexts to provide a backstop
>against leaks, like we normally do, instead of operating in the
>per-query context. It's noteworthy that nodeProjectSet already does so.
>In contrast to nodeProjectSet, I think we'd need an additional memory
>context however, as we eagerly evaluate ValuePerCall expressions - and
>we'd like to reset the context after each iteration.
>

Possibly, but I think most of the allocations already use the per-tuple
context. It's just the fcinfo that seems to be allocated outside of it,
so maybe we should just move it to that memory context.

>
>I think we also ought to use SetExprState->fcinfo, instead of allocating
>a separate allocation in ExecMakeTableFunctionResult(). But that's
>perhaps less important.
>

Maybe.

>
>> I wonder if we have the same issue elsewhere ...
>
>Quite possible - but I think in most cases we are using memory contexts
>to protect against leaks like this (which is more efficient than retail
>pfree'ing).
>

Yeah, probably. I had a quick look at some of those places (found by
looking for SizeForFunctionCallInfo and palloc on the same line) and
those that I looked at seemed fine.

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 





view thread (16+ messages)  latest in thread

Message-ID: <20191113172803.iwyjaikvhdkvftee@development>
Permalink:  ../20191113172803.iwyjaikvhdkvftee@development/
Also on:    postgresql.org/message-id/20191113172803.iwyjaikvhdkvftee@development

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-bugs@postgresql.org
  Cc: tomas.vondra@2ndquadrant.com, andres@anarazel.de, ben@lantern.is, pgsql-bugs@lists.postgresql.org
  Subject: Re: BUG #16112: large, unexpected memory consumption
  In-Reply-To: <20191113172803.iwyjaikvhdkvftee@development>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox