Date: Tue, 10 Nov 2020 20:50:39 +0100
From: Kurt Roeckx <kurt@roeckx.be>
To: Andres Freund <andres@anarazel.de>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #16707: Memory leak
Message-ID: <20201110195039.GG391173@roeckx.be>
References: <16707-f5df308978a55bf8@postgresql.org>
 <1999972.1604957673@sss.pgh.pa.us>
 <20201109220329.GC391173@roeckx.be>
 <2002012.1604960437@sss.pgh.pa.us>
 <20201110043127.sfkyvvjqy7x3er5k@alap3.anarazel.de>
 <20201110081120.GF391173@roeckx.be>
 <20201110193517.t5hbqkya67gb23rq@alap3.anarazel.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20201110193517.t5hbqkya67gb23rq@alap3.anarazel.de>
Precedence: bulk

On Tue, Nov 10, 2020 at 11:35:17AM -0800, Andres Freund wrote:
> Hi,
> 
> On 2020-11-10 09:11:20 +0100, Kurt Roeckx wrote:
> > On Mon, Nov 09, 2020 at 08:31:27PM -0800, Andres Freund wrote:
> > > As this is on a halfway recent linux, I suggest doing something like
> > > 
> > > $ grep ^Rss /proc/$pid/status
> > > RssAnon:	    6664 kB
> > > RssFile:	   69512 kB
> > > RssShmem:	   15788 kB
> > 
> > RssAnon:         1197064 kB
> > RssFile:           27420 kB
> > RssShmem:        4248052 kB
> 
> Ok, so it's actual allocations that are the problem. What kind of
> queries is this workload running?

I really have no idea, but I'll try to get an idea if the jit
thing doesn't work.

> There's one known (slow) memory leak in the JIT code / LLVM. Could you
> check if the issue vanishes if you disable JIT (jit = 0)?

I've just restarted it with jit = 0.

> Otherwise it might be useful to collect stack traces for memory
> allocations. You could try something like 'heaptrack' or add a perf
> probe on malloc, and do a perf profile.
> 
> E.g. something like
> perf probe -x /lib/x86_64-linux-gnu/libc.so.6 -a malloc
> perf record -e probe_libc:malloc --call-graph dwarf -p $pid_of_problematic_process

Let's first see what happens with jit disabled. If I still see it,
I'll try that.


Kurt