Date: Tue, 10 Nov 2020 09:09:13 +0100
From: Kurt Roeckx <kurt@roeckx.be>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #16707: Memory leak
Message-ID: <20201110080913.GD391173@roeckx.be>
References: <16707-f5df308978a55bf8@postgresql.org>
 <1999972.1604957673@sss.pgh.pa.us>
 <20201109220329.GC391173@roeckx.be>
 <2002012.1604960437@sss.pgh.pa.us>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <2002012.1604960437@sss.pgh.pa.us>
Precedence: bulk

On Mon, Nov 09, 2020 at 05:20:37PM -0500, Tom Lane wrote:
> Kurt Roeckx <kurt@roeckx.be> writes:
> > On Mon, Nov 09, 2020 at 04:34:33PM -0500, Tom Lane wrote:
> >> * look in postmaster log for memory dump
> 
> > ...
> > Grand total: 3575000 bytes in 533 blocks; 596232 free (450 chunks); 2978768 used
> 
> > Which was for this process:
> > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> > postgres 10000  2.6 16.3 5547172 5374656 ?     Ss   Nov08  54:10 postgres: synapse synapse [local] idle
> 
> Hm.  It would seem that whatever you're leaking was not allocated via
> palloc.  Have you got any extensions loaded into that backend?

synapse=> \dx
                 List of installed extensions
  Name   | Version |   Schema   |         Description
---------+---------+------------+------------------------------
 plpgsql | 1.0     | pg_catalog | PL/pgSQL procedural language
(1 row)

> It's also worth noting that if you've got 4GB of shared buffers,
> a total process vsize of 5.3GB doesn't seem all that far out of
> line.  I'm not quite convinced that you have a leak at all,
> as opposed to processes gradually touching more and more of the
> shared buffer arena.

Top says the shared size is now 4.1 GB, and has said so for a
while. On the other hand, the virtual size keeps growning. The
virtual size shouldn't change much over time. The resident size
will grow until the shared buffer is fully used, then should stay
around that value.

ps now says:
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
postgres 10000  2.2 16.6 5632572 5468232 ?     Ss   Nov08  61:09 postgres: synapse synapse [local] idle

At some point all 32 GB of RAM will get used, it starts to swap
out, then malloc starts to fail. (I've set Linux to not
allow overcommit (vm.overcommit_memory = 2), otherwise it would be
the OOM killer doing it's thing.

This is what an other process looks like:
postgres  8966  0.2  3.8 4387236 1276104 ?     Ss   Nov08   7:10 postgres: kurt certs [local] idle

top says that that process has 1.2 GB shared RAM, which also
matches it resident size.


Kurt