Subject: Re: Creating a function for exposing memory usage of backend process
To: Andres Freund <andres@anarazel.de>, Tom Lane <tgl@sss.pgh.pa.us>
Cc: Michael Paquier <michael@paquier.xyz>,
 torikoshia <torikoshia@oss.nttdata.com>,
 PostgreSQL-development <pgsql-hackers@postgresql.org>,
 Robert Haas <robertmhaas@gmail.com>,
 Kasahara Tatsuhito <kasahara.tatsuhito@gmail.com>
References: <e687d9ec8ca6e7f387131a2455bc9e1f@oss.nttdata.com>
 <34517933-7e90-f644-f606-86d7132b240b@oss.nttdata.com>
 <0b339b06646e2c56a3aa081699322346@oss.nttdata.com>
 <f1ae4071-31b8-d7aa-db5d-2781e5612e6f@oss.nttdata.com>
 <261eebda3e1909475162b968e5a4ad2b@oss.nttdata.com>
 <66ae5b88-208d-96fa-c20a-fb539b597859@oss.nttdata.com>
 <20200819135545.GC19121@paquier.xyz> <1414992.1597849297@sss.pgh.pa.us>
 <20200819225343.c7rxteydi35nhrdp@alap3.anarazel.de>
 <1554676.1597886946@sss.pgh.pa.us>
 <20200820014301.5pr7qjugnvpovu62@alap3.anarazel.de>
From: Fujii Masao <masao.fujii@oss.nttdata.com>
Message-ID: <d97ad430-444d-9ec8-21dd-a1584569309b@oss.nttdata.com>
Date: Thu, 20 Aug 2020 11:17:58 +0900
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:68.0)
 Gecko/20100101 Thunderbird/68.11.0
MIME-Version: 1.0
In-Reply-To: <20200820014301.5pr7qjugnvpovu62@alap3.anarazel.de>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk


On 2020/08/20 10:43, Andres Freund wrote:
> Hi,
> 
> On 2020-08-19 21:29:06 -0400, Tom Lane wrote:
>> Andres Freund <andres@anarazel.de> writes:
>>> On 2020-08-19 11:01:37 -0400, Tom Lane wrote:
>>>> I agree with that, but I think this patch has a bigger problem:
>>>> why bother at all?  It seems like a waste of code space and future
>>>> maintenance effort, because there is no use-case.
>>
>>> I don't agree with this at all. I think there's plenty use cases. It's
>>> e.g. very common to try to figure out why the memory usage of a process
>>> is high. Is it memory not returned to the OS? Is it caches that have
>>> grown too much etc.
>>
>> Oh, I agree completely that there are lots of use-cases for finding
>> out what a process' memory map looks like.  But this patch fails to
>> address any of them in a usable way.
> 
> Even just being able to see the memory usage in a queryable way is a
> huge benefit.

+1

> The difference over having to parse the log, then parse
> the memory usage dump, and then aggregate the data in there in a
> meaningful way is *huge*.  We've been slacking around lowering our
> memory usage, and I think the fact that it's annoying to analyze is a
> partial reason for that.

Agreed.

  
> I totally agree that it's not *enough*, but in contrast to you I think
> it's a good step. Subsequently we should add a way to get any backends
> memory usage.
> It's not too hard to imagine how to serialize it in a way that can be
> easily deserialized by another backend. I am imagining something like
> sending a procsignal that triggers (probably at CFR() time) a backend to
> write its own memory usage into pg_memusage/<pid> or something roughly
> like that.

Sounds good. Maybe we can also provide the SQL-callable function
or view to read pg_memusage/<pid>, to make the analysis easier.

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION