Subject: Re: Get memory contexts of an arbitrary backend process
To: torikoshia <torikoshia@oss.nttdata.com>,
 PostgreSQL-development <pgsql-hackers@postgresql.org>
Cc: Tom Lane <tgl@sss.pgh.pa.us>,
 Georgios Kokolatos <gkokolatos@protonmail.com>,
 Kasahara Tatsuhito <kasahara.tatsuhito@gmail.com>, craig@2ndquadrant.com
References: <0271f440ac77f2a4180e0e56ebd944d1@oss.nttdata.com>
 <270ec80dbbebfb5c6f2b9fdd7a7858b5@oss.nttdata.com>
 <160502083340.7362.4386116585349115146.pgcf@coridan.postgresql.org>
 <064d510f2d33edc37dfd6b8f3b6bd30e@oss.nttdata.com>
 <d42d3186-d647-9af4-7c1c-d6ffca8bff40@oss.nttdata.com>
 <1685539.1606959410@sss.pgh.pa.us>
 <83ff0aff04fffc95cbd1fc1637a0fccc@oss.nttdata.com>
 <28e1feb71b94ac9918830bc4a0c1844f@oss.nttdata.com>
 <CAP0=ZV+XcPnX+m9ROG07hs=DqwQCOLAojgvYCRogw=bFAv1+Eg@mail.gmail.com>
 <9a59401241e9269254e70c8e0ef1cc7a@oss.nttdata.com>
 <4ae7b086a5b49fdfec7c1154d6fc7a40@oss.nttdata.com>
 <40c1a4b19ad03beacee2642a3cd61c14@oss.nttdata.com>
 <9a50371e15e741e295accabc72a41df1@oss.nttdata.com>
 <bea016ad-d1a7-f01d-a7e8-01106a1de77f@oss.nttdata.com>
 <70ae4b79eb8b0dcf42161c80a00e3f22@oss.nttdata.com>
From: Fujii Masao <masao.fujii@oss.nttdata.com>
Message-ID: <49232304-1269-b2d4-c2ca-4b8683b29156@oss.nttdata.com>
Date: Thu, 18 Mar 2021 15:09:28 +0900
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:78.0)
 Gecko/20100101 Thunderbird/78.8.1
MIME-Version: 1.0
In-Reply-To: <70ae4b79eb8b0dcf42161c80a00e3f22@oss.nttdata.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Archived-At: 
 <https://www.postgresql.org/message-id/49232304-1269-b2d4-c2ca-4b8683b29156%40oss.nttdata.com>
Precedence: bulk


On 2021/03/17 22:24, torikoshia wrote:
> I remade the patch and introduced a function
> pg_print_backend_memory_contexts(PID) which prints the memory contexts of
> the specified PID to elog.

Thanks for the patch!


>    =# SELECT pg_print_backend_memory_contexts(450855);
> 
>    ** log output **
>    2021-03-17 15:21:01.942 JST [450855] LOG:  Printing memory contexts of PID 450855
>    2021-03-17 15:21:01.942 JST [450855] LOG:  level: 0 TopMemoryContext: 68720 total in 5 blocks; 16312 free (15 chunks); 52408 used
>    2021-03-17 15:21:01.942 JST [450855] LOG:  level: 1 Prepared Queries: 65536 total in 4 blocks; 35088 free (14 chunks); 30448 used
>    2021-03-17 15:21:01.942 JST [450855] LOG:  level: 1 pgstat TabStatusArray lookup hash table: 8192 total in 1 blocks; 1408 free (0 chunks); 6784 used
>    ..(snip)..
>    2021-03-17 15:21:01.942 JST [450855] LOG:  level: 2 CachedPlanSource: 4096 total in 3 blocks; 680 free (0 chunks); 3416 used: PREPARE hoge_200 AS SELECT * FROM pgbench_accounts WHERE aid = 1111111111111111111111111111111111111...
>    2021-03-17 15:21:01.942 JST [450855] LOG:  level: 3 CachedPlanQuery: 4096 total in 3 blocks; 464 free (0 chunks); 3632 used
>    ..(snip)..
>    2021-03-17 15:21:01.945 JST [450855] LOG:  level: 1 Timezones: 104128 total in 2 blocks; 2584 free (0 chunks); 101544 used
>    2021-03-17 15:21:01.945 JST [450855] LOG:  level: 1 ErrorContext: 8192 total in 1 blocks; 7928 free (5 chunks); 264 used
>    2021-03-17 15:21:01.945 JST [450855] LOG:  Grand total: 2802080 bytes in 1399 blocks; 480568 free (178 chunks); 2321512 used
> 
> 
> As above, the output is almost the same as MemoryContextStatsPrint()
> except for the way of expression of the level.
> MemoryContextStatsPrint() uses indents, but
> pg_print_backend_memory_contexts() writes it as "level: %d".

This format looks better to me.


> Since there was discussion about enlarging StringInfo may cause
> errors on OOM[1], this patch calls elog for each context.
> 
> As with MemoryContextStatsPrint(), each context shows 100
> children at most.
> I once thought it should be configurable, but something like
> pg_print_backend_memory_contexts(PID, num_children) needs to send
> the 'num_children' from requestor to dumper and it seems to require
> another infrastructure.
> Creating a new GUC for this seems overkill.
> If MemoryContextStatsPrint(), i.e. showing 100 children at most is
> enough, this hard limit may be acceptable.

Can't this number be passed via shared memory?


> Only superusers can call pg_print_backend_memory_contexts().

+	/* Only allow superusers to signal superuser-owned backends. */
+	if (superuser_arg(proc->roleId) && !superuser())

The patch seems to allow even non-superuser to request to print the memory
contexts if the target backend is owned by non-superuser. Is this intentional?
I think that only superuser should be allowed to execute
pg_print_backend_memory_contexts() whoever owns the target backend.
Because that function can cause lots of log messages.


> I'm going to add documentation and regression tests.

Thanks!

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION