From: torikoshia <torikoshia@oss.nttdata.com>
To: Fujii Masao <masao.fujii@oss.nttdata.com>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: pgsql-hackers@postgresql.org
Cc: tgl@sss.pgh.pa.us
Cc: gkokolatos@protonmail.com
Cc: kasahara.tatsuhito@gmail.com
Cc: craig@2ndquadrant.com
Subject: Re: Get memory contexts of an arbitrary backend process
Date: Wed, 31 Mar 2021 15:02:02 +0900
Message-ID: <f6f6373ae200c8a0e7466e4de9ca65eb@oss.nttdata.com> (raw)
In-Reply-To: <1f259e90-08c8-2202-be66-f7ee05d22cd2@oss.nttdata.com>
References: <20210326.131721.583838520933682740.horikyota.ntt@gmail.com>
<20210326.132836.985270106396322831.horikyota.ntt@gmail.com>
<e4cffb70-a672-fc7f-ef41-4c979c509832@oss.nttdata.com>
<20210326.140851.1169798659519899419.horikyota.ntt@gmail.com>
<fd78be66959cbf3957a329bb069ab737@oss.nttdata.com>
<168dbd34-c081-b832-e795-ffcd7c5be35c@oss.nttdata.com>
<bbecd722d4f8e261b350186ac4bf68a7@oss.nttdata.com>
<1f259e90-08c8-2202-be66-f7ee05d22cd2@oss.nttdata.com>
On 2021-03-31 04:36, Fujii Masao wrote:
> On 2021/03/30 22:06, torikoshia wrote:
>> Modified the patch according to the suggestions.
>
> Thanks for updating the patch!
>
> I applied the cosmetic changes to the patch and added the example of
> the function call into the document. Attached is the updated version
> of the patch. Could you check this version?
>
Thanks a lot!
+The memory contexts will be logged in the log file. For example:
When 'log_destination = stderr' and 'logging_collector = off', it does
not log in the file but in the stderr.
Description like below would be a bit more accurate but I'm wondering
it's repeating the same words.
+ The memory contexts will be logged based on the log configuration set.
For example:
How do you think?
+<programlisting>
+postgres=# SELECT pg_log_backend_memory_contexts(pg_backend_pid());
+ pg_log_backend_memory_contexts
+--------------------------------
+ t
+(1 row)
+
+The memory contexts will be logged in the log file. For example:
+LOG: logging memory contexts of PID 10377
+STATEMENT: SELECT pg_log_backend_memory_contexts(pg_backend_pid());
+LOG: level: 0; TopMemoryContext: 80800 total in 6 blocks; 14432 free
(5 chunks); 66368 used
+LOG: level: 1; pgstat TabStatusArray lookup hash table: 8192 total in
1 blocks; 1408 free (0 chunks); 6784 used
The line "The memory contexts will be logged in the log file. For
example:"
is neither nor SQL command and its outputs, it might be better to
differentiate it.
What about the following like attached patch?
+<programlisting>
+postgres=# SELECT pg_log_backend_memory_contexts(pg_backend_pid());
+ pg_log_backend_memory_contexts
+--------------------------------
+ t
+(1 row)
+</programlisting>
+The memory contexts will be logged in the log file. For example:
+<screen>
+LOG: logging memory contexts of PID 10377
+STATEMENT: SELECT pg_log_backend_memory_contexts(pg_backend_pid());
+LOG: level: 0; TopMemoryContext: 80800 total in 6 blocks; 14432 free
(5 chunks); 66368 used
+LOG: level: 1; pgstat TabStatusArray lookup hash table: 8192 total in
1 blocks; 1408 free (0 chunks); 6784 used
...(snip)...
+LOG: level: 1; ErrorContext: 8192 total in 1 blocks; 7928 free (3
chunks); 264 used
+LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88
chunks); 1029560 used
+</screen>
Regards.
Attachments:
[text/x-diff] v8-0001-add-memorycontext-elog-print.patch (26.2K, ../f6f6373ae200c8a0e7466e4de9ca65eb@oss.nttdata.com/2-v8-0001-add-memorycontext-elog-print.patch)
download | inline diff:diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgmlindex fbf6062d0a..ce01d51b21 100644--- a/doc/src/sgml/func.sgml+++ b/doc/src/sgml/func.sgml@@ -24917,6 +24917,23 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>+ <entry role="func_table_entry"><para role="func_signature">+ <indexterm>+ <primary>pg_log_backend_memory_contexts</primary>+ </indexterm>+ <function>pg_log_backend_memory_contexts</function> ( <parameter>pid</parameter> <type>integer</type> )+ <returnvalue>boolean</returnvalue>+ </para>+ <para>+ Logs the memory contexts whose backend process has the specified+ process ID.+ Memory contexts will be logged based on the log configuration set.+ See <xref linkend="runtime-config-logging"/> for more information.+ Only superusers can log the memory contexts.+ </para></entry>+ </row>+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -24987,6 +25004,36 @@ SELECT collation for ('foo' COLLATE "de_DE");
<structname>pg_stat_activity</structname> view.
</para>
+ <para>+ <function>pg_log_backend_memory_contexts</function> can be used+ to log the memory contexts of the backend process. For example,+<programlisting>+postgres=# SELECT pg_log_backend_memory_contexts(pg_backend_pid());+ pg_log_backend_memory_contexts +--------------------------------+ t+(1 row)+</programlisting>+The memory contexts will be logged in the log file. For example:+<screen>+LOG: logging memory contexts of PID 10377+STATEMENT: SELECT pg_log_backend_memory_contexts(pg_backend_pid());+LOG: level: 0; TopMemoryContext: 80800 total in 6 blocks; 14432 free (5 chunks); 66368 used+LOG: level: 1; pgstat TabStatusArray lookup hash table: 8192 total in 1 blocks; 1408 free (0 chunks); 6784 used+LOG: level: 1; TopTransactionContext: 8192 total in 1 blocks; 7720 free (1 chunks); 472 used+LOG: level: 1; RowDescriptionContext: 8192 total in 1 blocks; 6880 free (0 chunks); 1312 used+LOG: level: 1; MessageContext: 16384 total in 2 blocks; 5152 free (0 chunks); 11232 used+LOG: level: 1; Operator class cache: 8192 total in 1 blocks; 512 free (0 chunks); 7680 used+LOG: level: 1; smgr relation table: 16384 total in 2 blocks; 4544 free (3 chunks); 11840 used+LOG: level: 1; TransactionAbortContext: 32768 total in 1 blocks; 32504 free (0 chunks); 264 used+...+LOG: level: 1; ErrorContext: 8192 total in 1 blocks; 7928 free (3 chunks); 264 used+LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560 used+</screen>+ For more than 100 child contexts under the same parent one,+ 100 child contexts and a summary of the remaining ones will be logged.+ </para>+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.cindex c6a8d4611e..eac6895141 100644--- a/src/backend/storage/ipc/procsignal.c+++ b/src/backend/storage/ipc/procsignal.c@@ -30,6 +30,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "utils/memutils.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -657,6 +658,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_BARRIER))
HandleProcSignalBarrierInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))+ HandleLogMemoryContextInterrupt();+
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_DATABASE))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_DATABASE);
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.cindex 2b1b68109f..afaf3b1cce 100644--- a/src/backend/tcop/postgres.c+++ b/src/backend/tcop/postgres.c@@ -3295,6 +3295,9 @@ ProcessInterrupts(void)
if (ParallelMessagePending)
HandleParallelMessages();
++ if (LogMemoryContextPending)+ ProcessLogMemoryContextInterrupt();
}
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.cindex c02fa47550..fe9b7979e2 100644--- a/src/backend/utils/adt/mcxtfuncs.c+++ b/src/backend/utils/adt/mcxtfuncs.c@@ -18,6 +18,8 @@
#include "funcapi.h"
#include "miscadmin.h"
#include "mb/pg_wchar.h"
+#include "storage/proc.h"+#include "storage/procarray.h"
#include "utils/builtins.h"
/* ----------
@@ -61,7 +63,7 @@ PutMemoryContextsStatsTupleStore(Tuplestorestate *tupstore,
/* Examine the context itself */
memset(&stat, 0, sizeof(stat));
- (*context->methods->stats) (context, NULL, (void *) &level, &stat);+ (*context->methods->stats) (context, NULL, (void *) &level, &stat, true);
memset(values, 0, sizeof(values));
memset(nulls, 0, sizeof(nulls));
@@ -155,3 +157,59 @@ pg_get_backend_memory_contexts(PG_FUNCTION_ARGS)
return (Datum) 0;
}
++/*+ * pg_log_backend_memory_contexts+ * Signal a backend process to log its memory contexts.+ *+ * Only superusers are allowed to signal to log the memory contexts+ * because allowing any users to issue this request at an unbounded+ * rate would cause lots of log messages and which can lead to+ * denial of service.+ *+ * On receipt of this signal, a backend sets the flag in the signal+ * handler, and then which causes the next CHECK_FOR_INTERRUPTS()+ * to log the memory contexts.+ */+Datum+pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)+{+ int pid = PG_GETARG_INT32(0);+ PGPROC *proc = BackendPidGetProc(pid);++ /*+ * BackendPidGetProc returns NULL if the pid isn't valid; but by the time+ * we reach kill(), a process for which we get a valid proc here might+ * have terminated on its own. There's no way to acquire a lock on an+ * arbitrary process to prevent that. But since this mechanism is usually+ * used to debug a backend running and consuming lots of memory, that it+ * might end on its own first and its memory contexts are not logged is+ * not a problem.+ */+ if (proc == NULL)+ {+ /*+ * This is just a warning so a loop-through-resultset will not abort+ * if one backend logged its memory contexts during the run.+ */+ ereport(WARNING,+ (errmsg("PID %d is not a PostgreSQL server process", pid)));+ PG_RETURN_BOOL(false);+ }++ /* Only allow superusers to log memory contexts. */+ if (!superuser())+ ereport(ERROR,+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),+ errmsg("must be a superuser to log memory contexts")));++ if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, proc->backendId) < 0)+ {+ /* Again, just a warning to allow loops */+ ereport(WARNING,+ (errmsg("could not send signal to process %d: %m", pid)));+ PG_RETURN_BOOL(false);+ }++ PG_RETURN_BOOL(true);+}diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.cindex 73e0a672ae..6c27065f96 100644--- a/src/backend/utils/init/globals.c+++ b/src/backend/utils/init/globals.c@@ -34,6 +34,7 @@ volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t LogMemoryContextPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/backend/utils/mmgr/aset.c b/src/backend/utils/mmgr/aset.cindex ec6c130d0f..77872e77bc 100644--- a/src/backend/utils/mmgr/aset.c+++ b/src/backend/utils/mmgr/aset.c@@ -272,7 +272,8 @@ static Size AllocSetGetChunkSpace(MemoryContext context, void *pointer);
static bool AllocSetIsEmpty(MemoryContext context);
static void AllocSetStats(MemoryContext context,
MemoryStatsPrintFunc printfunc, void *passthru,
- MemoryContextCounters *totals);+ MemoryContextCounters *totals,+ bool print_to_stderr);
#ifdef MEMORY_CONTEXT_CHECKING
static void AllocSetCheck(MemoryContext context);
@@ -1336,11 +1337,12 @@ AllocSetIsEmpty(MemoryContext context)
* printfunc: if not NULL, pass a human-readable stats string to this.
* passthru: pass this pointer through to printfunc.
* totals: if not NULL, add stats about this context into *totals.
+ * print_to_stderr: print stats to stderr if true, elog otherwise.
*/
static void
AllocSetStats(MemoryContext context,
MemoryStatsPrintFunc printfunc, void *passthru,
- MemoryContextCounters *totals)+ MemoryContextCounters *totals, bool print_to_stderr)
{
AllocSet set = (AllocSet) context;
Size nblocks = 0;
@@ -1379,7 +1381,7 @@ AllocSetStats(MemoryContext context,
"%zu total in %zd blocks; %zu free (%zd chunks); %zu used",
totalspace, nblocks, freespace, freechunks,
totalspace - freespace);
- printfunc(context, passthru, stats_string);+ printfunc(context, passthru, stats_string, print_to_stderr);
}
if (totals)
diff --git a/src/backend/utils/mmgr/generation.c b/src/backend/utils/mmgr/generation.cindex 2b90034764..584cd614da 100644--- a/src/backend/utils/mmgr/generation.c+++ b/src/backend/utils/mmgr/generation.c@@ -155,7 +155,8 @@ static Size GenerationGetChunkSpace(MemoryContext context, void *pointer);
static bool GenerationIsEmpty(MemoryContext context);
static void GenerationStats(MemoryContext context,
MemoryStatsPrintFunc printfunc, void *passthru,
- MemoryContextCounters *totals);+ MemoryContextCounters *totals,+ bool print_to_stderr);
#ifdef MEMORY_CONTEXT_CHECKING
static void GenerationCheck(MemoryContext context);
@@ -665,6 +666,7 @@ GenerationIsEmpty(MemoryContext context)
* printfunc: if not NULL, pass a human-readable stats string to this.
* passthru: pass this pointer through to printfunc.
* totals: if not NULL, add stats about this context into *totals.
+ * print_to_stderr: print stats to stderr if true, elog otherwise.
*
* XXX freespace only accounts for empty space at the end of the block, not
* space of freed chunks (which is unknown).
@@ -672,7 +674,7 @@ GenerationIsEmpty(MemoryContext context)
static void
GenerationStats(MemoryContext context,
MemoryStatsPrintFunc printfunc, void *passthru,
- MemoryContextCounters *totals)+ MemoryContextCounters *totals, bool print_to_stderr)
{
GenerationContext *set = (GenerationContext *) context;
Size nblocks = 0;
@@ -704,7 +706,7 @@ GenerationStats(MemoryContext context,
"%zu total in %zd blocks (%zd chunks); %zu free (%zd chunks); %zu used",
totalspace, nblocks, nchunks, freespace,
nfreechunks, totalspace - freespace);
- printfunc(context, passthru, stats_string);+ printfunc(context, passthru, stats_string, print_to_stderr);
}
if (totals)
diff --git a/src/backend/utils/mmgr/mcxt.c b/src/backend/utils/mmgr/mcxt.cindex 84472b9158..bd41f8d31a 100644--- a/src/backend/utils/mmgr/mcxt.c+++ b/src/backend/utils/mmgr/mcxt.c@@ -23,6 +23,10 @@
#include "mb/pg_wchar.h"
#include "miscadmin.h"
+#include "storage/proc.h"+#include "storage/procarray.h"+#include "storage/procsignal.h"+#include "utils/fmgrprotos.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
@@ -55,9 +59,11 @@ MemoryContext PortalContext = NULL;
static void MemoryContextCallResetCallbacks(MemoryContext context);
static void MemoryContextStatsInternal(MemoryContext context, int level,
bool print, int max_children,
- MemoryContextCounters *totals);+ MemoryContextCounters *totals,+ bool print_to_stderr);
static void MemoryContextStatsPrint(MemoryContext context, void *passthru,
- const char *stats_string);+ const char *stats_string,+ bool print_to_stderr);
/*
* You should not do memory allocations within a critical section, because
@@ -499,7 +505,7 @@ void
MemoryContextStats(MemoryContext context)
{
/* A hard-wired limit on the number of children is usually good enough */
- MemoryContextStatsDetail(context, 100);+ MemoryContextStatsDetail(context, 100, true);
}
/*
@@ -508,19 +514,34 @@ MemoryContextStats(MemoryContext context)
* Entry point for use if you want to vary the number of child contexts shown.
*/
void
-MemoryContextStatsDetail(MemoryContext context, int max_children)+MemoryContextStatsDetail(MemoryContext context, int max_children,+ bool print_to_stderr)
{
MemoryContextCounters grand_totals;
memset(&grand_totals, 0, sizeof(grand_totals));
- MemoryContextStatsInternal(context, 0, true, max_children, &grand_totals);+ MemoryContextStatsInternal(context, 0, true, max_children, &grand_totals, print_to_stderr);- fprintf(stderr,- "Grand total: %zu bytes in %zd blocks; %zu free (%zd chunks); %zu used\n",- grand_totals.totalspace, grand_totals.nblocks,- grand_totals.freespace, grand_totals.freechunks,- grand_totals.totalspace - grand_totals.freespace);+ if (print_to_stderr)+ fprintf(stderr,+ "Grand total: %zu bytes in %zd blocks; %zu free (%zd chunks); %zu used\n",+ grand_totals.totalspace, grand_totals.nblocks,+ grand_totals.freespace, grand_totals.freechunks,+ grand_totals.totalspace - grand_totals.freespace);+ else++ /*+ * Use LOG_SERVER_ONLY to prevent the memory contexts from being sent+ * to the connected client.+ */+ ereport(LOG_SERVER_ONLY,+ (errhidestmt(true),+ errhidecontext(true),+ errmsg_internal("Grand total: %zu bytes in %zd blocks; %zu free (%zd chunks); %zu used",+ grand_totals.totalspace, grand_totals.nblocks,+ grand_totals.freespace, grand_totals.freechunks,+ grand_totals.totalspace - grand_totals.freespace)));
}
/*
@@ -533,7 +554,8 @@ MemoryContextStatsDetail(MemoryContext context, int max_children)
static void
MemoryContextStatsInternal(MemoryContext context, int level,
bool print, int max_children,
- MemoryContextCounters *totals)+ MemoryContextCounters *totals,+ bool print_to_stderr)
{
MemoryContextCounters local_totals;
MemoryContext child;
@@ -545,7 +567,7 @@ MemoryContextStatsInternal(MemoryContext context, int level,
context->methods->stats(context,
print ? MemoryContextStatsPrint : NULL,
(void *) &level,
- totals);+ totals, print_to_stderr);
/*
* Examine children. If there are more than max_children of them, we do
@@ -560,11 +582,13 @@ MemoryContextStatsInternal(MemoryContext context, int level,
if (ichild < max_children)
MemoryContextStatsInternal(child, level + 1,
print, max_children,
- totals);+ totals,+ print_to_stderr);
else
MemoryContextStatsInternal(child, level + 1,
false, max_children,
- &local_totals);+ &local_totals,+ print_to_stderr);
}
/* Deal with excess children */
@@ -572,18 +596,33 @@ MemoryContextStatsInternal(MemoryContext context, int level,
{
if (print)
{
- int i;-- for (i = 0; i <= level; i++)- fprintf(stderr, " ");- fprintf(stderr,- "%d more child contexts containing %zu total in %zd blocks; %zu free (%zd chunks); %zu used\n",- ichild - max_children,- local_totals.totalspace,- local_totals.nblocks,- local_totals.freespace,- local_totals.freechunks,- local_totals.totalspace - local_totals.freespace);+ if (print_to_stderr)+ {+ int i;++ for (i = 0; i <= level; i++)+ fprintf(stderr, " ");+ fprintf(stderr,+ "%d more child contexts containing %zu total in %zd blocks; %zu free (%zd chunks); %zu used\n",+ ichild - max_children,+ local_totals.totalspace,+ local_totals.nblocks,+ local_totals.freespace,+ local_totals.freechunks,+ local_totals.totalspace - local_totals.freespace);+ }+ else+ ereport(LOG_SERVER_ONLY,+ (errhidestmt(true),+ errhidecontext(true),+ errmsg_internal("level: %d; %d more child contexts containing %zu total in %zd blocks; %zu free (%zd chunks); %zu used",+ level,+ ichild - max_children,+ local_totals.totalspace,+ local_totals.nblocks,+ local_totals.freespace,+ local_totals.freechunks,+ local_totals.totalspace - local_totals.freespace)));
}
if (totals)
@@ -605,11 +644,13 @@ MemoryContextStatsInternal(MemoryContext context, int level,
*/
static void
MemoryContextStatsPrint(MemoryContext context, void *passthru,
- const char *stats_string)+ const char *stats_string,+ bool print_to_stderr)
{
int level = *(int *) passthru;
const char *name = context->name;
const char *ident = context->ident;
+ char truncated_ident[110];
int i;
/*
@@ -623,9 +664,8 @@ MemoryContextStatsPrint(MemoryContext context, void *passthru,
ident = NULL;
}
- for (i = 0; i < level; i++)- fprintf(stderr, " ");- fprintf(stderr, "%s: %s", name, stats_string);+ truncated_ident[0] = '\0';+
if (ident)
{
/*
@@ -637,24 +677,41 @@ MemoryContextStatsPrint(MemoryContext context, void *passthru,
int idlen = strlen(ident);
bool truncated = false;
+ strcpy(truncated_ident, ": ");+ i = strlen(truncated_ident);+
if (idlen > 100)
{
idlen = pg_mbcliplen(ident, idlen, 100);
truncated = true;
}
- fprintf(stderr, ": ");+
while (idlen-- > 0)
{
unsigned char c = *ident++;
if (c < ' ')
c = ' ';
- fputc(c, stderr);+ truncated_ident[i++] = c;
}
+ truncated_ident[i] = '\0';+
if (truncated)
- fprintf(stderr, "...");+ strcat(truncated_ident, "...");
}
- fputc('\n', stderr);++ if (print_to_stderr)+ {+ for (i = 0; i < level; i++)+ fprintf(stderr, " ");+ fprintf(stderr, "%s: %s%s\n", name, stats_string, truncated_ident);+ }+ else+ ereport(LOG_SERVER_ONLY,+ (errhidestmt(true),+ errhidecontext(true),+ errmsg_internal("level: %d; %s: %s%s",+ level, name, stats_string, truncated_ident)));
}
/*
@@ -946,6 +1003,52 @@ MemoryContextAllocExtended(MemoryContext context, Size size, int flags)
return ret;
}
+/*+ * HandleLogMemoryContextInterrupt+ * Handle receipt of an interrupt indicating logging of memory+ * contexts.+ *+ * All the actual work is deferred to ProcessLogMemoryContextInterrupt(),+ * because we cannot safely emit a log message inside the signal handler.+ */+void+HandleLogMemoryContextInterrupt(void)+{+ InterruptPending = true;+ LogMemoryContextPending = true;+ /* latch will be set by procsignal_sigusr1_handler */+}++/*+ * ProcessLogMemoryContextInterrupt+ * Perform logging of memory contexts of this backend process.+ *+ * Any backend that participates in ProcSignal signaling must arrange+ * to call this function if we see LogMemoryContextPending set.+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because+ * the target process for logging of memory contexts is a backend.+ */+void+ProcessLogMemoryContextInterrupt(void)+{+ LogMemoryContextPending = false;++ ereport(LOG,+ (errmsg("logging memory contexts of PID %d", MyProcPid)));++ /*+ * When a backend process is consuming huge memory, logging all its memory+ * contexts might overrun available disk space. To prevent this, we limit+ * the number of child contexts to log per parent to 100.+ *+ * As with MemoryContextStats(), we suppose that practical cases where the+ * dump gets long will typically be huge numbers of siblings under the+ * same parent context; while the additional debugging value from seeing+ * details about individual siblings beyond 100 will not be large.+ */+ MemoryContextStatsDetail(TopMemoryContext, 100, false);+}+
void *
palloc(Size size)
{
diff --git a/src/backend/utils/mmgr/slab.c b/src/backend/utils/mmgr/slab.cindex 9213be7c95..553dd7f667 100644--- a/src/backend/utils/mmgr/slab.c+++ b/src/backend/utils/mmgr/slab.c@@ -135,7 +135,8 @@ static Size SlabGetChunkSpace(MemoryContext context, void *pointer);
static bool SlabIsEmpty(MemoryContext context);
static void SlabStats(MemoryContext context,
MemoryStatsPrintFunc printfunc, void *passthru,
- MemoryContextCounters *totals);+ MemoryContextCounters *totals,+ bool print_to_stderr);
#ifdef MEMORY_CONTEXT_CHECKING
static void SlabCheck(MemoryContext context);
#endif
@@ -632,11 +633,13 @@ SlabIsEmpty(MemoryContext context)
* printfunc: if not NULL, pass a human-readable stats string to this.
* passthru: pass this pointer through to printfunc.
* totals: if not NULL, add stats about this context into *totals.
+ * print_to_stderr: print stats to stderr if true, elog otherwise.
*/
static void
SlabStats(MemoryContext context,
MemoryStatsPrintFunc printfunc, void *passthru,
- MemoryContextCounters *totals)+ MemoryContextCounters *totals,+ bool print_to_stderr)
{
SlabContext *slab = castNode(SlabContext, context);
Size nblocks = 0;
@@ -671,7 +674,7 @@ SlabStats(MemoryContext context,
"%zu total in %zd blocks; %zu free (%zd chunks); %zu used",
totalspace, nblocks, freespace, freechunks,
totalspace - freespace);
- printfunc(context, passthru, stats_string);+ printfunc(context, passthru, stats_string, print_to_stderr);
}
if (totals)
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.datindex bfb89e0575..8049587707 100644--- a/src/include/catalog/pg_proc.dat+++ b/src/include/catalog/pg_proc.dat@@ -7946,6 +7946,12 @@
proargnames => '{name, ident, parent, level, total_bytes, total_nblocks, free_bytes, free_chunks, used_bytes}',
prosrc => 'pg_get_backend_memory_contexts' },
+# log memory contexts of the specified backend+{ oid => '4543', descr => 'log memory contexts of the specified backend',+ proname => 'pg_log_backend_memory_contexts',+ provolatile => 'v', prorettype => 'bool',+ proargtypes => 'int4', prosrc => 'pg_log_backend_memory_contexts' },+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.hindex 013850ac28..081822823c 100644--- a/src/include/miscadmin.h+++ b/src/include/miscadmin.h@@ -84,6 +84,7 @@ extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/nodes/memnodes.h b/src/include/nodes/memnodes.hindex 9331ef80fd..e6a757d6a0 100644--- a/src/include/nodes/memnodes.h+++ b/src/include/nodes/memnodes.h@@ -52,7 +52,8 @@ typedef struct MemoryContextCounters
*/
typedef void (*MemoryStatsPrintFunc) (MemoryContext context, void *passthru,
- const char *stats_string);+ const char *stats_string,+ bool print_to_stderr);
typedef struct MemoryContextMethods
{
@@ -66,7 +67,8 @@ typedef struct MemoryContextMethods
bool (*is_empty) (MemoryContext context);
void (*stats) (MemoryContext context,
MemoryStatsPrintFunc printfunc, void *passthru,
- MemoryContextCounters *totals);+ MemoryContextCounters *totals,+ bool print_to_stderr);
#ifdef MEMORY_CONTEXT_CHECKING
void (*check) (MemoryContext context);
#endif
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.hindex 4ae7dc33b8..eec186be2e 100644--- a/src/include/storage/procsignal.h+++ b/src/include/storage/procsignal.h@@ -34,6 +34,7 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
diff --git a/src/include/utils/memutils.h b/src/include/utils/memutils.hindex 36aae4e51c..ff872274d4 100644--- a/src/include/utils/memutils.h+++ b/src/include/utils/memutils.h@@ -84,7 +84,8 @@ extern MemoryContext MemoryContextGetParent(MemoryContext context);
extern bool MemoryContextIsEmpty(MemoryContext context);
extern Size MemoryContextMemAllocated(MemoryContext context, bool recurse);
extern void MemoryContextStats(MemoryContext context);
-extern void MemoryContextStatsDetail(MemoryContext context, int max_children);+extern void MemoryContextStatsDetail(MemoryContext context, int max_children,+ bool print_to_stderr);
extern void MemoryContextAllowInCriticalSection(MemoryContext context,
bool allow);
@@ -144,6 +145,8 @@ extern void MemoryContextCreate(MemoryContext node,
MemoryContext parent,
const char *name);
+extern void HandleLogMemoryContextInterrupt(void);+extern void ProcessLogMemoryContextInterrupt(void);
/*
* Memory-context-type-specific functions
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.outindex d3acb98d04..e845042d38 100644--- a/src/test/regress/expected/misc_functions.out+++ b/src/test/regress/expected/misc_functions.out@@ -133,6 +133,19 @@ ERROR: function num_nulls() does not exist
LINE 1: SELECT num_nulls();
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
+--+-- pg_log_backend_memory_contexts()+--+-- Memory contexts are logged and they are not returned to the function.+-- Furthermore, their contents can vary depending on the timing. However,+-- we can at least verify that the code doesn't fail.+--+SELECT * FROM pg_log_backend_memory_contexts(pg_backend_pid());+ pg_log_backend_memory_contexts +--------------------------------+ t+(1 row)+
--
-- Test some built-in SRFs
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sqlindex 094e8f8296..a398349afc 100644--- a/src/test/regress/sql/misc_functions.sql+++ b/src/test/regress/sql/misc_functions.sql@@ -30,6 +30,15 @@ SELECT num_nulls(VARIADIC '{}'::int[]);
SELECT num_nonnulls();
SELECT num_nulls();
+--+-- pg_log_backend_memory_contexts()+--+-- Memory contexts are logged and they are not returned to the function.+-- Furthermore, their contents can vary depending on the timing. However,+-- we can at least verify that the code doesn't fail.+--+SELECT * FROM pg_log_backend_memory_contexts(pg_backend_pid());+
--
-- Test some built-in SRFs
--
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: torikoshia@oss.nttdata.com, masao.fujii@oss.nttdata.com, horikyota.ntt@gmail.com, tgl@sss.pgh.pa.us, gkokolatos@protonmail.com, kasahara.tatsuhito@gmail.com, craig@2ndquadrant.com
Subject: Re: Get memory contexts of an arbitrary backend process
In-Reply-To: <f6f6373ae200c8a0e7466e4de9ca65eb@oss.nttdata.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox