public inbox for [email protected]
help / color / mirror / Atom feedFrom: Rahila Syed <[email protected]>
To: Daniel Gustafsson <[email protected]>
Cc: Tomas Vondra <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Subject: Re: Enhancing Memory Context Statistics Reporting
Date: Thu, 13 Mar 2025 19:26:51 +0530
Message-ID: <CAH2L28vULzqoit+YCKR5UhdT+AR+b1Qcs6Hgpz6nQz6NBT2jug@mail.gmail.com> (raw)
In-Reply-To: <CAH2L28s8Etbz2XM0xiH=RyRHAnEAxMD2AVpvcHyhHEHTbf-Uqg@mail.gmail.com>
References: <[email protected]>
<[email protected]>
<CAH2L28uvYHX1CfmbcB9JPzR6+MzJ+kb9ob+NvFy3wUHM8HUkGA@mail.gmail.com>
<[email protected]>
<CAH2L28spk4c54k5pTSWRqu5LJmetQwTXtr=-F7Bnioxd425hqQ@mail.gmail.com>
<[email protected]>
<CAH2L28vEiz+Qxw=Y5Dze8_TxDW1kAyh_3R-BKUGvOH6r02W6cg@mail.gmail.com>
<[email protected]>
<CAH2L28vR622wV44XenbhWc7ETpNtjS_oeTha7OxMx35LjWPPqQ@mail.gmail.com>
<[email protected]>
<CAH2L28u7=fcgnY8bpM87moiJxt++wqWZXh2HxFabYjiHSg76Cg@mail.gmail.com>
<[email protected]>
<CAH2L28vYvC5DP+YccBE7VnC-khn3k_vd=MGxn8BFEpLc40ncHw@mail.gmail.com>
<[email protected]>
<CAH2L28sNHmj+aCJ=XkNb0an-XAs3eOUcU4orx2G9yviRL560fg@mail.gmail.com>
<CAH2L28sMnNy9DvzAAoiE8qQs0MRX9ALhaYAf2f-aLivL47Ryhw@mail.gmail.com>
<[email protected]>
<CAH2L28uayhv+AxgPLThexJ21NA8j7XFiYqu6rgsZSSNosvPjvg@mail.gmail.com>
<[email protected]>
<CAH2L28s8Etbz2XM0xiH=RyRHAnEAxMD2AVpvcHyhHEHTbf-Uqg@mail.gmail.com>
Hi,
Please find attached updated and rebased patches. It has the following
changes
1. To prevent memory leaks, ensure that the latest statistics published by
a process
are freed before it exits. This can be achieved by calling dsa_free in the
before_shmem_exit callback.
2. Add a level column to maintain consistency with the output of
pg_backend_memory_contexts.
Thank you,
Rahila Syed
On Tue, Mar 4, 2025 at 12:30 PM Rahila Syed <[email protected]> wrote:
> Hi Daniel,
>
> Thanks for the rebase, a few mostly superficial comments from a first
>> read-through.
>>
> Thank you for your comments.
>
>
>> The documentation refers to attributes in the return row but the format
>> of that
>> row isn't displayed which makes following along hard. I think we should
>> include a table or a programlisting showing the return data before this
>> paragraph.
>>
>> I included the sql function call and its output in programlisting format
> after the
> function description.
> Since the description was part of a table, I added this additional
> information at the
> end of that table.
>
>
>> +const char *
>> +AssignContextType(NodeTag type)
>> This function doesn't actually assign anything so the name is a bit
>> misleading,
>> it would be better with ContextTypeToString or something similar.
>>
>> Done.
>
>
>>
>> + * By default, only superusers or users with PG_READ_ALL_STATS are
>> allowed to
>> This sentence is really long and should probably be broken up.
>>
>> Fixed.
>
>>
>> + * The shared memory buffer has a limited size - it the process has too
>> many
>> s/it/if/
>>
>> Fixed.
>
>
>> + * If the publishing backend does not respond before the condition
>> variable
>> + * times out, which is set to MEMSTATS_WAIT_TIMEOUT, retry for max_tries
>> + * number of times, which is defined by user, before giving up and
>> + * returning previously published statistics, if any.
>> This comment should mention what happens if the process gives up and
>> there is
>> no previously published stats.
>>
>> Done.
>
>
>>
>> + int i;
>> ...
>> + for (i = 0; i < memCtxState[procNumber].total_stats; i++)
>> This can be rewritten as "for (int i = 0; .." since we allow C99.
>>
>> Done.
>
>
>>
>> + * process running and consuming lots of memory, that it might end on
>> its
>> + * own first and its memory contexts are not logged is not a problem.
>> This comment is copy/pasted from pg_log_backend_memory_contexts and while
>> it
>> mostly still apply it should at least be rewritten to not refer to
>> logging as
>> this function doesn't do that.
>>
>> Fixed.
>
>
>>
>> + ereport(WARNING,
>> + (errmsg("PID %d is not a PostgreSQL server process",
>> No need to add the extra parenthesis around errmsg anymore, so I think
>> new code
>> should omit those.
>>
>> Done.
>
>
>>
>> + errhint("Use pg_backend_memory_contexts view instead")));
>> Super nitpick, but errhints should be complete sentences ending with a
>> period.
>>
>> Done.
>
>
>>
>> + * statitics have previously been published by the backend. In which
>> case,
>> s/statitics/statistics/
>>
>> Fixed.
>
>
>>
>> + * statitics have previously been published by the backend. In which
>> case,
>> + * check if statistics are not older than curr_timestamp, if they are
>> wait
>> I think the sentence around the time check is needlessly confusing, could
>> it be
>> rewritten into something like:
>> "A valid DSA pointer isn't proof that statistics are available, it
>> can be
>> valid due to previously published stats. Check if the stats are
>> updated by
>> comparing the timestamp, if the stats are newer than our previously
>> recorded timestamp from before sending the procsignal they must by
>> definition be updated."
>>
>> Replaced accordingly.
>
>
>>
>> + /* Assert for dsa_handle to be valid */
>> Was this intended to be turned into an Assert call? Else it seems better
>> to remove.
>>
>
> Added an assert and removed the comment.
>
>
>> + if (print_location != PRINT_STATS_NONE)
>> This warrants a comment stating why it makes sense.
>>
>
>> + * Do not print the statistics if print_to_stderr is PRINT_STATS_NONE,
>> s/print_to_stderr/print_location/. Also, do we really need
>> print_to_stderr in
>> this function at all? It seems a bit awkward to combine a boolean and a
>> paramter for a tri-state value when the parameter holds the tri_state
>> already.
>> For readability I think just checking print_location will be better since
>> the
>> value will be clear, where print_to_stderr=false is less clear in a
>> tri-state
>> scenario.
>>
>> I removed the boolean print_to_stderr, the checks are now using
> the tri-state enum-print_location.
>
>
>> + * its ancestors to a list, inorder to compute a path.
>> s/inorder/in order/
>>
>> Fixed.
>
>
>>
>> + elog(LOG, "hash table corrupted, can't construct path value");
>> + break;
>> This will return either a NIL list or a partial path, but
>> PublishMemoryContext
>> doesn't really take into consideration that it might be so. Is this
>> really
>> benign to the point that we can blindly go on? Also, elog(LOG..) is
>> mostly for
>> tracing or debugging as elog() isn't intended for user facing errors.
>>
>> I agree that this should be addressed. I added a check for path value
> before
> storing it in shared memory. If the path is NIL, the path pointer in DSA
> will point
> to InvalidDsaPointer.
> When a client encounters an InvalidDsaPointer it will print NULL in the
> path column.
> Partial path scenario is unlikely IMO, and I am not sure if it warrants
> additional
> checks.
>
>
>> +static void
>> +compute_num_of_contexts(List *contexts, HTAB *context_id_lookup,
>> + int *stats_count, bool get_summary)
>> This function does a lot than compute the number of contexts so the name
>> seems
>> a bit misleading. Perhaps a rename to compute_contexts() or something
>> similar?
>>
>> Renamed to compute_contexts_count_and_ids.
>
>
>>
>> + memctx_info[curr_id].name = dsa_allocate0(area,
>> + strlen(clipped_ident) + 1);
>> These calls can use idlen instead of more strlen() calls no? While there
>> is no
>> performance benefit, it would increase readability IMHO if the code first
>> calculates a value, and then use it.
>>
>> Done.
>
>
>>
>> + strncpy(name,
>> + clipped_ident, strlen(clipped_ident));
>> Since clipped_ident has been nul terminated earlier there is no need to
>> use
>> strncpy, we can instead use strlcpy and take the target buffer size into
>> consideration rather than the input string length.
>>
>> Replaced with the strlcpy calls.
>
>
>>
>> PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts
>> */
>> + PROCSIG_GET_MEMORY_CONTEXT, /* ask backend to log the memory contexts
>> */
>> This comment should be different from the LOG_MEMORY_xx one.
>>
>> Fixed.
>
> +#define MEM_CONTEXT_SHMEM_STATS_SIZE 30
>> +#define MAX_TYPE_STRING_LENGTH 64
>> These are unused, from an earlier version of the patch perhaps?
>>
>> Removed
>
> + * Singe DSA area is created and used by all the processes,
>> s/Singe/Since/
>>
>
> Fixed.
>
> +typedef struct MemoryContextBackendState
>> This is only used in mcxtfuncs.c and can be moved there rather than being
>> exported in the header.
>>
>
> This is being used in mcxt.c too in the form of the variable memCtxState.
>
>
>>
>
> +} MemoryContextId;
>> This lacks an entry in the typedefs.list file.
>>
>> Added.
>
> Please find attached the updated patches with the above fixes.
>
> Thank you,
> Rahila Syed
>
Attachments:
[application/octet-stream] v17-0001-Preparatory-changes-for-reporting-memory-context-sta.patch (4.3K, ../CAH2L28vULzqoit+YCKR5UhdT+AR+b1Qcs6Hgpz6nQz6NBT2jug@mail.gmail.com/3-v17-0001-Preparatory-changes-for-reporting-memory-context-sta.patch)
download | inline diff:
From bce37aa953a3aefbe83d98bbb9fa931598bd347d Mon Sep 17 00:00:00 2001
From: Rahila Syed <[email protected]>
Date: Mon, 3 Feb 2025 15:33:19 +0530
Subject: [PATCH 1/2] Preparatory changes for reporting memory context
statistics
Ensure that MemoryContextStatsInternal can return number of
contexts. Also, provide an option in MemoryContextStatsInternal
to return without printing stats to either stderr or logs.
---
src/backend/utils/mmgr/mcxt.c | 65 +++++++++++++++++++++++++++++------
1 file changed, 55 insertions(+), 10 deletions(-)
diff --git a/src/backend/utils/mmgr/mcxt.c b/src/backend/utils/mmgr/mcxt.c
index 91060de0ab..34cdcdf2fd 100644
--- a/src/backend/utils/mmgr/mcxt.c
+++ b/src/backend/utils/mmgr/mcxt.c
@@ -135,6 +135,17 @@ static const MemoryContextMethods mcxt_methods[] = {
};
#undef BOGUS_MCTX
+/*
+ * This is passed to MemoryContextStatsInternal to determine whether
+ * to print context statistics or not and where to print them logs or
+ * stderr.
+ */
+typedef enum PrintDestination
+{
+ PRINT_STATS_TO_STDERR = 0,
+ PRINT_STATS_TO_LOGS,
+ PRINT_STATS_NONE
+} PrintDestination;
/*
* CurrentMemoryContext
@@ -162,7 +173,7 @@ static void MemoryContextCallResetCallbacks(MemoryContext context);
static void MemoryContextStatsInternal(MemoryContext context, int level,
int max_level, int max_children,
MemoryContextCounters *totals,
- bool print_to_stderr);
+ PrintDestination print_location, int *num_contexts);
static void MemoryContextStatsPrint(MemoryContext context, void *passthru,
const char *stats_string,
bool print_to_stderr);
@@ -831,11 +842,19 @@ MemoryContextStatsDetail(MemoryContext context,
bool print_to_stderr)
{
MemoryContextCounters grand_totals;
+ int num_contexts;
+ PrintDestination print_location;
memset(&grand_totals, 0, sizeof(grand_totals));
+ if (print_to_stderr)
+ print_location = PRINT_STATS_TO_STDERR;
+ else
+ print_location = PRINT_STATS_TO_LOGS;
+
+ /* num_contexts report number of contexts aggregated in the output */
MemoryContextStatsInternal(context, 0, max_level, max_children,
- &grand_totals, print_to_stderr);
+ &grand_totals, print_location, &num_contexts);
if (print_to_stderr)
fprintf(stderr,
@@ -876,18 +895,43 @@ static void
MemoryContextStatsInternal(MemoryContext context, int level,
int max_level, int max_children,
MemoryContextCounters *totals,
- bool print_to_stderr)
+ PrintDestination print_location, int *num_contexts)
{
MemoryContext child;
int ichild;
+ bool print_to_stderr = true;
+ check_stack_depth();
Assert(MemoryContextIsValid(context));
- /* Examine the context itself */
- context->methods->stats(context,
- MemoryContextStatsPrint,
- &level,
- totals, print_to_stderr);
+ if (print_location == PRINT_STATS_TO_STDERR)
+ print_to_stderr = true;
+ else if (print_location == PRINT_STATS_TO_LOGS)
+ print_to_stderr = false;
+
+ if (print_location != PRINT_STATS_NONE)
+ {
+ /* Examine the context itself */
+ context->methods->stats(context,
+ MemoryContextStatsPrint,
+ &level,
+ totals, print_to_stderr);
+ }
+
+ /*
+ * Do not print the statistics if print_to_stderr is PRINT_STATS_NONE,
+ * only compute totals.
+ */
+ else
+ {
+ /* Examine the context itself */
+ context->methods->stats(context,
+ NULL,
+ NULL,
+ totals, print_to_stderr);
+ }
+ /* Increment the context count */
+ *num_contexts = *num_contexts + 1;
/*
* Examine children.
@@ -907,7 +951,7 @@ MemoryContextStatsInternal(MemoryContext context, int level,
MemoryContextStatsInternal(child, level + 1,
max_level, max_children,
totals,
- print_to_stderr);
+ print_location, num_contexts);
}
}
@@ -925,6 +969,7 @@ MemoryContextStatsInternal(MemoryContext context, int level,
ichild++;
child = MemoryContextTraverseNext(child, context);
}
+ *num_contexts = *num_contexts + ichild;
if (print_to_stderr)
{
@@ -939,7 +984,7 @@ MemoryContextStatsInternal(MemoryContext context, int level,
local_totals.freechunks,
local_totals.totalspace - local_totals.freespace);
}
- else
+ else if (print_location != PRINT_STATS_NONE)
ereport(LOG_SERVER_ONLY,
(errhidestmt(true),
errhidecontext(true),
--
2.34.1
[application/octet-stream] v17-0002-Function-to-report-memory-context-statistics.patch (54.5K, ../CAH2L28vULzqoit+YCKR5UhdT+AR+b1Qcs6Hgpz6nQz6NBT2jug@mail.gmail.com/4-v17-0002-Function-to-report-memory-context-statistics.patch)
download | inline diff:
From 3a570e83771aa1ce9399f2717f16c704cb7a00c0 Mon Sep 17 00:00:00 2001
From: Rahila Syed <[email protected]>
Date: Mon, 3 Feb 2025 15:37:17 +0530
Subject: [PATCH 2/2] Function to report memory context statistics
This function sends a signal to a backend to publish
statistics of all its memory contexts. Signal handler
running in the backend process, sets a flag, which causes
it to copy its MemoryContextStats to a DSA, during the
next call to CHECK_FOR_INTERRUPTS().
If there are more statistics than that fit in 16MB, the
remaining statistics are copied as a cumulative
total of the remaining contexts.
Once its done, it signals the client backend using
a condition variable. The client backend wakes up, reads
the shared memory and returns these values in the form
of set of records, one for each memory context, to the
user, followed by a cumulative total of the remaining
contexts, if any.
If get_summary is true return statistics of all children
of TopMemoryContext with aggregated statistics of their
children.
User can pass num_of_tries which determines the total
number of wait cycles in a client backend for latest
statistics.
Each cycle wait timeout is set to 1 seconds. Post this
the client displays previously published statistics or
returns without results.
Each backend and auxiliary process has its own slot for
reporting the stats. There is an array of such memory slots
of size MaxBackends+NumofAuxiliary
processes in fixed shared memory. Each of these slots point
to a smaller dsa allocations within a single DSA,
which contains the stats to be shared by the corresponding
process.
Each slot has its own LW lock and condition variable for
synchronization and communication between the publishing
process and the client backend.
---
doc/src/sgml/func.sgml | 61 ++
src/backend/postmaster/autovacuum.c | 4 +
src/backend/postmaster/checkpointer.c | 4 +
src/backend/postmaster/interrupt.c | 4 +
src/backend/postmaster/pgarch.c | 4 +
src/backend/postmaster/startup.c | 4 +
src/backend/postmaster/walsummarizer.c | 4 +
src/backend/storage/ipc/ipci.c | 3 +
src/backend/storage/ipc/procsignal.c | 3 +
src/backend/tcop/postgres.c | 3 +
.../utils/activity/wait_event_names.txt | 1 +
src/backend/utils/adt/mcxtfuncs.c | 436 +++++++++++++-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/mmgr/mcxt.c | 567 +++++++++++++++++-
src/include/catalog/pg_proc.dat | 10 +
src/include/miscadmin.h | 1 +
src/include/storage/procsignal.h | 1 +
src/include/utils/memutils.h | 69 +++
src/test/regress/expected/sysviews.out | 14 +
src/test/regress/sql/sysviews.sql | 14 +
src/tools/pgindent/typedefs.list | 4 +
21 files changed, 1169 insertions(+), 43 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 51dd8ad657..b5eee4d4a7 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -28553,6 +28553,50 @@ acl | {postgres=arwdDxtm/postgres,foo=r/postgres}
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_get_process_memory_contexts</primary>
+ </indexterm>
+ <function>pg_get_process_memory_contexts</function> ( <parameter>pid</parameter> <type>integer</type>, <parameter>get_summary</parameter> <type>boolean</type>, <parameter>num_of_tries</parameter> <type>integer</type> )
+ <returnvalue>setof record</returnvalue>
+ </para>
+ <para>
+ This function handles requests to display the memory contexts of a
+ PostgreSQL process with the specified process ID (PID). It takes three
+ arguments: <parameter>PID</parameter>, <parameter>get_summary</parameter>
+ and <parameter>num_of_tries</parameter>. The function can send requests
+ to both backend and auxiliary processes.
+
+ After receiving memory context statistics from the target process, it
+ returns the results as one row per context. If all the contexts don't
+ fit within the pre-determined size limit, the remaining context statistics
+ are aggregated and a cumulative total is displayed. The num_agg_contexts
+ column indicates the number of contexts aggregated in the displayed
+ statistics. The num_agg_contexts value is typically 1, meaning that each
+ context's statistics are displayed separately.
+
+ When <parameter>get_summary</parameter> is set to true, statistics
+ for memory contexts at levels 1 and 2 are displayed, with level 1
+ representing the root node (i.e., TopMemoryContext).
+ Each level 2 context's statistics represent an aggregate of all its
+ child contexts' statistics, with num_agg_contexts indicating the number
+ of these aggregated child contexts.
+
+ When <parameter>get_summary</parameter> is set to false, the
+ num_agg_contexts value is 1, indicating that individual statistics are
+ being displayed.
+
+ <parameter>num_of_tries</parameter> indicates the number of times
+ the client will wait for the latest statistics. The wait per try is 1
+ second. This parameter can be increased if the user anticipates a delay
+ in the response from the reporting process. Conversely, if users are
+ frequently and periodically querying the process for statistics, or if
+ there are concurrent requests for statistics of the same process,
+ lowering the parameter might help achieve a faster response.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -28671,6 +28715,23 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_get_process_memory_contexts</function> can be used
+ to request the memory contexts statistics of any postgres process. For example:
+<programlisting>
+postgres=# SELECT * FROM pg_get_process_memory_contexts(
+ (SELECT pid FROM pg_stat_activity
+ WHERE backend_type = 'checkpointer')
+ , false, 5) LIMIT 1;
+ name | ident | type | path | total_bytes | total_nblocks | free_bytes | free_chunks | used_bytes | num_
+agg_contexts | stats_timestamp
+------------------+-------+----------+------+-------------+---------------+------------+-------------+------------+-----
+-------------+----------------------------------
+ TopMemoryContext | | AllocSet | {1} | 102664 | 4 | 3008 | 2 | 99656 |
+ 1 | 2025-03-04 10:01:57.590543+05:30
+</programlisting>
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 800815dfbc..6630fbf05c 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -779,6 +779,10 @@ ProcessAutoVacLauncherInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+ /* Publish memory contexts of this process */
+ if (PublishMemoryContextPending)
+ ProcessGetMemoryContextInterrupt();
+
/* Process sinval catchup interrupts that happened while sleeping */
ProcessCatchupInterrupt();
}
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 0e228d143a..dc1b228b71 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -661,6 +661,10 @@ ProcessCheckpointerInterrupts(void)
/* Perform logging of memory contexts of this process */
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ /* Publish memory contexts of this process */
+ if (PublishMemoryContextPending)
+ ProcessGetMemoryContextInterrupt();
}
/*
diff --git a/src/backend/postmaster/interrupt.c b/src/backend/postmaster/interrupt.c
index 0ae9bf906e..f24f574e74 100644
--- a/src/backend/postmaster/interrupt.c
+++ b/src/backend/postmaster/interrupt.c
@@ -48,6 +48,10 @@ ProcessMainLoopInterrupts(void)
/* Perform logging of memory contexts of this process */
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ /* Publish memory contexts of this process */
+ if (PublishMemoryContextPending)
+ ProcessGetMemoryContextInterrupt();
}
/*
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index dbe4e1d426..9cdba52054 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -865,6 +865,10 @@ ProcessPgArchInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+ /* Publish memory contexts of this process */
+ if (PublishMemoryContextPending)
+ ProcessGetMemoryContextInterrupt();
+
if (ConfigReloadPending)
{
char *archiveLib = pstrdup(XLogArchiveLibrary);
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index 27e86cf393..7149a67fcb 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -192,6 +192,10 @@ ProcessStartupProcInterrupts(void)
/* Perform logging of memory contexts of this process */
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ /* Publish memory contexts of this process */
+ if (PublishMemoryContextPending)
+ ProcessGetMemoryContextInterrupt();
}
diff --git a/src/backend/postmaster/walsummarizer.c b/src/backend/postmaster/walsummarizer.c
index ccba0f84e6..4da70f6e0a 100644
--- a/src/backend/postmaster/walsummarizer.c
+++ b/src/backend/postmaster/walsummarizer.c
@@ -877,6 +877,10 @@ ProcessWalSummarizerInterrupts(void)
/* Perform logging of memory contexts of this process */
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ /* Publish memory contexts of this process */
+ if (PublishMemoryContextPending)
+ ProcessGetMemoryContextInterrupt();
}
/*
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 174eed7036..5eee04d52a 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -50,6 +50,7 @@
#include "storage/sinvaladt.h"
#include "utils/guc.h"
#include "utils/injection_point.h"
+#include "utils/memutils.h"
/* GUCs */
int shared_memory_type = DEFAULT_SHARED_MEMORY_TYPE;
@@ -340,6 +341,8 @@ CreateOrAttachShmemStructs(void)
StatsShmemInit();
WaitEventCustomShmemInit();
InjectionPointShmemInit();
+ MemCtxShmemInit();
+ MemCtxBackendShmemInit();
}
/*
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 7d20196550..b59034fdc3 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -690,6 +690,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_GET_MEMORY_CONTEXT))
+ HandleGetMemoryContextInterrupt();
+
if (CheckProcSignal(PROCSIG_PARALLEL_APPLY_MESSAGE))
HandleParallelApplyMessageInterrupt();
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 55ab2da299..00bc19c3a7 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3500,6 +3500,9 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+ if (PublishMemoryContextPending)
+ ProcessGetMemoryContextInterrupt();
+
if (ParallelApplyMessagePending)
ProcessParallelApplyMessages();
}
diff --git a/src/backend/utils/activity/wait_event_names.txt b/src/backend/utils/activity/wait_event_names.txt
index 3c594415bf..fcf68b47b9 100644
--- a/src/backend/utils/activity/wait_event_names.txt
+++ b/src/backend/utils/activity/wait_event_names.txt
@@ -159,6 +159,7 @@ WAL_RECEIVER_EXIT "Waiting for the WAL receiver to exit."
WAL_RECEIVER_WAIT_START "Waiting for startup process to send initial data for streaming replication."
WAL_SUMMARY_READY "Waiting for a new WAL summary to be generated."
XACT_GROUP_UPDATE "Waiting for the group leader to update transaction status at transaction end."
+MEM_CTX_PUBLISH "Waiting for backend to publish memory information."
ABI_compatibility:
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index 396c2f223b..96a7c634e7 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -17,28 +17,26 @@
#include "funcapi.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
+#include "access/twophase.h"
+#include "catalog/pg_authid_d.h"
+#include "nodes/pg_list.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "utils/acl.h"
#include "utils/array.h"
#include "utils/builtins.h"
#include "utils/hsearch.h"
+#include "utils/memutils.h"
+#include "utils/wait_event_types.h"
/* ----------
* The max bytes for showing identifiers of MemoryContext.
* ----------
*/
#define MEMORY_CONTEXT_IDENT_DISPLAY_SIZE 1024
-
-/*
- * MemoryContextId
- * Used for storage of transient identifiers for
- * pg_get_backend_memory_contexts.
- */
-typedef struct MemoryContextId
-{
- MemoryContext context;
- int context_id;
-} MemoryContextId;
+struct MemoryContextBackendState *memCtxState = NULL;
+struct MemoryContextState *memCtxArea = NULL;
/*
* int_list_to_array
@@ -143,24 +141,7 @@ PutMemoryContextsStatsTupleStore(Tuplestorestate *tupstore,
else
nulls[1] = true;
- switch (context->type)
- {
- case T_AllocSetContext:
- type = "AllocSet";
- break;
- case T_GenerationContext:
- type = "Generation";
- break;
- case T_SlabContext:
- type = "Slab";
- break;
- case T_BumpContext:
- type = "Bump";
- break;
- default:
- type = "???";
- break;
- }
+ type = ContextTypeToString(context->type);
values[2] = CStringGetTextDatum(type);
values[3] = Int32GetDatum(list_length(path)); /* level */
@@ -175,6 +156,32 @@ PutMemoryContextsStatsTupleStore(Tuplestorestate *tupstore,
list_free(path);
}
+const char *
+ContextTypeToString(NodeTag type)
+{
+ const char *context_type;
+
+ switch (type)
+ {
+ case T_AllocSetContext:
+ context_type = "AllocSet";
+ break;
+ case T_GenerationContext:
+ context_type = "Generation";
+ break;
+ case T_SlabContext:
+ context_type = "Slab";
+ break;
+ case T_BumpContext:
+ context_type = "Bump";
+ break;
+ default:
+ context_type = "???";
+ break;
+ }
+ return (context_type);
+}
+
/*
* pg_get_backend_memory_contexts
* SQL SRF showing backend memory context.
@@ -281,7 +288,7 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
* to acquire a lock on an arbitrary process to prevent that. But since
* this mechanism is usually used to debug a backend or an auxiliary
* process running and consuming lots of memory, that it might end on its
- * own first and its memory contexts are not logged is not a problem.
+ * own first and its memory contexts are not reported is not a problem.
*/
if (proc == NULL)
{
@@ -290,7 +297,7 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
* if one backend terminated on its own during the run.
*/
ereport(WARNING,
- (errmsg("PID %d is not a PostgreSQL server process", pid)));
+ errmsg("PID %d is not a PostgreSQL server process", pid));
PG_RETURN_BOOL(false);
}
@@ -299,9 +306,374 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
{
/* Again, just a warning to allow loops */
ereport(WARNING,
- (errmsg("could not send signal to process %d: %m", pid)));
+ errmsg("could not send signal to process %d: %m", pid));
PG_RETURN_BOOL(false);
}
PG_RETURN_BOOL(true);
}
+
+/*
+ * pg_get_process_memory_contexts
+ * Signal a backend or an auxiliary process to send its memory contexts,
+ * wait for the results and display them.
+ *
+ * By default, only superusers or users with PG_READ_ALL_STATS are allowed to
+ * signal a process to return the memory contexts. This is because allowing
+ * any users to issue this request at an unbounded rate would cause lots of
+ * requests to be sent, which can lead to denial of service. Additional roles
+ * can be permitted with GRANT.
+ *
+ * On receipt of this signal, a backend or an auxiliary process sets the flag
+ * in the signal handler, which causes the next CHECK_FOR_INTERRUPTS()
+ * or process-specific interrupt handler to copy the memory context details
+ * to a dynamic shared memory space.
+ *
+ * The shared memory buffer has a limited size - if the process has too many
+ * memory contexts, the memory contexts that do not fit are summarized
+ * and represented as cumulative total at the end of the buffer.
+ *
+ * After sending the signal, wait on a condition variable. The publishing
+ * backend, after copying the data to shared memory, sends signal on that
+ * condition variable. There is one condition variable per publishing
+ * backend.
+ * Once condition variable is signalled, check if the memory context
+ * information is available for reading and display.
+ *
+ * If the publishing backend does not respond before the condition variable
+ * times out, which is set to MEMSTATS_WAIT_TIMEOUT, retry for max_tries
+ * number of times, which is defined by user, before giving up and
+ * returning previously published statistics, if any. If previous statistics
+ * do not exist, return NULL.
+ */
+Datum
+pg_get_process_memory_contexts(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ bool get_summary = PG_GETARG_BOOL(1);
+ PGPROC *proc;
+ ProcNumber procNumber = INVALID_PROC_NUMBER;
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ dsa_area *area;
+ MemoryContextEntry *memctx_info;
+ int num_retries = 0;
+ TimestampTz curr_timestamp;
+ int max_tries = PG_GETARG_INT32(2);
+
+ /*
+ * Only superusers or users with pg_read_all_stats privileges can view the
+ * memory context statistics of another process
+ */
+ if (!has_privs_of_role(GetUserId(), ROLE_PG_READ_ALL_STATS))
+ ereport(ERROR,
+ errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("memory context statistics privilege error"));
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ /*
+ * See if the process with given pid is a backend or an auxiliary process.
+ */
+ proc = BackendPidGetProc(pid);
+ if (proc == NULL)
+ proc = AuxiliaryPidGetProc(pid);
+
+ /*
+ * BackendPidGetProc() and AuxiliaryPidGetProc() return NULL if the pid
+ * isn't valid; but by the time we reach kill(), a process for which we
+ * get a valid proc here might have terminated on its own. There's no way
+ * to acquire a lock on an arbitrary process to prevent that. But since
+ * this mechanism is usually used to debug a backend or an auxiliary
+ * process running and consuming lots of memory, that it might end on its
+ * own first and its memory contexts are not logged is not a problem.
+ */
+ if (proc == NULL)
+ {
+ /*
+ * This is just a warning so a loop-through-resultset will not abort
+ * if one backend terminated on its own during the run.
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL server process",
+ pid)));
+ PG_RETURN_NULL();
+ }
+
+ procNumber = GetNumberFromPGProc(proc);
+ if (procNumber == MyProcNumber)
+ {
+ ereport(WARNING,
+ errmsg("cannot return statistics for local backend"),
+ errhint("Use pg_backend_memory_contexts view instead."));
+ PG_RETURN_NULL();
+ }
+
+ LWLockAcquire(&memCtxState[procNumber].lw_lock, LW_EXCLUSIVE);
+ memCtxState[procNumber].get_summary = get_summary;
+ LWLockRelease(&memCtxState[procNumber].lw_lock);
+
+ curr_timestamp = GetCurrentTimestamp();
+
+ /*
+ * Send a signal to a postgresql process, informing it we want it to
+ * produce information about memory contexts.
+ */
+ if (SendProcSignal(pid, PROCSIG_GET_MEMORY_CONTEXT, procNumber) < 0)
+ {
+ ereport(WARNING,
+ errmsg("could not send signal to process %d: %m", pid));
+ PG_RETURN_NULL();
+ }
+
+ /*
+ * A valid DSA pointer isn't proof that statistics are available, it can
+ * be valid due to previously published stats. Check if the stats are
+ * updated by comparing the timestamp, if the stats are newer than our
+ * previously recorded timestamp from before sending the procsignal, they
+ * must by definition be updated. Wait for max_tries *
+ * MEMSTATS_WAIT_TIMEOUT, following which display old statistics if
+ * available or return NULL.
+ */
+ while (1)
+ {
+ long msecs;
+
+ /*
+ * We expect to come out of sleep when the requested process has
+ * finished publishing the statistics, verified using the valid dsa
+ * pointer.
+ *
+ * Make sure that the information belongs to pid we requested
+ * information for, Otherwise loop back and wait for the server
+ * process to finish publishing statistics.
+ */
+ LWLockAcquire(&memCtxState[procNumber].lw_lock, LW_EXCLUSIVE);
+ msecs =
+ TimestampDifferenceMilliseconds(curr_timestamp,
+ memCtxState[procNumber].stats_timestamp);
+
+ /*
+ * Note in procnumber.h file says that a procNumber can be re-used for
+ * a different backend immediately after a backend exits. In case an
+ * old process' data was there and not updated by the current process
+ * in the slot identified by the procNumber, the pid of the requested
+ * process and the proc_id might not match.
+ */
+ if (memCtxState[procNumber].proc_id == pid)
+ {
+ /*
+ * Break if the latest stats have been read, indicated by
+ * statistics timestamp being newer than the current request
+ * timestamp.
+ */
+ if (DsaPointerIsValid(memCtxState[procNumber].memstats_dsa_pointer)
+ && msecs > 0)
+ {
+ LWLockRelease(&memCtxState[procNumber].lw_lock);
+ break;
+ }
+
+ }
+ LWLockRelease(&memCtxState[procNumber].lw_lock);
+
+ /*
+ * Recheck the state of the backend before sleeping on the condition
+ * variable
+ */
+ proc = BackendPidGetProc(pid);
+
+#define MEMSTATS_WAIT_TIMEOUT 1000
+ if (proc == NULL)
+ proc = AuxiliaryPidGetProc(pid);
+ if (proc == NULL)
+ {
+ ereport(WARNING,
+ errmsg("PID %d is not a PostgreSQL server process",
+ pid));
+ PG_RETURN_NULL();
+ }
+ if (ConditionVariableTimedSleep(&memCtxState[procNumber].memctx_cv,
+ MEMSTATS_WAIT_TIMEOUT,
+ WAIT_EVENT_MEM_CTX_PUBLISH))
+ {
+ /*
+ * Wait for max_tries defined by user, display previously
+ * published statistics if any, when max_tries are over.
+ */
+ if (num_retries > max_tries)
+ {
+ LWLockAcquire(&memCtxState[procNumber].lw_lock, LW_EXCLUSIVE);
+ /* Displaying previously published statistics */
+ if (DsaPointerIsValid(memCtxState[procNumber].memstats_dsa_pointer))
+ {
+ LWLockRelease(&memCtxState[procNumber].lw_lock);
+ break;
+ }
+ else
+ {
+ LWLockRelease(&memCtxState[procNumber].lw_lock);
+ PG_RETURN_NULL();
+ }
+ }
+ ereport(LOG,
+ errmsg("Wait for %d process to publish stats timed out, trying again",
+ pid));
+ num_retries = num_retries + 1;
+ }
+
+ }
+ /* We should land here only with a valid DSA handle */
+ LWLockAcquire(&memCtxArea->lw_lock, LW_EXCLUSIVE);
+ Assert(memCtxArea->memstats_dsa_handle != DSA_HANDLE_INVALID);
+ area = dsa_attach(memCtxArea->memstats_dsa_handle);
+ LWLockRelease(&memCtxArea->lw_lock);
+
+ /*
+ * Backend has finished publishing the stats, read them
+ *
+ * Read statistics of top level 1 and 2 contexts, if get_summary is true.
+ */
+ LWLockAcquire(&memCtxState[procNumber].lw_lock, LW_EXCLUSIVE);
+ memctx_info = (MemoryContextEntry *) dsa_get_address(area,
+ memCtxState[procNumber].memstats_dsa_pointer);
+
+#define PG_GET_PROCESS_MEMORY_CONTEXTS_COLS 12
+ for (int i = 0; i < memCtxState[procNumber].total_stats; i++)
+ {
+ ArrayType *path_array;
+ int path_length;
+ Datum values[PG_GET_PROCESS_MEMORY_CONTEXTS_COLS];
+ bool nulls[PG_GET_PROCESS_MEMORY_CONTEXTS_COLS];
+ char *name;
+ char *ident;
+ Datum *path_datum_array;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+
+ if (DsaPointerIsValid(memctx_info[i].name))
+ {
+ name = (char *) dsa_get_address(area, memctx_info[i].name);
+ values[0] = CStringGetTextDatum(name);
+ }
+ else
+ nulls[0] = true;
+ if (DsaPointerIsValid(memctx_info[i].ident))
+ {
+ ident = (char *) dsa_get_address(area, memctx_info[i].ident);
+ values[1] = CStringGetTextDatum(ident);
+ }
+ else
+ nulls[1] = true;
+
+ if (memctx_info[i].type != NULL)
+ values[2] = CStringGetTextDatum(memctx_info[i].type);
+ else
+ nulls[2] = true;
+
+ path_length = memctx_info[i].path_length;
+
+ if (DsaPointerIsValid(memctx_info[i].path))
+ {
+ path_datum_array = (Datum *) dsa_get_address(area, memctx_info[i].path);
+ path_array = construct_array_builtin(path_datum_array,
+ path_length, INT4OID);
+
+ values[3] = PointerGetDatum(path_array);
+ }
+ else
+ nulls[3] = true;
+ values[4] = Int32GetDatum(path_length); /* level */
+ values[5] = Int64GetDatum(memctx_info[i].totalspace);
+ values[6] = Int64GetDatum(memctx_info[i].nblocks);
+ values[7] = Int64GetDatum(memctx_info[i].freespace);
+ values[8] = Int64GetDatum(memctx_info[i].freechunks);
+ values[9] = Int64GetDatum(memctx_info[i].totalspace -
+ memctx_info[i].freespace);
+ values[10] = Int32GetDatum(memctx_info[i].num_agg_stats);
+ values[11] = TimestampTzGetDatum(memCtxState[procNumber].stats_timestamp);
+
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc,
+ values, nulls);
+ }
+ LWLockRelease(&memCtxState[procNumber].lw_lock);
+
+ ConditionVariableCancelSleep();
+ dsa_detach(area);
+
+ PG_RETURN_NULL();
+}
+
+/*
+ * Shared memory sizing for reporting memory context information.
+ */
+static Size
+MemCtxShmemSize(void)
+{
+ Size TotalProcs =
+ add_size(MaxBackends, add_size(NUM_AUXILIARY_PROCS, max_prepared_xacts));
+
+ return mul_size(TotalProcs, sizeof(MemoryContextBackendState));
+}
+
+/*
+ * Init shared memory for reporting memory context information.
+ */
+void
+MemCtxBackendShmemInit(void)
+{
+ bool found;
+ Size TotalProcs =
+ add_size(MaxBackends, add_size(NUM_AUXILIARY_PROCS, max_prepared_xacts));
+
+ memCtxState = (MemoryContextBackendState *) ShmemInitStruct("MemoryContextBackendState",
+ MemCtxShmemSize(),
+ &found);
+ if (!IsUnderPostmaster)
+ {
+ Assert(!found);
+
+ for (int i = 0; i < TotalProcs; i++)
+ {
+ ConditionVariableInit(&memCtxState[i].memctx_cv);
+
+ LWLockInitialize(&memCtxState[i].lw_lock,
+ LWLockNewTrancheId());
+ LWLockRegisterTranche(memCtxState[i].lw_lock.tranche,
+ "mem_context_backend_stats_reporting");
+
+ memCtxState[i].memstats_dsa_pointer = InvalidDsaPointer;
+ }
+ }
+ else
+ {
+ Assert(found);
+ }
+}
+
+/*
+ * Initialize shared memory for displaying memory
+ * context statistics
+ */
+void
+MemCtxShmemInit(void)
+{
+ bool found;
+
+ memCtxArea = (MemoryContextState *) ShmemInitStruct("MemoryContextState", sizeof(MemoryContextState),
+ &found);
+ if (!IsUnderPostmaster)
+ {
+ Assert(!found);
+
+ LWLockInitialize(&memCtxArea->lw_lock,
+ LWLockNewTrancheId());
+ LWLockRegisterTranche(memCtxArea->lw_lock.tranche,
+ "mem_context_stats_reporting");
+ memCtxArea->memstats_dsa_handle = DSA_HANDLE_INVALID;
+ }
+ else
+ {
+ Assert(found);
+ }
+}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index b844f9fdae..13938ccb0f 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -38,6 +38,7 @@ volatile sig_atomic_t TransactionTimeoutPending = false;
volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
+volatile sig_atomic_t PublishMemoryContextPending = false;
volatile sig_atomic_t IdleStatsUpdateTimeoutPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
diff --git a/src/backend/utils/mmgr/mcxt.c b/src/backend/utils/mmgr/mcxt.c
index 34cdcdf2fd..d1ad444b33 100644
--- a/src/backend/utils/mmgr/mcxt.c
+++ b/src/backend/utils/mmgr/mcxt.c
@@ -19,10 +19,18 @@
*-------------------------------------------------------------------------
*/
+#include <math.h>
#include "postgres.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
+#include "nodes/pg_list.h"
+#include "storage/fd.h"
+#include "storage/lwlock.h"
+#include "storage/dsm.h"
+#include "storage/ipc.h"
+#include "utils/dsa.h"
+#include "utils/hsearch.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
#include "utils/memutils_internal.h"
@@ -164,6 +172,7 @@ MemoryContext CacheMemoryContext = NULL;
MemoryContext MessageContext = NULL;
MemoryContext TopTransactionContext = NULL;
MemoryContext CurTransactionContext = NULL;
+static bool DsaCleanupRegistered = false;
/* This is a transient link to the active portal's memory context: */
MemoryContext PortalContext = NULL;
@@ -177,6 +186,17 @@ static void MemoryContextStatsInternal(MemoryContext context, int level,
static void MemoryContextStatsPrint(MemoryContext context, void *passthru,
const char *stats_string,
bool print_to_stderr);
+static void PublishMemoryContext(MemoryContextEntry *memctx_infos,
+ int curr_id, MemoryContext context,
+ List *path,
+ MemoryContextCounters stat,
+ int num_contexts, dsa_area *area);
+static void compute_contexts_count_and_ids(List *contexts, HTAB *context_id_lookup,
+ int *stats_count,
+ bool get_summary);
+static List *compute_context_path(MemoryContext c, HTAB *context_id_lookup);
+static void dsa_free_previous_stats(dsa_area *area, int total_stats, dsa_pointer prev_dsa_pointer);
+static void AtProcExit_memctx_dsa_free(int code, Datum arg);
/*
* You should not do memory allocations within a critical section, because
@@ -889,7 +909,8 @@ MemoryContextStatsDetail(MemoryContext context,
* One recursion level for MemoryContextStats
*
* Print stats for this context if possible, but in any case accumulate counts
- * into *totals (if not NULL).
+ * into *totals (if not NULL). The callers should make sure that print_location
+ * is set to PRINT_STATS_STDERR or PRINT_STATS_TO_LOGS or PRINT_STATS_NONE.
*/
static void
MemoryContextStatsInternal(MemoryContext context, int level,
@@ -899,36 +920,41 @@ MemoryContextStatsInternal(MemoryContext context, int level,
{
MemoryContext child;
int ichild;
- bool print_to_stderr = true;
check_stack_depth();
Assert(MemoryContextIsValid(context));
if (print_location == PRINT_STATS_TO_STDERR)
- print_to_stderr = true;
+ {
+ /* Examine the context itself */
+ context->methods->stats(context,
+ MemoryContextStatsPrint,
+ &level,
+ totals, true);
+ }
else if (print_location == PRINT_STATS_TO_LOGS)
- print_to_stderr = false;
-
- if (print_location != PRINT_STATS_NONE)
{
+
/* Examine the context itself */
context->methods->stats(context,
MemoryContextStatsPrint,
&level,
- totals, print_to_stderr);
+ totals, false);
}
/*
* Do not print the statistics if print_to_stderr is PRINT_STATS_NONE,
- * only compute totals.
+ * only compute totals. This is used in reporting of memory context
+ * statistics via a sql function. Last parameter is not relevant.
*/
else
{
+ Assert(print_location == PRINT_STATS_NONE);
/* Examine the context itself */
context->methods->stats(context,
NULL,
NULL,
- totals, print_to_stderr);
+ totals, false);
}
/* Increment the context count */
*num_contexts = *num_contexts + 1;
@@ -971,7 +997,7 @@ MemoryContextStatsInternal(MemoryContext context, int level,
}
*num_contexts = *num_contexts + ichild;
- if (print_to_stderr)
+ if (print_location == PRINT_STATS_TO_STDERR)
{
for (int i = 0; i <= level; i++)
fprintf(stderr, " ");
@@ -984,7 +1010,7 @@ MemoryContextStatsInternal(MemoryContext context, int level,
local_totals.freechunks,
local_totals.totalspace - local_totals.freespace);
}
- else if (print_location != PRINT_STATS_NONE)
+ else if (print_location == PRINT_STATS_TO_LOGS)
ereport(LOG_SERVER_ONLY,
(errhidestmt(true),
errhidecontext(true),
@@ -1321,6 +1347,21 @@ HandleLogMemoryContextInterrupt(void)
/* latch will be set by procsignal_sigusr1_handler */
}
+/*
+ * HandleGetMemoryContextInterrupt
+ * Handle receipt of an interrupt indicating publishing of memory
+ * contexts.
+ *
+ * All the actual work is deferred to ProcessLogMemoryContextInterrupt()
+ */
+void
+HandleGetMemoryContextInterrupt(void)
+{
+ InterruptPending = true;
+ PublishMemoryContextPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
/*
* ProcessLogMemoryContextInterrupt
* Perform logging of memory contexts of this backend process.
@@ -1358,6 +1399,510 @@ ProcessLogMemoryContextInterrupt(void)
MemoryContextStatsDetail(TopMemoryContext, 100, 100, false);
}
+/*
+ * ProcessGetMemoryContextInterrupt
+ * Generate information about memory contexts used by the process.
+ *
+ * Performs a breadth first search on the memory context tree, thus parents
+ * statistics are reported before children in the monitoring function output.
+ *
+ * Statistics per context for all the processes are shared via the same dynamic
+ * shared area. The statistics for contexts that exceed the pre-determined size
+ * limit, are captured as a cumulative total at the end of individual statistics.
+ *
+ * If get_summary is true, we traverse the memory context tree recursively in
+ * depth first search manner to cover all the children of a parent context, to be
+ * able to display a cumulative total of memory consumption by a parent.
+ */
+void
+ProcessGetMemoryContextInterrupt(void)
+{
+ List *contexts;
+
+ HASHCTL ctl;
+ HTAB *context_id_lookup;
+ int context_id = 0;
+ MemoryContext stat_cxt;
+ MemoryContextEntry *meminfo;
+ bool get_summary = false;
+
+ dsa_area *area = NULL;
+ int max_stats;
+ int idx = MyProcNumber;
+ int stats_count = 0;
+ MemoryContextCounters stat;
+ int num_individual_stats = 0;
+
+ PublishMemoryContextPending = false;
+
+ /*
+ * Make a new context that will contain the hash table, to ease the
+ * cleanup.
+ */
+ stat_cxt = AllocSetContextCreate(CurrentMemoryContext,
+ "Memory context statistics",
+ ALLOCSET_DEFAULT_SIZES);
+
+ /*
+ * The hash table used for constructing "path" column of the view, similar
+ * to its local backend counterpart.
+ */
+ ctl.keysize = sizeof(MemoryContext);
+ ctl.entrysize = sizeof(MemoryContextId);
+ ctl.hcxt = stat_cxt;
+
+ context_id_lookup = hash_create("pg_get_remote_backend_memory_contexts",
+ 256,
+ &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+
+ /* List of contexts to process in the next round - start at the top. */
+ contexts = list_make1(TopMemoryContext);
+
+ /* Compute the number of stats that can fit in the defined limit */
+ max_stats = (MAX_NUM_DEFAULT_SEGMENTS * DSA_DEFAULT_INIT_SEGMENT_SIZE)
+ / (sizeof(MemoryContextEntry) + (MEM_CONTEXT_MAX_LEVEL
+ * sizeof(Datum)) + (2 * MEMORY_CONTEXT_IDENT_SHMEM_SIZE));
+
+ LWLockAcquire(&memCtxState[idx].lw_lock, LW_EXCLUSIVE);
+ get_summary = memCtxState[idx].get_summary;
+ LWLockRelease(&memCtxState[idx].lw_lock);
+
+ /*
+ * Traverse the memory context tree to find total number of contexts. If
+ * summary is requested report the total number of contexts at level 1 and
+ * 2 from the top. Also, populate the hash table of context ids.
+ */
+ compute_contexts_count_and_ids(contexts, context_id_lookup, &stats_count,
+ get_summary);
+
+ /*
+ * Allocate memory in this process's dsa for storing statistics of the the
+ * memory contexts upto max_stats, for contexts that don't fit within a
+ * limit, a cumulative total is written as the last record in the DSA
+ * segment.
+ */
+ stats_count = (stats_count > max_stats) ? max_stats : stats_count;
+
+ LWLockAcquire(&memCtxArea->lw_lock, LW_EXCLUSIVE);
+
+ /*
+ * Create a DSA and send handle to the the client process after storing
+ * the context statistics. If number of contexts exceed a predefined
+ * limit(8MB), a cumulative total is stored for such contexts.
+ */
+ if (memCtxArea->memstats_dsa_handle == DSA_HANDLE_INVALID)
+ {
+ MemoryContext oldcontext = CurrentMemoryContext;
+ dsa_handle handle;
+
+ MemoryContextSwitchTo(TopMemoryContext);
+
+ area = dsa_create(memCtxArea->lw_lock.tranche);
+
+ handle = dsa_get_handle(area);
+ MemoryContextSwitchTo(oldcontext);
+
+ dsa_pin_mapping(area);
+
+ /*
+ * Pin the dsa area, this is to make sure the area remains attachable
+ * even if current backend exits. This is done so that a waiting
+ * client gets the stats even after a process exits.
+ */
+ dsa_pin(area);
+
+ /* Set the handle in shared memory */
+ memCtxArea->memstats_dsa_handle = handle;
+ }
+
+ /*
+ * If DSA exists, created by another process publishing statistics, attach
+ * to it.
+ */
+ else if (area == NULL)
+ {
+ MemoryContext oldcontext = CurrentMemoryContext;
+
+ MemoryContextSwitchTo(TopMemoryContext);
+ area = dsa_attach(memCtxArea->memstats_dsa_handle);
+ MemoryContextSwitchTo(oldcontext);
+ dsa_pin_mapping(area);
+ }
+ LWLockRelease(&memCtxArea->lw_lock);
+
+ /*
+ * Register a callback to free the allocated dsa pointers on process exit
+ * This is done after we have a valid dsa area.
+ */
+ if (!DsaCleanupRegistered)
+ {
+ before_shmem_exit(AtProcExit_memctx_dsa_free, PointerGetDatum(area));
+ DsaCleanupRegistered = true;
+ }
+
+ /*
+ * Hold the process specific lock to protect writes to process specific
+ * memory. This way two processes publishing statistics do not block each
+ * other.
+ */
+ LWLockAcquire(&memCtxState[idx].lw_lock, LW_EXCLUSIVE);
+ memCtxState[idx].proc_id = MyProcPid;
+
+ if (DsaPointerIsValid(memCtxState[idx].memstats_dsa_pointer))
+ {
+ /*
+ * Free any previous allocations, free the name, ident and path
+ * pointers before freeing the pointer that contains them.
+ */
+ dsa_free_previous_stats(area, memCtxState[idx].total_stats,
+ memCtxState[idx].memstats_dsa_pointer);
+ dsa_free(area, memCtxState[idx].memstats_dsa_pointer);
+ memCtxState[idx].memstats_dsa_pointer = InvalidDsaPointer;
+ }
+ memCtxState[idx].memstats_dsa_pointer = dsa_allocate0(area,
+ stats_count * sizeof(MemoryContextEntry));
+
+ meminfo = (MemoryContextEntry *) dsa_get_address(area,
+ memCtxState[idx].memstats_dsa_pointer);
+
+ if (get_summary)
+ {
+ int ctx_id = 0;
+ List *path = NIL;
+
+ /* Copy TopMemoryContext statistics to DSA */
+ memset(&stat, 0, sizeof(stat));
+ (*TopMemoryContext->methods->stats) (TopMemoryContext, NULL, NULL,
+ &stat, true);
+ path = lcons_int(1, path);
+ PublishMemoryContext(meminfo, ctx_id, TopMemoryContext, path, stat,
+ 1, area);
+ ctx_id = ctx_id + 1;
+
+ /*
+ * Copy statistics for each of TopMemoryContexts children(XXX. Make it
+ * capped at 100). This includes statistics of all of their children
+ * upto level 100.
+ */
+
+ for (MemoryContext c = TopMemoryContext->firstchild; c != NULL;
+ c = c->nextchild)
+ {
+ MemoryContextCounters grand_totals;
+ int num_contexts = 0;
+
+ path = NIL;
+ memset(&grand_totals, 0, sizeof(grand_totals));
+
+ MemoryContextStatsInternal(c, 0, 100, 100, &grand_totals,
+ PRINT_STATS_NONE, &num_contexts);
+
+ path = compute_context_path(c, context_id_lookup);
+
+ PublishMemoryContext(meminfo, ctx_id, c, path,
+ grand_totals, num_contexts, area);
+ ctx_id = ctx_id + 1;
+ }
+ memCtxState[idx].total_stats = ctx_id;
+ goto cleanup;
+ }
+ foreach_ptr(MemoryContextData, cur, contexts)
+ {
+ List *path = NIL;
+ char *name;
+
+ /*
+ * Figure out the transient context_id of this context and each of its
+ * ancestors.
+ */
+ path = compute_context_path(cur, context_id_lookup);
+
+ if (context_id <= (max_stats - 2))
+ {
+ /* Examine the context stats */
+ memset(&stat, 0, sizeof(stat));
+ (*cur->methods->stats) (cur, NULL, NULL, &stat, true);
+ /* Copy statistics to DSA memory */
+ PublishMemoryContext(meminfo, context_id, cur, path, stat, 1, area);
+ }
+ else
+ {
+ /* Examine the context stats */
+ memset(&stat, 0, sizeof(stat));
+ (*cur->methods->stats) (cur, NULL, NULL, &stat, true);
+
+ meminfo[max_stats - 1].totalspace += stat.totalspace;
+ meminfo[max_stats - 1].nblocks += stat.nblocks;
+ meminfo[max_stats - 1].freespace += stat.freespace;
+ meminfo[max_stats - 1].freechunks += stat.freechunks;
+ }
+
+ /*
+ * DSA max limit is reached, write aggregate of the remaining
+ * statistics.
+ */
+ if (context_id == (max_stats - 2) && context_id < (stats_count - 1))
+ {
+ num_individual_stats = context_id + 1;
+ meminfo[max_stats - 1].name = dsa_allocate0(area, 17);
+ name = dsa_get_address(area, meminfo[max_stats - 1].name);
+ strncpy(name, "Remaining Totals", 16);
+ meminfo[max_stats - 1].ident = InvalidDsaPointer;
+ meminfo[max_stats - 1].path = InvalidDsaPointer;
+ meminfo[max_stats - 1].type = NULL;
+ }
+ context_id++;
+ }
+ /* No aggregated contexts, individual statistics reported */
+ if (context_id < (max_stats - 2))
+ {
+ memCtxState[idx].total_stats = context_id;
+ }
+ /* Report number of aggregated memory contexts */
+ else
+ {
+ meminfo[max_stats - 1].num_agg_stats = context_id -
+ num_individual_stats;
+
+ /*
+ * Total stats equals num_individual_stats + 1 record for cumulative
+ * statistics.
+ */
+ memCtxState[idx].total_stats = num_individual_stats + 1;
+ }
+cleanup:
+
+ /*
+ * Signal all the waiting client backends after setting the exit condition
+ * flag
+ */
+ memCtxState[idx].stats_timestamp = GetCurrentTimestamp();
+ LWLockRelease(&memCtxState[idx].lw_lock);
+ ConditionVariableBroadcast(&memCtxState[idx].memctx_cv);
+ /* Delete the hash table memory context */
+ MemoryContextDelete(stat_cxt);
+
+ dsa_detach(area);
+}
+
+/*
+ * Append the transient context_id of this context and each of
+ * its ancestors to a list, in order to compute a path.
+ */
+static List *
+compute_context_path(MemoryContext c, HTAB *context_id_lookup)
+{
+ bool found;
+ List *path = NIL;
+
+ for (MemoryContext cur_context = c; cur_context != NULL; cur_context = cur_context->parent)
+ {
+ MemoryContextId *cur_entry;
+
+ cur_entry = hash_search(context_id_lookup, &cur_context, HASH_FIND, &found);
+
+ if (!found)
+ {
+ ereport(LOG,
+ errmsg("hash table corrupted, can't construct path value"));
+ break;
+ }
+ path = lcons_int(cur_entry->context_id, path);
+ }
+ return path;
+}
+
+/*
+ * Return the number of contexts allocated currently by the backend
+ * Assign context ids to each of the contexts.
+ */
+static void
+compute_contexts_count_and_ids(List *contexts, HTAB *context_id_lookup,
+ int *stats_count, bool get_summary)
+{
+ foreach_ptr(MemoryContextData, cur, contexts)
+ {
+ MemoryContextId *entry;
+ bool found;
+
+ entry = (MemoryContextId *) hash_search(context_id_lookup, &cur,
+ HASH_ENTER, &found);
+ Assert(!found);
+
+ /* context id starts with 1 */
+ entry->context_id = (++(*stats_count));
+
+ /* Append the children of the current context to the main list. */
+ for (MemoryContext c = cur->firstchild; c != NULL; c = c->nextchild)
+ {
+ if (get_summary)
+ {
+ entry = (MemoryContextId *) hash_search(context_id_lookup, &c,
+ HASH_ENTER, &found);
+ Assert(!found);
+
+ entry->context_id = (++(*stats_count));
+ }
+
+ contexts = lappend(contexts, c);
+ }
+
+ /*
+ * In summary only the first two level(from top) contexts are
+ * displayed
+ */
+ if (get_summary)
+ break;
+ }
+
+}
+
+/* Copy the memory context statistics of a single context to a dsa memory */
+static void
+PublishMemoryContext(MemoryContextEntry *memctx_info, int curr_id,
+ MemoryContext context, List *path,
+ MemoryContextCounters stat, int num_contexts,
+ dsa_area *area)
+{
+ char clipped_ident[MEMORY_CONTEXT_IDENT_SHMEM_SIZE];
+ char *name;
+ char *ident;
+ Datum *path_array;
+
+ if (context->name != NULL)
+ {
+ Assert(strlen(context->name) < MEMORY_CONTEXT_IDENT_SHMEM_SIZE);
+ memctx_info[curr_id].name = dsa_allocate0(area, strlen(context->name) + 1);
+ name = (char *) dsa_get_address(area, memctx_info[curr_id].name);
+ strncpy(name, context->name, strlen(context->name));
+ }
+ else
+ memctx_info[curr_id].name = InvalidDsaPointer;
+
+ /* Trim and copy the identifier if it is not set to NULL */
+ if (context->ident != NULL)
+ {
+ int idlen = strlen(context->ident);
+
+ /*
+ * Some identifiers such as SQL query string can be very long,
+ * truncate oversize identifiers.
+ */
+ if (idlen >= MEMORY_CONTEXT_IDENT_SHMEM_SIZE)
+ idlen = pg_mbcliplen(context->ident, idlen,
+ MEMORY_CONTEXT_IDENT_SHMEM_SIZE - 1);
+
+ memcpy(clipped_ident, context->ident, idlen);
+ clipped_ident[idlen] = '\0';
+
+ /*
+ * To be consistent with logging output, we label dynahash contexts
+ * with just the hash table name as with MemoryContextStatsPrint().
+ */
+ if (!strncmp(context->name, "dynahash", 8))
+ {
+ dsa_free(area, memctx_info[curr_id].name);
+ memctx_info[curr_id].name = dsa_allocate0(area,
+ strlen(clipped_ident) + 1);
+ name = (char *) dsa_get_address(area,
+ memctx_info[curr_id].name);
+ strlcpy(name,
+ clipped_ident, idlen + 1);
+ memctx_info[curr_id].ident = InvalidDsaPointer;
+ }
+ else
+ {
+
+ memctx_info[curr_id].ident = dsa_allocate0(area,
+ strlen(clipped_ident) + 1);
+ ident = (char *) dsa_get_address(area,
+ memctx_info[curr_id].ident);
+ strlcpy(ident,
+ clipped_ident, idlen + 1);
+ }
+ }
+ else
+ memctx_info[curr_id].ident = InvalidDsaPointer;
+ /* Allocate dsa memory for storing path information */
+ if (path == NIL)
+ memctx_info[curr_id].path = InvalidDsaPointer;
+ else
+ {
+ memctx_info[curr_id].path_length = list_length(path);
+ memctx_info[curr_id].path = dsa_allocate0(area,
+ memctx_info[curr_id].path_length
+ * sizeof(Datum));
+ path_array = (Datum *) dsa_get_address(area, memctx_info[curr_id].path);
+ foreach_int(i, path)
+ path_array[foreach_current_index(i)] = Int32GetDatum(i);
+ }
+ memctx_info[curr_id].type = ContextTypeToString(context->type);
+ memctx_info[curr_id].totalspace = stat.totalspace;
+ memctx_info[curr_id].nblocks = stat.nblocks;
+ memctx_info[curr_id].freespace = stat.freespace;
+ memctx_info[curr_id].freechunks = stat.freechunks;
+ memctx_info[curr_id].num_agg_stats = num_contexts;
+}
+
+static void
+dsa_free_previous_stats(dsa_area *area, int total_stats,
+ dsa_pointer prev_dsa_pointer)
+{
+ MemoryContextEntry *meminfo;
+
+ meminfo = (MemoryContextEntry *) dsa_get_address(area, prev_dsa_pointer);
+ for (int i = 0; i < total_stats; i++)
+ {
+ if (DsaPointerIsValid(meminfo[i].name))
+ dsa_free(area, meminfo[i].name);
+
+ if (DsaPointerIsValid(meminfo[i].ident))
+ dsa_free(area, meminfo[i].ident);
+
+ if (DsaPointerIsValid(meminfo[i].path))
+ dsa_free(area, meminfo[i].path);
+ }
+}
+
+/*
+ * Free the memory context statistics stored by this process
+ * in dsa area.
+ */
+static void
+AtProcExit_memctx_dsa_free(int code, Datum arg)
+{
+ int idx = MyProcNumber;
+ dsm_segment *dsm_seg = NULL;
+ dsa_area *area;
+
+ if (memCtxArea->memstats_dsa_handle != DSA_HANDLE_INVALID)
+ dsm_seg = dsm_find_mapping(memCtxArea->memstats_dsa_handle);
+ else
+ return;
+
+ if (dsm_seg == NULL)
+ area = dsa_attach(memCtxArea->memstats_dsa_handle);
+ else
+ area = (dsa_area *) DatumGetPointer(arg);
+
+ LWLockAcquire(&memCtxState[idx].lw_lock, LW_EXCLUSIVE);
+
+ if (DsaPointerIsValid(memCtxState[idx].memstats_dsa_pointer))
+ {
+ /*
+ * Free the memory context statistics, free the name, ident and path
+ * pointers before freeing the pointer that contains these pointers
+ * and integer statistics.
+ */
+ dsa_free_previous_stats(area, memCtxState[idx].total_stats,
+ memCtxState[idx].memstats_dsa_pointer);
+ dsa_free(area, memCtxState[idx].memstats_dsa_pointer);
+ memCtxState[idx].memstats_dsa_pointer = InvalidDsaPointer;
+ }
+ LWLockRelease(&memCtxState[idx].lw_lock);
+}
+
void *
palloc(Size size)
{
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 42e427f8fe..a999206b07 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8506,6 +8506,16 @@
prorettype => 'bool', proargtypes => 'int4',
prosrc => 'pg_log_backend_memory_contexts' },
+# publishing memory contexts of the specified postgres process
+{ oid => '2173', descr => 'publish memory contexts of the specified backend',
+ proname => 'pg_get_process_memory_contexts', provolatile => 'v',
+ prorows => '100', proretset => 't', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4 bool int4',
+ proallargtypes => '{int4,bool,int4,text,text,text,_int4,int4,int8,int8,int8,int8,int8,int4,timestamptz}',
+ proargmodes => '{i,i,i,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{oid, summary, num_of_tries, name, ident, type, path, level, total_bytes, total_nblocks, free_bytes, free_chunks, used_bytes, num_agg_contexts, stats_timestamp}',
+ prosrc => 'pg_get_process_memory_contexts' },
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 6f16794eb6..63f74e3ce1 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -96,6 +96,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleStatsUpdateTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t PublishMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 022fd8ed93..5d4b2fbfc9 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_GET_MEMORY_CONTEXT, /* ask backend to send the memory contexts */
PROCSIG_PARALLEL_APPLY_MESSAGE, /* Message from parallel apply workers */
/* Recovery conflict reasons */
diff --git a/src/include/utils/memutils.h b/src/include/utils/memutils.h
index 8abc26abce..a8d1956a82 100644
--- a/src/include/utils/memutils.h
+++ b/src/include/utils/memutils.h
@@ -18,6 +18,9 @@
#define MEMUTILS_H
#include "nodes/memnodes.h"
+#include "storage/condition_variable.h"
+#include "storage/lmgr.h"
+#include "utils/dsa.h"
/*
@@ -48,7 +51,10 @@
#define AllocHugeSizeIsValid(size) ((Size) (size) <= MaxAllocHugeSize)
+#define MEMORY_CONTEXT_IDENT_SHMEM_SIZE 128
+#define MEM_CONTEXT_MAX_LEVEL 64
+#define MAX_NUM_DEFAULT_SEGMENTS 8
/*
* Standard top-level memory contexts.
*
@@ -319,4 +325,67 @@ pg_memory_is_all_zeros(const void *ptr, size_t len)
return true;
}
+/* Dynamic shared memory state for statistics per context */
+typedef struct MemoryContextEntry
+{
+ dsa_pointer name;
+ dsa_pointer ident;
+ dsa_pointer path;
+ const char *type;
+ int path_length;
+ int64 totalspace;
+ int64 nblocks;
+ int64 freespace;
+ int64 freechunks;
+ int num_agg_stats;
+} MemoryContextEntry;
+
+/*
+ * Static shared memory state representing the DSA area
+ * created for memory context statistics reporting.
+ * Single DSA area is created and used by all the processes,
+ * each having its specific dsa allocations for sharing memory
+ * statistics, tracked by per backend static shared memory state.
+ */
+typedef struct MemoryContextState
+{
+ dsa_handle memstats_dsa_handle;
+ LWLock lw_lock;
+} MemoryContextState;
+
+/*
+ * Per backend static shared memory state for memory
+ * context statistics reporting.
+ */
+typedef struct MemoryContextBackendState
+{
+ ConditionVariable memctx_cv;
+ LWLock lw_lock;
+ int proc_id;
+ int total_stats;
+ bool get_summary;
+ dsa_pointer memstats_dsa_pointer;
+ TimestampTz stats_timestamp;
+} MemoryContextBackendState;
+
+
+/*
+ * MemoryContextId
+ * Used for storage of transient identifiers for
+ * pg_get_backend_memory_contexts.
+ */
+typedef struct MemoryContextId
+{
+ MemoryContext context;
+ int context_id;
+} MemoryContextId;
+
+extern PGDLLIMPORT MemoryContextBackendState *memCtxState;
+extern PGDLLIMPORT MemoryContextState *memCtxArea;
+extern void ProcessGetMemoryContextInterrupt(void);
+extern const char *ContextTypeToString(NodeTag type);
+extern void HandleGetMemoryContextInterrupt(void);
+extern void MemCtxShmemInit(void);
+extern void MemCtxBackendShmemInit(void);
+
#endif /* MEMUTILS_H */
diff --git a/src/test/regress/expected/sysviews.out b/src/test/regress/expected/sysviews.out
index 83228cfca2..dca20ae1a2 100644
--- a/src/test/regress/expected/sysviews.out
+++ b/src/test/regress/expected/sysviews.out
@@ -232,3 +232,17 @@ select * from pg_timezone_abbrevs where abbrev = 'LMT';
LMT | @ 7 hours 52 mins 58 secs ago | f
(1 row)
+DO $$
+DECLARE
+ launcher_pid int;
+ r RECORD;
+BEGIN
+ SELECT pid from pg_stat_activity where backend_type='autovacuum launcher'
+ INTO launcher_pid;
+
+ select type, name, ident
+ from pg_get_process_memory_contexts(launcher_pid, false, 20)
+ where path = '{1}' into r;
+ RAISE NOTICE '%', r;
+END $$;
+NOTICE: (AllocSet,TopMemoryContext,)
diff --git a/src/test/regress/sql/sysviews.sql b/src/test/regress/sql/sysviews.sql
index 66179f026b..4767351d4e 100644
--- a/src/test/regress/sql/sysviews.sql
+++ b/src/test/regress/sql/sysviews.sql
@@ -101,3 +101,17 @@ select count(distinct utc_offset) >= 24 as ok from pg_timezone_abbrevs;
-- One specific case we can check without much fear of breakage
-- is the historical local-mean-time value used for America/Los_Angeles.
select * from pg_timezone_abbrevs where abbrev = 'LMT';
+
+DO $$
+DECLARE
+ launcher_pid int;
+ r RECORD;
+BEGIN
+ SELECT pid from pg_stat_activity where backend_type='autovacuum launcher'
+ INTO launcher_pid;
+
+ select type, name, ident
+ from pg_get_process_memory_contexts(launcher_pid, false, 20)
+ where path = '{1}' into r;
+ RAISE NOTICE '%', r;
+END $$;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 93339ef3c5..11e6dad561 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1643,12 +1643,16 @@ MemoizeState
MemoizeTuple
MemoryChunk
MemoryContext
+MemoryContextBackendState
MemoryContextCallback
MemoryContextCallbackFunction
MemoryContextCounters
MemoryContextData
+MemoryContextEntry
+MemoryContextId
MemoryContextMethodID
MemoryContextMethods
+MemoryContextState
MemoryStatsPrintFunc
MergeAction
MergeActionState
--
2.34.1
view thread (58+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected]
Subject: Re: Enhancing Memory Context Statistics Reporting
In-Reply-To: <CAH2L28vULzqoit+YCKR5UhdT+AR+b1Qcs6Hgpz6nQz6NBT2jug@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox