public inbox for [email protected]
help / color / mirror / Atom feedAdd the ability to limit the amount of memory that can be allocated to backends.
22+ messages / 12 participants
[nested] [flat]
* Add the ability to limit the amount of memory that can be allocated to backends.
@ 2022-08-31 16:50 Reid Thompson <[email protected]>
2022-08-31 17:34 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Justin Pryzby <[email protected]>
2022-09-01 02:48 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Kyotaro Horiguchi <[email protected]>
2022-09-02 07:30 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Drouvot, Bertrand <[email protected]>
2022-09-02 08:52 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. David Rowley <[email protected]>
0 siblings, 4 replies; 22+ messages in thread
From: Reid Thompson @ 2022-08-31 16:50 UTC (permalink / raw)
To: pgsql-hackers <[email protected]>; +Cc: [email protected]
Hi Hackers,
Add the ability to limit the amount of memory that can be allocated to
backends.
This builds on the work that adds backend memory allocated to
pg_stat_activity
https://www.postgresql.org/message-id/67bb5c15c0489cb499723b0340f16e10c22485ec.camel%40crunchydata.c...
Both patches are attached.
Add GUC variable max_total_backend_memory.
Specifies a limit to the amount of memory (MB) that may be allocated to
backends in total (i.e. this is not a per user or per backend limit).
If unset, or set to 0 it is disabled. It is intended as a resource to
help avoid the OOM killer. A backend request that would push the total
over the limit will be denied with an out of memory error causing that
backends current query/transaction to fail. Due to the dynamic nature
of memory allocations, this limit is not exact. If within 1.5MB of the
limit and two backends request 1MB each at the same time both may be
allocated exceeding the limit. Further requests will not be allocated
until dropping below the limit. Keep this in mind when setting this
value to avoid the OOM killer. Currently, this limit does not affect
auxiliary backend processes, this list of non-affected backend
processes is open for discussion as to what should/should not be
included. Backend memory allocations are displayed in the
pg_stat_activity view.
--
Reid Thompson
Senior Software Engineer
Crunchy Data, Inc.
[email protected]
www.crunchydata.com
Attachments:
[text/x-patch] 001-dev-max-memory.patch (11.9K, ../../[email protected]/2-001-dev-max-memory.patch)
download | inline diff:
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index a5cd4e44c7..caf958310a 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -2079,6 +2079,32 @@ include_dir 'conf.d'
</listitem>
</varlistentry>
+ <varlistentry id="guc-max-total-backend-memory" xreflabel="max_total_backend_memory">
+ <term><varname>max_total_backend_memory</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>max_total_backend_memory</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies a limit to the amount of memory (MB) that may be allocated to
+ backends in total (i.e. this is not a per user or per backend limit).
+ If unset, or set to 0 it is disabled. A backend request that would push
+ the total over the limit will be denied with an out of memory error
+ causing that backends current query/transaction to fail. Due to the dynamic
+ nature of memory allocations, this limit is not exact. If within 1.5MB of
+ the limit and two backends request 1MB each at the same time both may be
+ allocated exceeding the limit. Further requests will not be allocated until
+ dropping below the limit. Keep this in mind when setting this value. This
+ limit does not affect auxiliary backend processes
+ <xref linkend="glossary-auxiliary-proc"/> . Backend memory allocations
+ (<varname>backend_mem_allocated</varname>) are displayed in the
+ <link linkend="monitoring-pg-stat-activity-view"><structname>pg_stat_activity</structname></link>
+ view.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/src/backend/storage/ipc/dsm_impl.c b/src/backend/storage/ipc/dsm_impl.c
index 269ad2fe53..808ffe75f2 100644
--- a/src/backend/storage/ipc/dsm_impl.c
+++ b/src/backend/storage/ipc/dsm_impl.c
@@ -253,6 +253,10 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
return true;
}
+ /* Do not exceed maximum allowed memory allocation */
+ if (op == DSM_OP_CREATE && exceeds_max_total_bkend_mem(request_size))
+ return false;
+
/*
* Create new segment or open an existing one for attach.
*
@@ -524,6 +528,10 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size,
int flags = IPCProtection;
size_t segsize;
+ /* Do not exceed maximum allowed memory allocation */
+ if (op == DSM_OP_CREATE && exceeds_max_total_bkend_mem(request_size))
+ return false;
+
/*
* Allocate the memory BEFORE acquiring the resource, so that we don't
* leak the resource if memory allocation fails.
@@ -718,6 +726,10 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size,
return true;
}
+ /* Do not exceed maximum allowed memory allocation */
+ if (op == DSM_OP_CREATE && exceeds_max_total_bkend_mem(request_size))
+ return false;
+
/* Create new segment or open an existing one for attach. */
if (op == DSM_OP_CREATE)
{
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 17a00587f8..9137a000ae 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -44,6 +44,8 @@
*/
bool pgstat_track_activities = false;
int pgstat_track_activity_query_size = 1024;
+/* Max backend memory allocation allowed (MB). 0 = disabled */
+int max_total_bkend_mem = 0;
/* exposed so that backend_progress.c can access it */
@@ -1253,3 +1255,107 @@ pgstat_report_backend_mem_allocated_decrease(uint64 deallocation)
beentry->backend_mem_allocated -= deallocation;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
+
+/* ----------
+ * pgstat_get_all_backend_memory_allocated() -
+ *
+ * Return a uint64 representing the current shared memory allocated to all
+ * backends. This looks directly at the BackendStatusArray, and so will
+ * provide current information regardless of the age of our transaction's
+ * snapshot of the status array.
+ * In the future we will likely utilize additional values - perhaps limit
+ * backend allocation by user/role, etc.
+ * ----------
+ */
+uint64
+pgstat_get_all_backend_memory_allocated(void)
+{
+ PgBackendStatus *beentry;
+ int i;
+ uint64 all_backend_memory_allocated = 0;
+
+ beentry = BackendStatusArray;
+
+ /*
+ * We probably shouldn't get here before shared memory has been set up,
+ * but be safe.
+ */
+ if (beentry == NULL || BackendActivityBuffer == NULL)
+ return 0;
+
+ /*
+ * We include AUX procs in all backend memory calculation
+ */
+ for (i = 1; i <= NumBackendStatSlots; i++)
+ {
+ /*
+ * We use a volatile pointer here to ensure the compiler doesn't try to
+ * get cute.
+ */
+ volatile PgBackendStatus *vbeentry = beentry;
+ bool found;
+ uint64 backend_mem_allocated = 0;
+
+ for (;;)
+ {
+ int before_changecount;
+ int after_changecount;
+
+ pgstat_begin_read_activity(vbeentry, before_changecount);
+
+ /* Ignore invalid entries, which may contain invalid data.
+ * See pgstat_beshutdown_hook()
+ */
+ if (vbeentry->st_procpid > 0)
+ backend_mem_allocated = vbeentry->backend_mem_allocated;
+
+ pgstat_end_read_activity(vbeentry, after_changecount);
+
+ if ((found = pgstat_read_activity_complete(before_changecount,
+ after_changecount)))
+ break;
+
+ /* Make sure we can break out of loop if stuck... */
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ if (found)
+ all_backend_memory_allocated += backend_mem_allocated;
+
+ beentry++;
+ }
+
+ return all_backend_memory_allocated;
+}
+
+/*
+ * Determine if allocation request will exceed max backend memory allowed.
+ * Do not apply to auxiliary processes.
+ */
+bool
+exceeds_max_total_bkend_mem(uint64 allocation_request)
+{
+ bool result = false;
+
+ if (MyAuxProcType != NotAnAuxProcess)
+ return result;
+
+ /* Convert max_total_bkend_mem to bytes for comparison */
+ if (max_total_bkend_mem &&
+ pgstat_get_all_backend_memory_allocated() +
+ allocation_request > (uint64)max_total_bkend_mem * 1024 * 1024)
+ {
+ /*
+ * Explicitely identify the OOM being a result of this
+ * configuration parameter vs a system failure to allocate OOM.
+ */
+ elog(WARNING,
+ "request will exceed postgresql.conf defined max_total_backend_memory limit (%lu > %lu)",
+ pgstat_get_all_backend_memory_allocated() +
+ allocation_request, (uint64)max_total_bkend_mem * 1024 * 1024);
+
+ result = true;
+ }
+
+ return result;
+}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 9fbbfb1be5..ab8d83c235 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -3664,6 +3664,17 @@ static struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_total_backend_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Restrict total backend memory allocations to this max."),
+ gettext_noop("0 turns this feature off."),
+ GUC_UNIT_MB
+ },
+ &max_total_bkend_mem,
+ 0, 0, INT_MAX,
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, 0, 0, 0, NULL, NULL, NULL
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 90bec0502c..8e944f6511 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -155,6 +155,9 @@
# mmap
# (change requires restart)
#min_dynamic_shared_memory = 0MB # (change requires restart)
+#max_total_backend_memory = 0MB # Restrict total backend memory allocations
+ # to this max (in MB). 0 turns this feature
+ # off.
# - Disk -
diff --git a/src/backend/utils/mmgr/aset.c b/src/backend/utils/mmgr/aset.c
index c91f8efa4d..ac9a1ced3f 100644
--- a/src/backend/utils/mmgr/aset.c
+++ b/src/backend/utils/mmgr/aset.c
@@ -428,6 +428,10 @@ AllocSetContextCreateInternal(MemoryContext parent,
else
firstBlockSize = Max(firstBlockSize, initBlockSize);
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(firstBlockSize))
+ return NULL;
+
/*
* Allocate the initial block. Unlike other aset.c blocks, it starts with
* the context header and its block header follows that.
@@ -720,6 +724,11 @@ AllocSetAlloc(MemoryContext context, Size size)
{
chunk_size = MAXALIGN(size);
blksize = chunk_size + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ;
+
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
block = (AllocBlock) malloc(blksize);
if (block == NULL)
return NULL;
@@ -911,6 +920,10 @@ AllocSetAlloc(MemoryContext context, Size size)
while (blksize < required_size)
blksize <<= 1;
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
/* Try to allocate it */
block = (AllocBlock) malloc(blksize);
@@ -1133,6 +1146,10 @@ AllocSetRealloc(void *pointer, Size size)
blksize = chksize + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ;
oldblksize = block->endptr - ((char *) block);
+ /* Do not exceed maximum allowed memory allocation */
+ if (blksize > oldblksize && exceeds_max_total_bkend_mem(blksize - oldblksize))
+ return NULL;
+
block = (AllocBlock) realloc(block, blksize);
if (block == NULL)
{
diff --git a/src/backend/utils/mmgr/generation.c b/src/backend/utils/mmgr/generation.c
index 36e5b3f94d..1d5720836c 100644
--- a/src/backend/utils/mmgr/generation.c
+++ b/src/backend/utils/mmgr/generation.c
@@ -192,6 +192,9 @@ GenerationContextCreate(MemoryContext parent,
else
allocSize = Max(allocSize, initBlockSize);
+ if (exceeds_max_total_bkend_mem(allocSize))
+ return NULL;
+
/*
* Allocate the initial block. Unlike other generation.c blocks, it
* starts with the context header and its block header follows that.
@@ -361,6 +364,9 @@ GenerationAlloc(MemoryContext context, Size size)
{
Size blksize = required_size + Generation_BLOCKHDRSZ;
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
block = (GenerationBlock *) malloc(blksize);
if (block == NULL)
return NULL;
@@ -464,6 +470,9 @@ GenerationAlloc(MemoryContext context, Size size)
if (blksize < required_size)
blksize = pg_nextpower2_size_t(required_size);
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
block = (GenerationBlock *) malloc(blksize);
if (block == NULL)
diff --git a/src/backend/utils/mmgr/slab.c b/src/backend/utils/mmgr/slab.c
index efc8bcfaa7..5cf0cfff86 100644
--- a/src/backend/utils/mmgr/slab.c
+++ b/src/backend/utils/mmgr/slab.c
@@ -177,6 +177,10 @@ SlabContextCreate(MemoryContext parent,
headerSize += chunksPerBlock * sizeof(bool);
#endif
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(headerSize))
+ return NULL;
+
slab = (SlabContext *) malloc(headerSize);
if (slab == NULL)
{
@@ -331,6 +335,10 @@ SlabAlloc(MemoryContext context, Size size)
*/
if (slab->minFreeChunks == 0)
{
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(slab->blockSize))
+ return NULL;
+
block = (SlabBlock *) malloc(slab->blockSize);
if (block == NULL)
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index 9bdc4197bd..3b940ff98e 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -270,6 +270,7 @@ typedef struct LocalPgBackendStatus
*/
extern PGDLLIMPORT bool pgstat_track_activities;
extern PGDLLIMPORT int pgstat_track_activity_query_size;
+extern PGDLLIMPORT int max_total_bkend_mem;
/* ----------
@@ -321,6 +322,7 @@ extern int pgstat_fetch_stat_numbackends(void);
extern PgBackendStatus *pgstat_fetch_stat_beentry(int beid);
extern LocalPgBackendStatus *pgstat_fetch_stat_local_beentry(int beid);
extern char *pgstat_clip_activity(const char *raw_activity);
+extern bool exceeds_max_total_bkend_mem(uint64 allocation_request);
#endif /* BACKEND_STATUS_H */
[text/x-patch] 001-pg-stat-activity-backend-memory-allocated.patch (26.9K, ../../[email protected]/3-001-pg-stat-activity-backend-memory-allocated.patch)
download | inline diff:
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 1d9509a2f6..40ae638f25 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -947,6 +947,18 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>backend_mem_allocated</structfield> <type>bigint</type>
+ </para>
+ <para>
+ The byte count of memory allocated to this backend. Dynamic shared memory
+ allocations are included only in the value displayed for the backend that
+ created them, they are not included in the value for backends that are
+ attached to them to avoid double counting.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>query</structfield> <type>text</type>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 5a844b63a1..d23f0e9dbb 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -863,6 +863,7 @@ CREATE VIEW pg_stat_activity AS
S.backend_xid,
s.backend_xmin,
S.query_id,
+ S.backend_mem_allocated,
S.query,
S.backend_type
FROM pg_stat_get_activity(NULL) AS S
diff --git a/src/backend/storage/ipc/dsm_impl.c b/src/backend/storage/ipc/dsm_impl.c
index e1b90c5de4..269ad2fe53 100644
--- a/src/backend/storage/ipc/dsm_impl.c
+++ b/src/backend/storage/ipc/dsm_impl.c
@@ -66,6 +66,7 @@
#include "postmaster/postmaster.h"
#include "storage/dsm_impl.h"
#include "storage/fd.h"
+#include "utils/backend_status.h"
#include "utils/guc.h"
#include "utils/memutils.h"
@@ -232,6 +233,13 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pgstat_activity when the creator destroys the
+ * allocation.
+ */
+ if (op == DSM_OP_DESTROY)
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
*mapped_address = NULL;
*mapped_size = 0;
if (op == DSM_OP_DESTROY && shm_unlink(name) != 0)
@@ -332,6 +340,36 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pgstat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ {
+ /*
+ * Posix creation calls dsm_impl_posix_resize implying that resizing
+ * occurs or may be added in the future. As implemented
+ * dsm_impl_posix_resize utilizes fallocate or truncate, passing the
+ * whole new size as input, growing the allocation as needed * (only
+ * truncate supports shrinking). We update by replacing the * old
+ * allocation with the new.
+ */
+#if defined(HAVE_POSIX_FALLOCATE) && defined(__linux__)
+ /*
+ * posix_fallocate does not shrink allocations, adjust only on
+ * allocation increase.
+ */
+ if (request_size > *mapped_size)
+ {
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
+ pgstat_report_backend_mem_allocated_increase(request_size);
+ }
+#else
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
+ pgstat_report_backend_mem_allocated_increase(request_size);
+#endif
+ }
*mapped_address = address;
*mapped_size = request_size;
close(fd);
@@ -537,6 +575,14 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pgstat_activity when the creator destroys the
+ * allocation.
+ */
+ if (op == DSM_OP_DESTROY)
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
*mapped_address = NULL;
*mapped_size = 0;
if (op == DSM_OP_DESTROY && shmctl(ident, IPC_RMID, NULL) < 0)
@@ -584,6 +630,13 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pgstat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ pgstat_report_backend_mem_allocated_increase(request_size);
*mapped_address = address;
*mapped_size = request_size;
@@ -652,6 +705,13 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size,
return false;
}
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pgstat_activity when the creator destroys the
+ * allocation.
+ */
+ if (op == DSM_OP_DESTROY)
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
*impl_private = NULL;
*mapped_address = NULL;
*mapped_size = 0;
@@ -768,6 +828,12 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size,
return false;
}
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pgstat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ pgstat_report_backend_mem_allocated_increase(info.RegionSize);
*mapped_address = address;
*mapped_size = info.RegionSize;
*impl_private = hmap;
@@ -812,6 +878,13 @@ dsm_impl_mmap(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pgstat_activity when the creator destroys the
+ * allocation.
+ */
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
*mapped_address = NULL;
*mapped_size = 0;
if (op == DSM_OP_DESTROY && unlink(name) != 0)
@@ -933,6 +1006,13 @@ dsm_impl_mmap(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pgstat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ pgstat_report_backend_mem_allocated_increase(request_size);
*mapped_address = address;
*mapped_size = request_size;
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index c7ed1e6d7a..17a00587f8 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -49,6 +49,8 @@ int pgstat_track_activity_query_size = 1024;
/* exposed so that backend_progress.c can access it */
PgBackendStatus *MyBEEntry = NULL;
+/* Memory allocated to this backend prior to pgstats initialization */
+uint64 backend_mem_allocated = 0;
static PgBackendStatus *BackendStatusArray = NULL;
static char *BackendAppnameBuffer = NULL;
@@ -400,6 +402,13 @@ pgstat_bestart(void)
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = UINT64CONST(0);
+ /*
+ * Move sum of memory allocated prior to pgstats initialization to pgstats
+ * and zero the local variable.
+ */
+ lbeentry.backend_mem_allocated = backend_mem_allocated;
+ backend_mem_allocated = 0;
+
/*
* we don't zero st_progress_param here to save cycles; nobody should
* examine it until st_progress_command has been set to something other
@@ -1148,3 +1157,99 @@ pgstat_clip_activity(const char *raw_activity)
return activity;
}
+
+/* --------
+ * pgstat_report_backend_mem_allocated_increase() -
+ *
+ * Called to report increase in memory allocated for this backend
+ * --------
+ */
+void
+pgstat_report_backend_mem_allocated_increase(uint64 allocation)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+
+ if (!beentry || !pgstat_track_activities)
+ {
+ /*
+ * Account for memory before pgstats is initialized. This will be
+ * migrated to pgstats on initialization.
+ */
+ backend_mem_allocated += allocation;
+
+ return;
+ }
+
+ /*
+ * Update my status entry, following the protocol of bumping
+ * st_changecount before and after. We use a volatile pointer here to
+ * ensure the compiler doesn't try to get cute.
+ */
+ PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
+ beentry->backend_mem_allocated += allocation;
+ PGSTAT_END_WRITE_ACTIVITY(beentry);
+}
+
+/* --------
+ * pgstat_report_backend_mem_allocated_decrease() -
+ *
+ * Called to report decrease in memory allocated for this backend
+ * --------
+ */
+void
+pgstat_report_backend_mem_allocated_decrease(uint64 deallocation)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+
+ /*
+ * Cases may occur where shared memory from a previous postmaster
+ * invocation still exist. These are cleaned up at startup by
+ * dsm_cleanup_using_control_segment. Limit decreasing memory allocated to
+ * zero in case no corresponding prior increase exists or decrease has
+ * already been accounted for.
+ */
+
+ if (!beentry || !pgstat_track_activities)
+ {
+ /*
+ * Account for memory before pgstats is initialized. This will be
+ * migrated to pgstats on initialization. Do not allow
+ * backend_mem_allocated to go below zero. If pgstats has not been
+ * initialized, we are in startup and we set backend_mem_allocated to
+ * zero in cases where it would go negative and skip generating an
+ * ereport.
+ */
+ if (deallocation > backend_mem_allocated)
+ backend_mem_allocated = 0;
+ else
+ backend_mem_allocated -= deallocation;
+
+ return;
+ }
+
+ /*
+ * Do not allow backend_mem_allocated to go below zero. ereport if we
+ * would have. There's no need for a lock around the read here asit's
+ * being referenced from the same backend which means that there shouldn't
+ * be concurrent writes. We want to generate an ereport in these cases.
+ */
+ if (deallocation > beentry->backend_mem_allocated)
+ {
+ ereport(LOG, (errmsg("decrease reduces reported backend memory allocated below zero; setting reported to 0")));
+
+ /*
+ * Overwrite deallocation with current backend_mem_allocated so we end
+ * up at zero.
+ */
+ deallocation = beentry->backend_mem_allocated;
+ }
+
+ /*
+ * Update my status entry, following the protocol of bumping
+ * st_changecount before and after. We use a volatile pointer here to
+ * ensure the compiler doesn't try to get cute.
+ */
+ PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
+ beentry->backend_mem_allocated -= deallocation;
+ PGSTAT_END_WRITE_ACTIVITY(beentry);
+}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 4cca30aae7..1574aa8049 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -536,7 +536,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum
pg_stat_get_activity(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_ACTIVITY_COLS 30
+#define PG_STAT_GET_ACTIVITY_COLS 31
int num_backends = pgstat_fetch_stat_numbackends();
int curr_backend;
int pid = PG_ARGISNULL(0) ? -1 : PG_GETARG_INT32(0);
@@ -610,6 +610,8 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
else
nulls[16] = true;
+ values[30] = UInt64GetDatum(beentry->backend_mem_allocated);
+
/* Values only available to role member or pg_read_all_stats */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
diff --git a/src/backend/utils/mmgr/aset.c b/src/backend/utils/mmgr/aset.c
index b6eeb8abab..c91f8efa4d 100644
--- a/src/backend/utils/mmgr/aset.c
+++ b/src/backend/utils/mmgr/aset.c
@@ -47,6 +47,7 @@
#include "postgres.h"
#include "port/pg_bitutils.h"
+#include "utils/backend_status.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
#include "utils/memutils_memorychunk.h"
@@ -509,6 +510,7 @@ AllocSetContextCreateInternal(MemoryContext parent,
name);
((MemoryContext) set)->mem_allocated = firstBlockSize;
+ pgstat_report_backend_mem_allocated_increase(firstBlockSize);
return (MemoryContext) set;
}
@@ -532,6 +534,7 @@ AllocSetReset(MemoryContext context)
AllocBlock block;
Size keepersize PG_USED_FOR_ASSERTS_ONLY
= set->keeper->endptr - ((char *) set);
+ uint64 deallocation = 0;
AssertArg(AllocSetIsValid(set));
@@ -571,6 +574,7 @@ AllocSetReset(MemoryContext context)
{
/* Normal case, release the block */
context->mem_allocated -= block->endptr - ((char *) block);
+ deallocation += block->endptr - ((char *) block);
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
@@ -581,6 +585,7 @@ AllocSetReset(MemoryContext context)
}
Assert(context->mem_allocated == keepersize);
+ pgstat_report_backend_mem_allocated_decrease(deallocation);
/* Reset block size allocation sequence, too */
set->nextBlockSize = set->initBlockSize;
@@ -600,6 +605,7 @@ AllocSetDelete(MemoryContext context)
AllocBlock block = set->blocks;
Size keepersize PG_USED_FOR_ASSERTS_ONLY
= set->keeper->endptr - ((char *) set);
+ uint64 deallocation = 0;
AssertArg(AllocSetIsValid(set));
@@ -635,11 +641,13 @@ AllocSetDelete(MemoryContext context)
freelist->first_free = (AllocSetContext *) oldset->header.nextchild;
freelist->num_free--;
+ deallocation += oldset->header.mem_allocated;
/* All that remains is to free the header/initial block */
free(oldset);
}
Assert(freelist->num_free == 0);
+ pgstat_report_backend_mem_allocated_decrease(deallocation);
}
/* Now add the just-deleted context to the freelist. */
@@ -656,7 +664,10 @@ AllocSetDelete(MemoryContext context)
AllocBlock next = block->next;
if (block != set->keeper)
+ {
context->mem_allocated -= block->endptr - ((char *) block);
+ deallocation += block->endptr - ((char *) block);
+ }
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
@@ -669,6 +680,8 @@ AllocSetDelete(MemoryContext context)
}
Assert(context->mem_allocated == keepersize);
+ pgstat_report_backend_mem_allocated_decrease(deallocation +
+ context->mem_allocated);
/* Finally, free the context header, including the keeper block */
free(set);
@@ -712,6 +725,7 @@ AllocSetAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_backend_mem_allocated_increase(blksize);
block->aset = set;
block->freeptr = block->endptr = ((char *) block) + blksize;
@@ -916,6 +930,7 @@ AllocSetAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_backend_mem_allocated_increase(blksize);
block->aset = set;
block->freeptr = ((char *) block) + ALLOC_BLOCKHDRSZ;
@@ -1016,6 +1031,7 @@ AllocSetFree(void *pointer)
block->next->prev = block->prev;
set->header.mem_allocated -= block->endptr - ((char *) block);
+ pgstat_report_backend_mem_allocated_decrease(block->endptr - ((char *) block));
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
@@ -1127,7 +1143,9 @@ AllocSetRealloc(void *pointer, Size size)
/* updated separately, not to underflow when (oldblksize > blksize) */
set->header.mem_allocated -= oldblksize;
+ pgstat_report_backend_mem_allocated_decrease(oldblksize);
set->header.mem_allocated += blksize;
+ pgstat_report_backend_mem_allocated_increase(blksize);
block->freeptr = block->endptr = ((char *) block) + blksize;
diff --git a/src/backend/utils/mmgr/generation.c b/src/backend/utils/mmgr/generation.c
index b39894ec94..36e5b3f94d 100644
--- a/src/backend/utils/mmgr/generation.c
+++ b/src/backend/utils/mmgr/generation.c
@@ -37,6 +37,7 @@
#include "lib/ilist.h"
#include "port/pg_bitutils.h"
+#include "utils/backend_status.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
#include "utils/memutils_memorychunk.h"
@@ -258,6 +259,7 @@ GenerationContextCreate(MemoryContext parent,
name);
((MemoryContext) set)->mem_allocated = firstBlockSize;
+ pgstat_report_backend_mem_allocated_increase(firstBlockSize);
return (MemoryContext) set;
}
@@ -274,6 +276,7 @@ GenerationReset(MemoryContext context)
{
GenerationContext *set = (GenerationContext *) context;
dlist_mutable_iter miter;
+ uint64 deallocation = 0;
AssertArg(GenerationIsValid(set));
@@ -296,9 +299,14 @@ GenerationReset(MemoryContext context)
if (block == set->keeper)
GenerationBlockMarkEmpty(block);
else
+ {
+ deallocation += block->blksize;
GenerationBlockFree(set, block);
+ }
}
+ pgstat_report_backend_mem_allocated_decrease(deallocation);
+
/* set it so new allocations to make use of the keeper block */
set->block = set->keeper;
@@ -319,6 +327,9 @@ GenerationDelete(MemoryContext context)
{
/* Reset to release all releasable GenerationBlocks */
GenerationReset(context);
+
+ pgstat_report_backend_mem_allocated_decrease(context->mem_allocated);
+
/* And free the context header and keeper block */
free(context);
}
@@ -355,6 +366,7 @@ GenerationAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_backend_mem_allocated_increase(blksize);
/* block with a single (used) chunk */
block->context = set;
@@ -458,6 +470,7 @@ GenerationAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_backend_mem_allocated_increase(blksize);
/* initialize the new block */
GenerationBlockInit(set, block, blksize);
@@ -691,6 +704,8 @@ GenerationFree(void *pointer)
dlist_delete(&block->node);
set->header.mem_allocated -= block->blksize;
+ pgstat_report_backend_mem_allocated_decrease(block->blksize);
+
free(block);
}
diff --git a/src/backend/utils/mmgr/slab.c b/src/backend/utils/mmgr/slab.c
index 2d70adef09..efc8bcfaa7 100644
--- a/src/backend/utils/mmgr/slab.c
+++ b/src/backend/utils/mmgr/slab.c
@@ -53,6 +53,7 @@
#include "postgres.h"
#include "lib/ilist.h"
+#include "utils/backend_status.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
#include "utils/memutils_memorychunk.h"
@@ -218,6 +219,12 @@ SlabContextCreate(MemoryContext parent,
parent,
name);
+ /*
+ * If SlabContextCreate is updated to add headerSize to
+ * context->mem_allocated, then update here and SlabDelete appropriately
+ */
+ pgstat_report_backend_mem_allocated_increase(headerSize);
+
return (MemoryContext) slab;
}
@@ -233,6 +240,7 @@ SlabReset(MemoryContext context)
{
int i;
SlabContext *slab = castNode(SlabContext, context);
+ uint64 deallocation = 0;
Assert(slab);
@@ -258,9 +266,11 @@ SlabReset(MemoryContext context)
free(block);
slab->nblocks--;
context->mem_allocated -= slab->blockSize;
+ deallocation += slab->blockSize;
}
}
+ pgstat_report_backend_mem_allocated_decrease(deallocation);
slab->minFreeChunks = 0;
Assert(slab->nblocks == 0);
@@ -274,8 +284,17 @@ SlabReset(MemoryContext context)
void
SlabDelete(MemoryContext context)
{
+ /*
+ * Until header allocation is included in context->mem_allocated cast to
+ * slab and decrement the headerSize
+ */
+ SlabContext *slab = castNode(SlabContext, context);
+
/* Reset to release all the SlabBlocks */
SlabReset(context);
+
+ pgstat_report_backend_mem_allocated_decrease(slab->headerSize);
+
/* And free the context header */
free(context);
}
@@ -344,6 +363,7 @@ SlabAlloc(MemoryContext context, Size size)
slab->minFreeChunks = slab->chunksPerBlock;
slab->nblocks += 1;
context->mem_allocated += slab->blockSize;
+ pgstat_report_backend_mem_allocated_increase(slab->blockSize);
}
/* grab the block from the freelist (even the new block is there) */
@@ -511,6 +531,7 @@ SlabFree(void *pointer)
free(block);
slab->nblocks--;
slab->header.mem_allocated -= slab->blockSize;
+ pgstat_report_backend_mem_allocated_decrease(slab->blockSize);
}
else
dlist_push_head(&slab->freelist[block->nfree], &block->node);
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index be47583122..e1bfb85b25 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5340,9 +5340,9 @@
proname => 'pg_stat_get_activity', prorows => '100', proisstrict => 'f',
proretset => 't', provolatile => 's', proparallel => 'r',
prorettype => 'record', proargtypes => 'int4',
- proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,int4,int8}',
- proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,leader_pid,query_id}',
+ proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,int4,int8,int8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,leader_pid,query_id,backend_mem_allocated}',
prosrc => 'pg_stat_get_activity' },
{ oid => '3318',
descr => 'statistics: information about progress of backends running maintenance command',
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index 7403bca25e..9bdc4197bd 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -168,6 +168,9 @@ typedef struct PgBackendStatus
/* query identifier, optionally computed using post_parse_analyze_hook */
uint64 st_query_id;
+
+ /* Current memory allocated to this backend */
+ uint64 backend_mem_allocated;
} PgBackendStatus;
@@ -305,7 +308,9 @@ extern const char *pgstat_get_backend_current_activity(int pid, bool checkUser);
extern const char *pgstat_get_crashed_backend_activity(int pid, char *buffer,
int buflen);
extern uint64 pgstat_get_my_query_id(void);
-
+extern void pgstat_report_backend_mem_allocated_increase(uint64 allocation);
+extern void pgstat_report_backend_mem_allocated_decrease(uint64 deallocation);
+extern uint64 pgstat_get_all_backend_memory_allocated(void);
/* ----------
* Support functions for the SQL-callable functions to
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 7ec3d2688f..674e5c6fe7 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1757,9 +1757,10 @@ pg_stat_activity| SELECT s.datid,
s.backend_xid,
s.backend_xmin,
s.query_id,
+ s.backend_mem_allocated,
s.query,
s.backend_type
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, backend_mem_allocated)
LEFT JOIN pg_database d ON ((s.datid = d.oid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_all_indexes| SELECT c.oid AS relid,
@@ -1871,7 +1872,7 @@ pg_stat_gssapi| SELECT s.pid,
s.gss_auth AS gss_authenticated,
s.gss_princ AS principal,
s.gss_enc AS encrypted
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, backend_mem_allocated)
WHERE (s.client_port IS NOT NULL);
pg_stat_progress_analyze| SELECT s.pid,
s.datid,
@@ -2052,7 +2053,7 @@ pg_stat_replication| SELECT s.pid,
w.sync_priority,
w.sync_state,
w.reply_time
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, backend_mem_allocated)
JOIN pg_stat_get_wal_senders() w(pid, state, sent_lsn, write_lsn, flush_lsn, replay_lsn, write_lag, flush_lag, replay_lag, sync_priority, sync_state, reply_time) ON ((s.pid = w.pid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_replication_slots| SELECT s.slot_name,
@@ -2086,7 +2087,7 @@ pg_stat_ssl| SELECT s.pid,
s.ssl_client_dn AS client_dn,
s.ssl_client_serial AS client_serial,
s.ssl_issuer_dn AS issuer_dn
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, backend_mem_allocated)
WHERE (s.client_port IS NOT NULL);
pg_stat_subscription| SELECT su.oid AS subid,
su.subname,
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: Add the ability to limit the amount of memory that can be allocated to backends.
2022-08-31 16:50 Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
@ 2022-08-31 17:34 ` Justin Pryzby <[email protected]>
2022-09-04 03:40 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
3 siblings, 1 reply; 22+ messages in thread
From: Justin Pryzby @ 2022-08-31 17:34 UTC (permalink / raw)
To: Reid Thompson <[email protected]>; +Cc: [email protected]
On Wed, Aug 31, 2022 at 12:50:19PM -0400, Reid Thompson wrote:
> Hi Hackers,
>
> Add the ability to limit the amount of memory that can be allocated to
> backends.
>
> This builds on the work that adds backend memory allocated to
> pg_stat_activity
> https://www.postgresql.org/message-id/67bb5c15c0489cb499723b0340f16e10c22485ec.camel%40crunchydata.c...
> Both patches are attached.
You should name the patches with different prefixes, like
001,002,003 Otherwise, cfbot may try to apply them in the wrong order.
git format-patch is the usual tool for that.
> + Specifies a limit to the amount of memory (MB) that may be allocated to
MB are just the default unit, right ?
The user should be allowed to write max_total_backend_memory='2GB'
> + backends in total (i.e. this is not a per user or per backend limit).
> + If unset, or set to 0 it is disabled. A backend request that would push
> + the total over the limit will be denied with an out of memory error
> + causing that backends current query/transaction to fail. Due to the dynamic
backend's
> + nature of memory allocations, this limit is not exact. If within 1.5MB of
> + the limit and two backends request 1MB each at the same time both may be
> + allocated exceeding the limit. Further requests will not be allocated until
allocated, and exceed the limit
> +bool
> +exceeds_max_total_bkend_mem(uint64 allocation_request)
> +{
> + bool result = false;
> +
> + if (MyAuxProcType != NotAnAuxProcess)
> + return result;
The double negative is confusing, so could use a comment.
> + /* Convert max_total_bkend_mem to bytes for comparison */
> + if (max_total_bkend_mem &&
> + pgstat_get_all_backend_memory_allocated() +
> + allocation_request > (uint64)max_total_bkend_mem * 1024 * 1024)
> + {
> + /*
> + * Explicitely identify the OOM being a result of this
> + * configuration parameter vs a system failure to allocate OOM.
> + */
> + elog(WARNING,
> + "request will exceed postgresql.conf defined max_total_backend_memory limit (%lu > %lu)",
> + pgstat_get_all_backend_memory_allocated() +
> + allocation_request, (uint64)max_total_bkend_mem * 1024 * 1024);
I think it should be ereport() rather than elog(), which is
internal-only, and not-translated.
> + {"max_total_backend_memory", PGC_SIGHUP, RESOURCES_MEM,
> + gettext_noop("Restrict total backend memory allocations to this max."),
> + gettext_noop("0 turns this feature off."),
> + GUC_UNIT_MB
> + },
> + &max_total_bkend_mem,
> + 0, 0, INT_MAX,
> + NULL, NULL, NULL
I think this needs a maximum like INT_MAX/1024/1024
> +uint64
> +pgstat_get_all_backend_memory_allocated(void)
> +{
...
> + for (i = 1; i <= NumBackendStatSlots; i++)
> + {
It's looping over every backend for each allocation.
Do you know if there's any performance impact of that ?
I think it may be necessary to track the current allocation size in
shared memory (with atomic increments?). Maybe decrements would need to
be exactly accounted for, or otherwise Assert() that the value is not
negative. I don't know how expensive it'd be to have conditionals for
each decrement, but maybe the value would only be decremented at
strategic times, like at transaction commit or backend shutdown.
--
Justin
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: Add the ability to limit the amount of memory that can be allocated to backends.
2022-08-31 16:50 Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-08-31 17:34 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Justin Pryzby <[email protected]>
@ 2022-09-04 03:40 ` Reid Thompson <[email protected]>
2022-09-09 17:14 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Justin Pryzby <[email protected]>
0 siblings, 1 reply; 22+ messages in thread
From: Reid Thompson @ 2022-09-04 03:40 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]
On Wed, 2022-08-31 at 12:34 -0500, Justin Pryzby wrote:
> You should name the patches with different prefixes, like
> 001,002,003 Otherwise, cfbot may try to apply them in the wrong
> order.
> git format-patch is the usual tool for that.
Thanks for the pointer. My experience with git in the past has been
minimal and basic.
> > + Specifies a limit to the amount of memory (MB) that may be
> > allocated to
>
> MB are just the default unit, right ?
> The user should be allowed to write max_total_backend_memory='2GB'
Correct. Default units are MB. Other unit types are converted to MB.
> > + causing that backends current query/transaction to fail.
>
> backend's
> > + allocated exceeding the limit. Further requests will not
>
> allocated, and exceed the limit
>
> > + if (MyAuxProcType != NotAnAuxProcess)
> The double negative is confusing, so could use a comment.
> > + elog(WARNING,
> I think it should be ereport() rather than elog(), which is
> internal-only, and not-translated.
Corrected/added the the above items. Attached patches with the corrections.
> > + 0, 0, INT_MAX,
> > + NULL, NULL, NULL
> I think this needs a maximum like INT_MAX/1024/1024
Is this noting that we'd set a ceiling of 2048MB?
> > + for (i = 1; i <= NumBackendStatSlots; i++)
> > + {
>
> It's looping over every backend for each allocation.
> Do you know if there's any performance impact of that ?
I'm not very familiar with how to test performance impact, I'm open to
suggestions. I have performed the below pgbench tests and noted the basic
tps differences in the table.
Test 1:
branch master
CFLAGS="-I/usr/include/python3.8/ " /home/rthompso/src/git/postgres/configure --silent --prefix=/home/rthompso/src/git/postgres/install/master --with-openssl --with-tcl --with-tclconfig=/usr/lib/tcl8.6 --with-perl --with-libxml --with-libxslt --with-python --with-gssapi --with-systemd --with-ldap --enable-nls
make -s -j12 && make -s install
initdb
default postgresql.conf settings
init pgbench pgbench -U rthompso -p 5433 -h localhost -i -s 50 testpgbench
10 iterations
for ctr in {1..10}; do { time pgbench -p 5433 -h localhost -c 10 -j 10 -t 50000 testpgbench; } 2>&1 | tee -a pgstatsResultsNoLimitSet; done
Test 2:
branch pg-stat-activity-backend-memory-allocated
CFLAGS="-I/usr/include/python3.8/ " /home/rthompso/src/git/postgres/configure --silent --prefix=/home/rthompso/src/git/postgres/install/pg-stats-memory/ --with-openssl --with-tcl --with-tclconfig=/usr/lib/tcl8.6 --with-perl --with-libxml --with-libxslt --with-python --with-gssapi --with-systemd --with-ldap --enable-nls
make -s -j12 && make -s install
initdb
default postgresql.conf settings
init pgbench pgbench -U rthompso -p 5433 -h localhost -i -s 50
testpgbench
10 iterations
for ctr in {1..10}; do { time pgbench -p 5433 -h localhost -c 10 -j 10 -t 50000 testpgbench; } 2>&1 | tee -a pgstatsResultsPg-stats-memory; done
Test 3:
branch dev-max-memory
CFLAGS="-I/usr/include/python3.8/ " /home/rthompso/src/git/postgres/configure --silent --prefix=/home/rthompso/src/git/postgres/install/dev-max-memory/ --with-openssl --with-tcl --with-tclconfig=/usr/lib/tcl8.6 --with-perl --with-libxml --with-libxslt --with-python --with-gssapi --with-systemd --with-ldap --enable-nls
make -s -j12 && make -s install
initdb
default postgresql.conf settings
init pgbench pgbench -U rthompso -p 5433 -h localhost -i -s 50 testpgbench
10 iterations
for ctr in {1..10}; do { time pgbench -p 5433 -h localhost -c 10 -j 10 -t 50000 testpgbench; } 2>&1 | tee -a pgstatsResultsDev-max-memory; done
Test 4:
branch dev-max-memory
CFLAGS="-I/usr/include/python3.8/ " /home/rthompso/src/git/postgres/configure --silent --prefix=/home/rthompso/src/git/postgres/install/dev-max-memory/ --with-openssl --with-tcl --with-tclconfig=/usr/lib/tcl8.6 --with-perl --with-libxml --with-libxslt --with-python --with-gssapi --with-systemd --with-ldap --enable-nls
make -s -j12 && make -s install
initdb
non-default postgresql.conf setting for max_total_backend_memory = 100MB
init pgbench pgbench -U rthompso -p 5433 -h localhost -i -s 50 testpgbench
10 iterations
for ctr in {1..10}; do { time pgbench -p 5433 -h localhost -c 10 -j 10 -t 50000 testpgbench; } 2>&1 | tee -a pgstatsResultsDev-max-memory100MB; done
Laptop
11th Gen Intel(R) Core(TM) i7-11850H @ 2.50GHz 8 Cores 16 threads
32GB RAM
SSD drive
Averages from the 10 runs and tps difference over the 10 runs
|------------------+------------------+------------------------+-------------------+------------------+-------------------+---------------+------------------|
| Test Run | Master | Track Memory Allocated | Diff from Master | Max Mem off | Diff from Master | Max Mem 100MB | Diff from Master |
| Set 1 | Test 1 | Test 2 | | Test 3 | | Test 4 | |
| latency average | 2.43390909090909 | 2.44327272727273 | | 2.44381818181818 | | 2.6843 | |
| tps inc conn est | 3398.99291372727 | 3385.40984336364 | -13.583070363637 | 3385.08184309091 | -13.9110706363631 | 3729.5363413 | 330.54342757273 |
| tps exc conn est | 3399.12185727273 | 3385.52527490909 | -13.5965823636366 | 3385.22100872727 | -13.9008485454547 | 3729.7097607 | 330.58790342727 |
|------------------+------------------+------------------------+-------------------+------------------+-------------------+---------------+------------------|
| Set 2 | | | | | | | |
| latency average | 2.691 | 2.6895 | 2 | 2.69 | 3 | 2.6827 | 4 |
| tps inc conn est | 3719.56 | 3721.7587106 | 2.1987106 | 3720.3 | .74 | 3730.86 | 11.30 |
| tps exc conn est | 3719.71 | 3721.9268465 | 2.2168465 | 3720.47 | .76 | 3731.02 | 11.31 |
|------------------+------------------+------------------------+-------------------+------------------+-------------------+---------------+------------------|
> I think it may be necessary to track the current allocation size in
> shared memory (with atomic increments?). Maybe decrements would need
> to
> be exactly accounted for, or otherwise Assert() that the value is not
> negative. I don't know how expensive it'd be to have conditionals
> for
> each decrement, but maybe the value would only be decremented at
> strategic times, like at transaction commit or backend shutdown.
>
--
Reid Thompson
Senior Software Engineer
Crunchy Data, Inc.
[email protected]
www.crunchydata.com
Attachments:
[text/x-patch] 0002-Add-the-ability-to-limit-the-amount-of-memory-that-c.patch (13.7K, ../../[email protected]/2-0002-Add-the-ability-to-limit-the-amount-of-memory-that-c.patch)
download | inline diff:
From 57a79b5f72af510f8c4b9ea65f5ffb4fe1fb7798 Mon Sep 17 00:00:00 2001
From: Reid Thompson <[email protected]>
Date: Sat, 4 Jun 2022 22:23:59 -0400
Subject: [PATCH 2/2] Add the ability to limit the amount of memory that can be
allocated to backends.
This builds on the work that adds backend memory allocated to pg_stat_activity.
Add GUC variable max_total_backend_memory.
Specifies a limit to the amount of memory (MB) that may be allocated to backends
in total (i.e. this is not a per user or per backend limit). If unset, or set to
0 it is disabled. It is intended as a resource to help avoid the OOM killer on
LINUX and manage resources in general. A backend request that would push the
total over the limit will be denied with an out of memory error causing that
backend's current query/transaction to fail. Due to the dynamic nature of memory
allocations, this limit is not exact. If within 1.5MB of the limit and two
backends request 1MB each at the same time both may be allocated, and exceed the
limit. Further requests will not be allocated until dropping below the limit.
Keep this in mind when setting this value. This limit does not affect auxiliary
backend processes. Backend memory allocations are displayed in the
pg_stat_activity view.
---
doc/src/sgml/config.sgml | 26 +++++
src/backend/storage/ipc/dsm_impl.c | 12 ++
src/backend/utils/activity/backend_status.c | 107 ++++++++++++++++++
src/backend/utils/misc/guc.c | 11 ++
src/backend/utils/misc/postgresql.conf.sample | 3 +
src/backend/utils/mmgr/aset.c | 17 +++
src/backend/utils/mmgr/generation.c | 9 ++
src/backend/utils/mmgr/slab.c | 8 ++
src/include/utils/backend_status.h | 2 +
9 files changed, 195 insertions(+)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index a5cd4e44c7..e70ea71ba1 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -2079,6 +2079,32 @@ include_dir 'conf.d'
</listitem>
</varlistentry>
+ <varlistentry id="guc-max-total-backend-memory" xreflabel="max_total_backend_memory">
+ <term><varname>max_total_backend_memory</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>max_total_backend_memory</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies a limit to the amount of memory (MB) that may be allocated to
+ backends in total (i.e. this is not a per user or per backend limit).
+ If unset, or set to 0 it is disabled. A backend request that would
+ push the total over the limit will be denied with an out of memory
+ error causing that backend's current query/transaction to fail. Due to
+ the dynamic nature of memory allocations, this limit is not exact. If
+ within 1.5MB of the limit and two backends request 1MB each at the same
+ time both may be allocated, and exceed the limit. Further requests will
+ not be allocated until dropping below the limit. Keep this in mind when
+ setting this value. This limit does not affect auxiliary backend
+ processes <xref linkend="glossary-auxiliary-proc"/> . Backend memory
+ allocations (<varname>backend_mem_allocated</varname>) are displayed in
+ the <link linkend="monitoring-pg-stat-activity-view"><structname>pg_stat_activity</structname></link>
+ view.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/src/backend/storage/ipc/dsm_impl.c b/src/backend/storage/ipc/dsm_impl.c
index 3356bb65b5..cc061056a3 100644
--- a/src/backend/storage/ipc/dsm_impl.c
+++ b/src/backend/storage/ipc/dsm_impl.c
@@ -253,6 +253,10 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
return true;
}
+ /* Do not exceed maximum allowed memory allocation */
+ if (op == DSM_OP_CREATE && exceeds_max_total_bkend_mem(request_size))
+ return false;
+
/*
* Create new segment or open an existing one for attach.
*
@@ -524,6 +528,10 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size,
int flags = IPCProtection;
size_t segsize;
+ /* Do not exceed maximum allowed memory allocation */
+ if (op == DSM_OP_CREATE && exceeds_max_total_bkend_mem(request_size))
+ return false;
+
/*
* Allocate the memory BEFORE acquiring the resource, so that we don't
* leak the resource if memory allocation fails.
@@ -718,6 +726,10 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size,
return true;
}
+ /* Do not exceed maximum allowed memory allocation */
+ if (op == DSM_OP_CREATE && exceeds_max_total_bkend_mem(request_size))
+ return false;
+
/* Create new segment or open an existing one for attach. */
if (op == DSM_OP_CREATE)
{
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 45da3af213..8ef9b0ffd5 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -44,6 +44,8 @@
*/
bool pgstat_track_activities = false;
int pgstat_track_activity_query_size = 1024;
+/* Max backend memory allocation allowed (MB). 0 = disabled */
+int max_total_bkend_mem = 0;
/* exposed so that backend_progress.c can access it */
@@ -1253,3 +1255,108 @@ pgstat_report_backend_mem_allocated_decrease(uint64 deallocation)
beentry->backend_mem_allocated -= deallocation;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
+
+/* ----------
+ * pgstat_get_all_backend_memory_allocated() -
+ *
+ * Return a uint64 representing the current shared memory allocated to all
+ * backends. This looks directly at the BackendStatusArray, and so will
+ * provide current information regardless of the age of our transaction's
+ * snapshot of the status array.
+ * In the future we will likely utilize additional values - perhaps limit
+ * backend allocation by user/role, etc.
+ * ----------
+ */
+uint64
+pgstat_get_all_backend_memory_allocated(void)
+{
+ PgBackendStatus *beentry;
+ int i;
+ uint64 all_backend_memory_allocated = 0;
+
+ beentry = BackendStatusArray;
+
+ /*
+ * We probably shouldn't get here before shared memory has been set up,
+ * but be safe.
+ */
+ if (beentry == NULL || BackendActivityBuffer == NULL)
+ return 0;
+
+ /*
+ * We include AUX procs in all backend memory calculation
+ */
+ for (i = 1; i <= NumBackendStatSlots; i++)
+ {
+ /*
+ * We use a volatile pointer here to ensure the compiler doesn't try to
+ * get cute.
+ */
+ volatile PgBackendStatus *vbeentry = beentry;
+ bool found;
+ uint64 backend_mem_allocated = 0;
+
+ for (;;)
+ {
+ int before_changecount;
+ int after_changecount;
+
+ pgstat_begin_read_activity(vbeentry, before_changecount);
+
+ /* Ignore invalid entries, which may contain invalid data.
+ * See pgstat_beshutdown_hook()
+ */
+ if (vbeentry->st_procpid > 0)
+ backend_mem_allocated = vbeentry->backend_mem_allocated;
+
+ pgstat_end_read_activity(vbeentry, after_changecount);
+
+ if ((found = pgstat_read_activity_complete(before_changecount,
+ after_changecount)))
+ break;
+
+ /* Make sure we can break out of loop if stuck... */
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ if (found)
+ all_backend_memory_allocated += backend_mem_allocated;
+
+ beentry++;
+ }
+
+ return all_backend_memory_allocated;
+}
+
+/*
+ * Determine if allocation request will exceed max backend memory allowed.
+ * Do not apply to auxiliary processes.
+ */
+bool
+exceeds_max_total_bkend_mem(uint64 allocation_request)
+{
+ bool result = false;
+
+ /* Exclude auxiliary processes from the check */
+ if (MyAuxProcType != NotAnAuxProcess)
+ return result;
+
+ /* Convert max_total_bkend_mem to bytes for comparison */
+ if (max_total_bkend_mem &&
+ pgstat_get_all_backend_memory_allocated() +
+ allocation_request > (uint64)max_total_bkend_mem * 1024 * 1024)
+ {
+ /*
+ * Explicitely identify the OOM being a result of this
+ * configuration parameter vs a system failure to allocate OOM.
+ */
+ ereport(WARNING,
+ errmsg("request will exceed postgresql.conf defined max_total_backend_memory limit (%lu > %lu)",
+ pgstat_get_all_backend_memory_allocated() +
+ allocation_request, (uint64)max_total_bkend_mem * 1024 * 1024));
+
+ result = true;
+ }
+
+ return result;
+}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 9fbbfb1be5..ab8d83c235 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -3664,6 +3664,17 @@ static struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_total_backend_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Restrict total backend memory allocations to this max."),
+ gettext_noop("0 turns this feature off."),
+ GUC_UNIT_MB
+ },
+ &max_total_bkend_mem,
+ 0, 0, INT_MAX,
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, 0, 0, 0, NULL, NULL, NULL
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 90bec0502c..8e944f6511 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -155,6 +155,9 @@
# mmap
# (change requires restart)
#min_dynamic_shared_memory = 0MB # (change requires restart)
+#max_total_backend_memory = 0MB # Restrict total backend memory allocations
+ # to this max (in MB). 0 turns this feature
+ # off.
# - Disk -
diff --git a/src/backend/utils/mmgr/aset.c b/src/backend/utils/mmgr/aset.c
index c91f8efa4d..ac9a1ced3f 100644
--- a/src/backend/utils/mmgr/aset.c
+++ b/src/backend/utils/mmgr/aset.c
@@ -428,6 +428,10 @@ AllocSetContextCreateInternal(MemoryContext parent,
else
firstBlockSize = Max(firstBlockSize, initBlockSize);
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(firstBlockSize))
+ return NULL;
+
/*
* Allocate the initial block. Unlike other aset.c blocks, it starts with
* the context header and its block header follows that.
@@ -720,6 +724,11 @@ AllocSetAlloc(MemoryContext context, Size size)
{
chunk_size = MAXALIGN(size);
blksize = chunk_size + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ;
+
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
block = (AllocBlock) malloc(blksize);
if (block == NULL)
return NULL;
@@ -911,6 +920,10 @@ AllocSetAlloc(MemoryContext context, Size size)
while (blksize < required_size)
blksize <<= 1;
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
/* Try to allocate it */
block = (AllocBlock) malloc(blksize);
@@ -1133,6 +1146,10 @@ AllocSetRealloc(void *pointer, Size size)
blksize = chksize + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ;
oldblksize = block->endptr - ((char *) block);
+ /* Do not exceed maximum allowed memory allocation */
+ if (blksize > oldblksize && exceeds_max_total_bkend_mem(blksize - oldblksize))
+ return NULL;
+
block = (AllocBlock) realloc(block, blksize);
if (block == NULL)
{
diff --git a/src/backend/utils/mmgr/generation.c b/src/backend/utils/mmgr/generation.c
index 36e5b3f94d..1d5720836c 100644
--- a/src/backend/utils/mmgr/generation.c
+++ b/src/backend/utils/mmgr/generation.c
@@ -192,6 +192,9 @@ GenerationContextCreate(MemoryContext parent,
else
allocSize = Max(allocSize, initBlockSize);
+ if (exceeds_max_total_bkend_mem(allocSize))
+ return NULL;
+
/*
* Allocate the initial block. Unlike other generation.c blocks, it
* starts with the context header and its block header follows that.
@@ -361,6 +364,9 @@ GenerationAlloc(MemoryContext context, Size size)
{
Size blksize = required_size + Generation_BLOCKHDRSZ;
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
block = (GenerationBlock *) malloc(blksize);
if (block == NULL)
return NULL;
@@ -464,6 +470,9 @@ GenerationAlloc(MemoryContext context, Size size)
if (blksize < required_size)
blksize = pg_nextpower2_size_t(required_size);
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
block = (GenerationBlock *) malloc(blksize);
if (block == NULL)
diff --git a/src/backend/utils/mmgr/slab.c b/src/backend/utils/mmgr/slab.c
index e0e69b394e..63c07120dd 100644
--- a/src/backend/utils/mmgr/slab.c
+++ b/src/backend/utils/mmgr/slab.c
@@ -177,6 +177,10 @@ SlabContextCreate(MemoryContext parent,
headerSize += chunksPerBlock * sizeof(bool);
#endif
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(headerSize))
+ return NULL;
+
slab = (SlabContext *) malloc(headerSize);
if (slab == NULL)
{
@@ -331,6 +335,10 @@ SlabAlloc(MemoryContext context, Size size)
*/
if (slab->minFreeChunks == 0)
{
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(slab->blockSize))
+ return NULL;
+
block = (SlabBlock *) malloc(slab->blockSize);
if (block == NULL)
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index 9bdc4197bd..3b940ff98e 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -270,6 +270,7 @@ typedef struct LocalPgBackendStatus
*/
extern PGDLLIMPORT bool pgstat_track_activities;
extern PGDLLIMPORT int pgstat_track_activity_query_size;
+extern PGDLLIMPORT int max_total_bkend_mem;
/* ----------
@@ -321,6 +322,7 @@ extern int pgstat_fetch_stat_numbackends(void);
extern PgBackendStatus *pgstat_fetch_stat_beentry(int beid);
extern LocalPgBackendStatus *pgstat_fetch_stat_local_beentry(int beid);
extern char *pgstat_clip_activity(const char *raw_activity);
+extern bool exceeds_max_total_bkend_mem(uint64 allocation_request);
#endif /* BACKEND_STATUS_H */
--
2.25.1
[text/x-patch] 0001-Add-tracking-of-backend-memory-allocated-to-pg_stat_.patch (28.5K, ../../[email protected]/3-0001-Add-tracking-of-backend-memory-allocated-to-pg_stat_.patch)
download | inline diff:
From 584a04f1b53948049e73165a4ffdd544c950ab0d Mon Sep 17 00:00:00 2001
From: Reid Thompson <[email protected]>
Date: Thu, 11 Aug 2022 12:01:25 -0400
Subject: [PATCH 1/2] Add tracking of backend memory allocated to
pg_stat_activity
This new field displays the current bytes of memory allocated to the
backend process. It is updated as memory for the process is
malloc'd/free'd. Memory allocated to items on the freelist is included in
the displayed value. Dynamic shared memory allocations are included
only in the value displayed for the backend that created them, they are
not included in the value for backends that are attached to them to
avoid double counting. On occasion, orphaned memory segments may be
cleaned up on postmaster startup. This may result in decreasing the sum
without a prior increment. We limit the floor of backend_mem_allocated
to zero. Updated pg_stat_activity documentation for the new column.
---
doc/src/sgml/monitoring.sgml | 12 +++
src/backend/catalog/system_views.sql | 1 +
src/backend/storage/ipc/dsm_impl.c | 80 +++++++++++++++
src/backend/utils/activity/backend_status.c | 105 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 4 +-
src/backend/utils/mmgr/aset.c | 18 ++++
src/backend/utils/mmgr/generation.c | 15 +++
src/backend/utils/mmgr/slab.c | 21 ++++
src/include/catalog/pg_proc.dat | 6 +-
src/include/utils/backend_status.h | 7 +-
src/test/regress/expected/rules.out | 9 +-
11 files changed, 269 insertions(+), 9 deletions(-)
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 1d9509a2f6..40ae638f25 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -947,6 +947,18 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>backend_mem_allocated</structfield> <type>bigint</type>
+ </para>
+ <para>
+ The byte count of memory allocated to this backend. Dynamic shared memory
+ allocations are included only in the value displayed for the backend that
+ created them, they are not included in the value for backends that are
+ attached to them to avoid double counting.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>query</structfield> <type>text</type>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 5a844b63a1..d23f0e9dbb 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -863,6 +863,7 @@ CREATE VIEW pg_stat_activity AS
S.backend_xid,
s.backend_xmin,
S.query_id,
+ S.backend_mem_allocated,
S.query,
S.backend_type
FROM pg_stat_get_activity(NULL) AS S
diff --git a/src/backend/storage/ipc/dsm_impl.c b/src/backend/storage/ipc/dsm_impl.c
index e1b90c5de4..3356bb65b5 100644
--- a/src/backend/storage/ipc/dsm_impl.c
+++ b/src/backend/storage/ipc/dsm_impl.c
@@ -66,6 +66,7 @@
#include "postmaster/postmaster.h"
#include "storage/dsm_impl.h"
#include "storage/fd.h"
+#include "utils/backend_status.h"
#include "utils/guc.h"
#include "utils/memutils.h"
@@ -232,6 +233,13 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ if (op == DSM_OP_DESTROY)
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
*mapped_address = NULL;
*mapped_size = 0;
if (op == DSM_OP_DESTROY && shm_unlink(name) != 0)
@@ -332,6 +340,36 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ {
+ /*
+ * Posix creation calls dsm_impl_posix_resize implying that resizing
+ * occurs or may be added in the future. As implemented
+ * dsm_impl_posix_resize utilizes fallocate or truncate, passing the
+ * whole new size as input, growing the allocation as needed (only
+ * truncate supports shrinking). We update by replacing the old
+ * allocation with the new.
+ */
+#if defined(HAVE_POSIX_FALLOCATE) && defined(__linux__)
+ /*
+ * posix_fallocate does not shrink allocations, adjust only on
+ * allocation increase.
+ */
+ if (request_size > *mapped_size)
+ {
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
+ pgstat_report_backend_mem_allocated_increase(request_size);
+ }
+#else
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
+ pgstat_report_backend_mem_allocated_increase(request_size);
+#endif
+ }
*mapped_address = address;
*mapped_size = request_size;
close(fd);
@@ -537,6 +575,14 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ if (op == DSM_OP_DESTROY)
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
*mapped_address = NULL;
*mapped_size = 0;
if (op == DSM_OP_DESTROY && shmctl(ident, IPC_RMID, NULL) < 0)
@@ -584,6 +630,13 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ pgstat_report_backend_mem_allocated_increase(request_size);
*mapped_address = address;
*mapped_size = request_size;
@@ -652,6 +705,13 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size,
return false;
}
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ if (op == DSM_OP_DESTROY)
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
*impl_private = NULL;
*mapped_address = NULL;
*mapped_size = 0;
@@ -768,6 +828,12 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size,
return false;
}
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ pgstat_report_backend_mem_allocated_increase(info.RegionSize);
*mapped_address = address;
*mapped_size = info.RegionSize;
*impl_private = hmap;
@@ -812,6 +878,13 @@ dsm_impl_mmap(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
*mapped_address = NULL;
*mapped_size = 0;
if (op == DSM_OP_DESTROY && unlink(name) != 0)
@@ -933,6 +1006,13 @@ dsm_impl_mmap(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ pgstat_report_backend_mem_allocated_increase(request_size);
*mapped_address = address;
*mapped_size = request_size;
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index c7ed1e6d7a..45da3af213 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -49,6 +49,8 @@ int pgstat_track_activity_query_size = 1024;
/* exposed so that backend_progress.c can access it */
PgBackendStatus *MyBEEntry = NULL;
+/* Memory allocated to this backend prior to pgstats initialization */
+uint64 backend_mem_allocated = 0;
static PgBackendStatus *BackendStatusArray = NULL;
static char *BackendAppnameBuffer = NULL;
@@ -400,6 +402,13 @@ pgstat_bestart(void)
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = UINT64CONST(0);
+ /*
+ * Move sum of memory allocated prior to pgstats initialization to pgstats
+ * and zero the local variable.
+ */
+ lbeentry.backend_mem_allocated = backend_mem_allocated;
+ backend_mem_allocated = 0;
+
/*
* we don't zero st_progress_param here to save cycles; nobody should
* examine it until st_progress_command has been set to something other
@@ -1148,3 +1157,99 @@ pgstat_clip_activity(const char *raw_activity)
return activity;
}
+
+/* --------
+ * pgstat_report_backend_mem_allocated_increase() -
+ *
+ * Called to report increase in memory allocated for this backend
+ * --------
+ */
+void
+pgstat_report_backend_mem_allocated_increase(uint64 allocation)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+
+ if (!beentry || !pgstat_track_activities)
+ {
+ /*
+ * Account for memory before pgstats is initialized. This will be
+ * migrated to pgstats on initialization.
+ */
+ backend_mem_allocated += allocation;
+
+ return;
+ }
+
+ /*
+ * Update my status entry, following the protocol of bumping
+ * st_changecount before and after. We use a volatile pointer here to
+ * ensure the compiler doesn't try to get cute.
+ */
+ PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
+ beentry->backend_mem_allocated += allocation;
+ PGSTAT_END_WRITE_ACTIVITY(beentry);
+}
+
+/* --------
+ * pgstat_report_backend_mem_allocated_decrease() -
+ *
+ * Called to report decrease in memory allocated for this backend
+ * --------
+ */
+void
+pgstat_report_backend_mem_allocated_decrease(uint64 deallocation)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+
+ /*
+ * Cases may occur where shared memory from a previous postmaster
+ * invocation still exist. These are cleaned up at startup by
+ * dsm_cleanup_using_control_segment. Limit decreasing memory allocated to
+ * zero in case no corresponding prior increase exists or decrease has
+ * already been accounted for.
+ */
+
+ if (!beentry || !pgstat_track_activities)
+ {
+ /*
+ * Account for memory before pgstats is initialized. This will be
+ * migrated to pgstats on initialization. Do not allow
+ * backend_mem_allocated to go below zero. If pgstats has not been
+ * initialized, we are in startup and we set backend_mem_allocated to
+ * zero in cases where it would go negative and skip generating an
+ * ereport.
+ */
+ if (deallocation > backend_mem_allocated)
+ backend_mem_allocated = 0;
+ else
+ backend_mem_allocated -= deallocation;
+
+ return;
+ }
+
+ /*
+ * Do not allow backend_mem_allocated to go below zero. ereport if we
+ * would have. There's no need for a lock around the read here as it's
+ * being referenced from the same backend which means that there shouldn't
+ * be concurrent writes. We want to generate an ereport in these cases.
+ */
+ if (deallocation > beentry->backend_mem_allocated)
+ {
+ ereport(LOG, errmsg("decrease reduces reported backend memory allocated below zero; setting reported to 0"));
+
+ /*
+ * Overwrite deallocation with current backend_mem_allocated so we end
+ * up at zero.
+ */
+ deallocation = beentry->backend_mem_allocated;
+ }
+
+ /*
+ * Update my status entry, following the protocol of bumping
+ * st_changecount before and after. We use a volatile pointer here to
+ * ensure the compiler doesn't try to get cute.
+ */
+ PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
+ beentry->backend_mem_allocated -= deallocation;
+ PGSTAT_END_WRITE_ACTIVITY(beentry);
+}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 4cca30aae7..1574aa8049 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -536,7 +536,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum
pg_stat_get_activity(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_ACTIVITY_COLS 30
+#define PG_STAT_GET_ACTIVITY_COLS 31
int num_backends = pgstat_fetch_stat_numbackends();
int curr_backend;
int pid = PG_ARGISNULL(0) ? -1 : PG_GETARG_INT32(0);
@@ -610,6 +610,8 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
else
nulls[16] = true;
+ values[30] = UInt64GetDatum(beentry->backend_mem_allocated);
+
/* Values only available to role member or pg_read_all_stats */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
diff --git a/src/backend/utils/mmgr/aset.c b/src/backend/utils/mmgr/aset.c
index b6eeb8abab..c91f8efa4d 100644
--- a/src/backend/utils/mmgr/aset.c
+++ b/src/backend/utils/mmgr/aset.c
@@ -47,6 +47,7 @@
#include "postgres.h"
#include "port/pg_bitutils.h"
+#include "utils/backend_status.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
#include "utils/memutils_memorychunk.h"
@@ -509,6 +510,7 @@ AllocSetContextCreateInternal(MemoryContext parent,
name);
((MemoryContext) set)->mem_allocated = firstBlockSize;
+ pgstat_report_backend_mem_allocated_increase(firstBlockSize);
return (MemoryContext) set;
}
@@ -532,6 +534,7 @@ AllocSetReset(MemoryContext context)
AllocBlock block;
Size keepersize PG_USED_FOR_ASSERTS_ONLY
= set->keeper->endptr - ((char *) set);
+ uint64 deallocation = 0;
AssertArg(AllocSetIsValid(set));
@@ -571,6 +574,7 @@ AllocSetReset(MemoryContext context)
{
/* Normal case, release the block */
context->mem_allocated -= block->endptr - ((char *) block);
+ deallocation += block->endptr - ((char *) block);
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
@@ -581,6 +585,7 @@ AllocSetReset(MemoryContext context)
}
Assert(context->mem_allocated == keepersize);
+ pgstat_report_backend_mem_allocated_decrease(deallocation);
/* Reset block size allocation sequence, too */
set->nextBlockSize = set->initBlockSize;
@@ -600,6 +605,7 @@ AllocSetDelete(MemoryContext context)
AllocBlock block = set->blocks;
Size keepersize PG_USED_FOR_ASSERTS_ONLY
= set->keeper->endptr - ((char *) set);
+ uint64 deallocation = 0;
AssertArg(AllocSetIsValid(set));
@@ -635,11 +641,13 @@ AllocSetDelete(MemoryContext context)
freelist->first_free = (AllocSetContext *) oldset->header.nextchild;
freelist->num_free--;
+ deallocation += oldset->header.mem_allocated;
/* All that remains is to free the header/initial block */
free(oldset);
}
Assert(freelist->num_free == 0);
+ pgstat_report_backend_mem_allocated_decrease(deallocation);
}
/* Now add the just-deleted context to the freelist. */
@@ -656,7 +664,10 @@ AllocSetDelete(MemoryContext context)
AllocBlock next = block->next;
if (block != set->keeper)
+ {
context->mem_allocated -= block->endptr - ((char *) block);
+ deallocation += block->endptr - ((char *) block);
+ }
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
@@ -669,6 +680,8 @@ AllocSetDelete(MemoryContext context)
}
Assert(context->mem_allocated == keepersize);
+ pgstat_report_backend_mem_allocated_decrease(deallocation +
+ context->mem_allocated);
/* Finally, free the context header, including the keeper block */
free(set);
@@ -712,6 +725,7 @@ AllocSetAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_backend_mem_allocated_increase(blksize);
block->aset = set;
block->freeptr = block->endptr = ((char *) block) + blksize;
@@ -916,6 +930,7 @@ AllocSetAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_backend_mem_allocated_increase(blksize);
block->aset = set;
block->freeptr = ((char *) block) + ALLOC_BLOCKHDRSZ;
@@ -1016,6 +1031,7 @@ AllocSetFree(void *pointer)
block->next->prev = block->prev;
set->header.mem_allocated -= block->endptr - ((char *) block);
+ pgstat_report_backend_mem_allocated_decrease(block->endptr - ((char *) block));
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
@@ -1127,7 +1143,9 @@ AllocSetRealloc(void *pointer, Size size)
/* updated separately, not to underflow when (oldblksize > blksize) */
set->header.mem_allocated -= oldblksize;
+ pgstat_report_backend_mem_allocated_decrease(oldblksize);
set->header.mem_allocated += blksize;
+ pgstat_report_backend_mem_allocated_increase(blksize);
block->freeptr = block->endptr = ((char *) block) + blksize;
diff --git a/src/backend/utils/mmgr/generation.c b/src/backend/utils/mmgr/generation.c
index b39894ec94..36e5b3f94d 100644
--- a/src/backend/utils/mmgr/generation.c
+++ b/src/backend/utils/mmgr/generation.c
@@ -37,6 +37,7 @@
#include "lib/ilist.h"
#include "port/pg_bitutils.h"
+#include "utils/backend_status.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
#include "utils/memutils_memorychunk.h"
@@ -258,6 +259,7 @@ GenerationContextCreate(MemoryContext parent,
name);
((MemoryContext) set)->mem_allocated = firstBlockSize;
+ pgstat_report_backend_mem_allocated_increase(firstBlockSize);
return (MemoryContext) set;
}
@@ -274,6 +276,7 @@ GenerationReset(MemoryContext context)
{
GenerationContext *set = (GenerationContext *) context;
dlist_mutable_iter miter;
+ uint64 deallocation = 0;
AssertArg(GenerationIsValid(set));
@@ -296,9 +299,14 @@ GenerationReset(MemoryContext context)
if (block == set->keeper)
GenerationBlockMarkEmpty(block);
else
+ {
+ deallocation += block->blksize;
GenerationBlockFree(set, block);
+ }
}
+ pgstat_report_backend_mem_allocated_decrease(deallocation);
+
/* set it so new allocations to make use of the keeper block */
set->block = set->keeper;
@@ -319,6 +327,9 @@ GenerationDelete(MemoryContext context)
{
/* Reset to release all releasable GenerationBlocks */
GenerationReset(context);
+
+ pgstat_report_backend_mem_allocated_decrease(context->mem_allocated);
+
/* And free the context header and keeper block */
free(context);
}
@@ -355,6 +366,7 @@ GenerationAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_backend_mem_allocated_increase(blksize);
/* block with a single (used) chunk */
block->context = set;
@@ -458,6 +470,7 @@ GenerationAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_backend_mem_allocated_increase(blksize);
/* initialize the new block */
GenerationBlockInit(set, block, blksize);
@@ -691,6 +704,8 @@ GenerationFree(void *pointer)
dlist_delete(&block->node);
set->header.mem_allocated -= block->blksize;
+ pgstat_report_backend_mem_allocated_decrease(block->blksize);
+
free(block);
}
diff --git a/src/backend/utils/mmgr/slab.c b/src/backend/utils/mmgr/slab.c
index 2d70adef09..e0e69b394e 100644
--- a/src/backend/utils/mmgr/slab.c
+++ b/src/backend/utils/mmgr/slab.c
@@ -53,6 +53,7 @@
#include "postgres.h"
#include "lib/ilist.h"
+#include "utils/backend_status.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
#include "utils/memutils_memorychunk.h"
@@ -218,6 +219,12 @@ SlabContextCreate(MemoryContext parent,
parent,
name);
+ /*
+ * If SlabContextCreate is updated to add headerSize to
+ * context->mem_allocated, then update here and SlabDelete appropriately
+ */
+ pgstat_report_backend_mem_allocated_increase(headerSize);
+
return (MemoryContext) slab;
}
@@ -233,6 +240,7 @@ SlabReset(MemoryContext context)
{
int i;
SlabContext *slab = castNode(SlabContext, context);
+ uint64 deallocation = 0;
Assert(slab);
@@ -258,9 +266,11 @@ SlabReset(MemoryContext context)
free(block);
slab->nblocks--;
context->mem_allocated -= slab->blockSize;
+ deallocation += slab->blockSize;
}
}
+ pgstat_report_backend_mem_allocated_decrease(deallocation);
slab->minFreeChunks = 0;
Assert(slab->nblocks == 0);
@@ -274,8 +284,17 @@ SlabReset(MemoryContext context)
void
SlabDelete(MemoryContext context)
{
+ /*
+ * Until header allocation is included in context->mem_allocated, cast to
+ * slab and decrement the headerSize
+ */
+ SlabContext *slab = castNode(SlabContext, context);
+
/* Reset to release all the SlabBlocks */
SlabReset(context);
+
+ pgstat_report_backend_mem_allocated_decrease(slab->headerSize);
+
/* And free the context header */
free(context);
}
@@ -344,6 +363,7 @@ SlabAlloc(MemoryContext context, Size size)
slab->minFreeChunks = slab->chunksPerBlock;
slab->nblocks += 1;
context->mem_allocated += slab->blockSize;
+ pgstat_report_backend_mem_allocated_increase(slab->blockSize);
}
/* grab the block from the freelist (even the new block is there) */
@@ -511,6 +531,7 @@ SlabFree(void *pointer)
free(block);
slab->nblocks--;
slab->header.mem_allocated -= slab->blockSize;
+ pgstat_report_backend_mem_allocated_decrease(slab->blockSize);
}
else
dlist_push_head(&slab->freelist[block->nfree], &block->node);
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index be47583122..e1bfb85b25 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5340,9 +5340,9 @@
proname => 'pg_stat_get_activity', prorows => '100', proisstrict => 'f',
proretset => 't', provolatile => 's', proparallel => 'r',
prorettype => 'record', proargtypes => 'int4',
- proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,int4,int8}',
- proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,leader_pid,query_id}',
+ proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,int4,int8,int8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,leader_pid,query_id,backend_mem_allocated}',
prosrc => 'pg_stat_get_activity' },
{ oid => '3318',
descr => 'statistics: information about progress of backends running maintenance command',
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index 7403bca25e..9bdc4197bd 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -168,6 +168,9 @@ typedef struct PgBackendStatus
/* query identifier, optionally computed using post_parse_analyze_hook */
uint64 st_query_id;
+
+ /* Current memory allocated to this backend */
+ uint64 backend_mem_allocated;
} PgBackendStatus;
@@ -305,7 +308,9 @@ extern const char *pgstat_get_backend_current_activity(int pid, bool checkUser);
extern const char *pgstat_get_crashed_backend_activity(int pid, char *buffer,
int buflen);
extern uint64 pgstat_get_my_query_id(void);
-
+extern void pgstat_report_backend_mem_allocated_increase(uint64 allocation);
+extern void pgstat_report_backend_mem_allocated_decrease(uint64 deallocation);
+extern uint64 pgstat_get_all_backend_memory_allocated(void);
/* ----------
* Support functions for the SQL-callable functions to
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 7ec3d2688f..674e5c6fe7 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1757,9 +1757,10 @@ pg_stat_activity| SELECT s.datid,
s.backend_xid,
s.backend_xmin,
s.query_id,
+ s.backend_mem_allocated,
s.query,
s.backend_type
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, backend_mem_allocated)
LEFT JOIN pg_database d ON ((s.datid = d.oid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_all_indexes| SELECT c.oid AS relid,
@@ -1871,7 +1872,7 @@ pg_stat_gssapi| SELECT s.pid,
s.gss_auth AS gss_authenticated,
s.gss_princ AS principal,
s.gss_enc AS encrypted
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, backend_mem_allocated)
WHERE (s.client_port IS NOT NULL);
pg_stat_progress_analyze| SELECT s.pid,
s.datid,
@@ -2052,7 +2053,7 @@ pg_stat_replication| SELECT s.pid,
w.sync_priority,
w.sync_state,
w.reply_time
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, backend_mem_allocated)
JOIN pg_stat_get_wal_senders() w(pid, state, sent_lsn, write_lsn, flush_lsn, replay_lsn, write_lag, flush_lag, replay_lag, sync_priority, sync_state, reply_time) ON ((s.pid = w.pid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_replication_slots| SELECT s.slot_name,
@@ -2086,7 +2087,7 @@ pg_stat_ssl| SELECT s.pid,
s.ssl_client_dn AS client_dn,
s.ssl_client_serial AS client_serial,
s.ssl_issuer_dn AS issuer_dn
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, backend_mem_allocated)
WHERE (s.client_port IS NOT NULL);
pg_stat_subscription| SELECT su.oid AS subid,
su.subname,
--
2.25.1
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: Add the ability to limit the amount of memory that can be allocated to backends.
2022-08-31 16:50 Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-08-31 17:34 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Justin Pryzby <[email protected]>
2022-09-04 03:40 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
@ 2022-09-09 17:14 ` Justin Pryzby <[email protected]>
2022-09-12 16:25 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
0 siblings, 1 reply; 22+ messages in thread
From: Justin Pryzby @ 2022-09-09 17:14 UTC (permalink / raw)
To: Reid Thompson <[email protected]>; +Cc: [email protected]
On Sat, Sep 03, 2022 at 11:40:03PM -0400, Reid Thompson wrote:
> > > + 0, 0, INT_MAX,
> > > + NULL, NULL, NULL
> > I think this needs a maximum like INT_MAX/1024/1024
>
> Is this noting that we'd set a ceiling of 2048MB?
The reason is that you're later multiplying it by 1024*1024, so you need
to limit it to avoid overflowing. Compare with
min_dynamic_shared_memory, Log_RotationSize, maintenance_work_mem,
autovacuum_work_mem.
typo: Explicitely
+ errmsg("request will exceed postgresql.conf defined max_total_backend_memory limit (%lu > %lu)",
I wouldn't mention postgresql.conf - it could be in
postgresql.auto.conf, or an include file, or a -c parameter.
Suggest: allocation would exceed max_total_backend_memory limit...
+ ereport(LOG, errmsg("decrease reduces reported backend memory allocated below zero; setting reported to 0"));
Suggest: deallocation would decrease backend memory below zero;
+ {"max_total_backend_memory", PGC_SIGHUP, RESOURCES_MEM,
Should this be PGC_SU_BACKEND to allow a superuser to set a higher
limit (or no limit)?
There's compilation warning under mingw cross compile due to
sizeof(long). See d914eb347 and other recent commits which I guess is
the current way to handle this.
http://cfbot.cputube.org/reid-thompson.html
For performance test, you'd want to check what happens with a large
number of max_connections (and maybe a large number of clients). TPS
isn't the only thing that matters. For example, a utility command might
sometimes do a lot of allocations (or deallocations), or a
"parameterized nested loop" may loop over over many outer tuples and
reset for each. There's also a lot of places that reset to a
"per-tuple" context. I started looking at its performance, but nothing
to show yet.
Would you keep people copied on your replies ("reply all") ? Otherwise
I (at least) may miss them. I think that's what's typical on these
lists (and the list tool is smart enough not to send duplicates to
people who are direct recipients).
--
Justin
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: Add the ability to limit the amount of memory that can be allocated to backends.
2022-08-31 16:50 Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-08-31 17:34 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Justin Pryzby <[email protected]>
2022-09-04 03:40 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-09-09 17:14 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Justin Pryzby <[email protected]>
@ 2022-09-12 16:25 ` Reid Thompson <[email protected]>
2022-09-15 08:07 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Ibrar Ahmed <[email protected]>
0 siblings, 1 reply; 22+ messages in thread
From: Reid Thompson @ 2022-09-12 16:25 UTC (permalink / raw)
To: Justin Pryzby <[email protected]>; +Cc: [email protected]; [email protected]
On Fri, 2022-09-09 at 12:14 -0500, Justin Pryzby wrote:
> On Sat, Sep 03, 2022 at 11:40:03PM -0400, Reid Thompson wrote:
> > > > + 0, 0, INT_MAX,
> > > > + NULL, NULL, NULL
> > > I think this needs a maximum like INT_MAX/1024/1024
> >
> > Is this noting that we'd set a ceiling of 2048MB?
>
> The reason is that you're later multiplying it by 1024*1024, so you
> need
> to limit it to avoid overflowing. Compare with
> min_dynamic_shared_memory, Log_RotationSize, maintenance_work_mem,
> autovacuum_work_mem.
What I originally attempted to implement is:
GUC "max_total_backend_memory" max value as INT_MAX = 2147483647 MB
(2251799812636672 bytes). And the other variables and comparisons as
bytes represented as uint64 to avoid overflow.
Is this invalid?
> typo: Explicitely
corrected
> + errmsg("request will exceed postgresql.conf
> defined max_total_backend_memory limit (%lu > %lu)",
>
> I wouldn't mention postgresql.conf - it could be in
> postgresql.auto.conf, or an include file, or a -c parameter.
> Suggest: allocation would exceed max_total_backend_memory limit...
>
updated
>
> + ereport(LOG, errmsg("decrease reduces reported
> backend memory allocated below zero; setting reported to 0"));
>
> Suggest: deallocation would decrease backend memory below zero;
updated
> + {"max_total_backend_memory", PGC_SIGHUP,
> RESOURCES_MEM,
>
>
>
> Should this be PGC_SU_BACKEND to allow a superuser to set a higher
> limit (or no limit)?
Sounds good to me. I'll update to that.
Would PGC_SUSET be too open?
> There's compilation warning under mingw cross compile due to
> sizeof(long). See d914eb347 and other recent commits which I guess
> is
> the current way to handle this.
> http://cfbot.cputube.org/reid-thompson.html
updated %lu to %llu and changed cast from uint64 to
unsigned long long in the ereport call
> For performance test, you'd want to check what happens with a large
> number of max_connections (and maybe a large number of clients). TPS
> isn't the only thing that matters. For example, a utility command
> might
> sometimes do a lot of allocations (or deallocations), or a
> "parameterized nested loop" may loop over over many outer tuples and
> reset for each. There's also a lot of places that reset to a
> "per-tuple" context. I started looking at its performance, but
> nothing
> to show yet.
Thanks
> Would you keep people copied on your replies ("reply all") ?
> Otherwise
> I (at least) may miss them. I think that's what's typical on these
> lists (and the list tool is smart enough not to send duplicates to
> people who are direct recipients).
Ok - will do, thanks.
--
Reid Thompson
Senior Software Engineer
Crunchy Data, Inc.
[email protected]
www.crunchydata.com
Attachments:
[text/x-patch] 0001-Add-tracking-of-backend-memory-allocated-to-pg_stat_.patch (28.5K, ../../[email protected]/2-0001-Add-tracking-of-backend-memory-allocated-to-pg_stat_.patch)
download | inline diff:
From 584a04f1b53948049e73165a4ffdd544c950ab0d Mon Sep 17 00:00:00 2001
From: Reid Thompson <[email protected]>
Date: Thu, 11 Aug 2022 12:01:25 -0400
Subject: [PATCH 1/2] Add tracking of backend memory allocated to
pg_stat_activity
This new field displays the current bytes of memory allocated to the
backend process. It is updated as memory for the process is
malloc'd/free'd. Memory allocated to items on the freelist is included in
the displayed value. Dynamic shared memory allocations are included
only in the value displayed for the backend that created them, they are
not included in the value for backends that are attached to them to
avoid double counting. On occasion, orphaned memory segments may be
cleaned up on postmaster startup. This may result in decreasing the sum
without a prior increment. We limit the floor of backend_mem_allocated
to zero. Updated pg_stat_activity documentation for the new column.
---
doc/src/sgml/monitoring.sgml | 12 +++
src/backend/catalog/system_views.sql | 1 +
src/backend/storage/ipc/dsm_impl.c | 80 +++++++++++++++
src/backend/utils/activity/backend_status.c | 105 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 4 +-
src/backend/utils/mmgr/aset.c | 18 ++++
src/backend/utils/mmgr/generation.c | 15 +++
src/backend/utils/mmgr/slab.c | 21 ++++
src/include/catalog/pg_proc.dat | 6 +-
src/include/utils/backend_status.h | 7 +-
src/test/regress/expected/rules.out | 9 +-
11 files changed, 269 insertions(+), 9 deletions(-)
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 1d9509a2f6..40ae638f25 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -947,6 +947,18 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>backend_mem_allocated</structfield> <type>bigint</type>
+ </para>
+ <para>
+ The byte count of memory allocated to this backend. Dynamic shared memory
+ allocations are included only in the value displayed for the backend that
+ created them, they are not included in the value for backends that are
+ attached to them to avoid double counting.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>query</structfield> <type>text</type>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 5a844b63a1..d23f0e9dbb 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -863,6 +863,7 @@ CREATE VIEW pg_stat_activity AS
S.backend_xid,
s.backend_xmin,
S.query_id,
+ S.backend_mem_allocated,
S.query,
S.backend_type
FROM pg_stat_get_activity(NULL) AS S
diff --git a/src/backend/storage/ipc/dsm_impl.c b/src/backend/storage/ipc/dsm_impl.c
index e1b90c5de4..3356bb65b5 100644
--- a/src/backend/storage/ipc/dsm_impl.c
+++ b/src/backend/storage/ipc/dsm_impl.c
@@ -66,6 +66,7 @@
#include "postmaster/postmaster.h"
#include "storage/dsm_impl.h"
#include "storage/fd.h"
+#include "utils/backend_status.h"
#include "utils/guc.h"
#include "utils/memutils.h"
@@ -232,6 +233,13 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ if (op == DSM_OP_DESTROY)
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
*mapped_address = NULL;
*mapped_size = 0;
if (op == DSM_OP_DESTROY && shm_unlink(name) != 0)
@@ -332,6 +340,36 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ {
+ /*
+ * Posix creation calls dsm_impl_posix_resize implying that resizing
+ * occurs or may be added in the future. As implemented
+ * dsm_impl_posix_resize utilizes fallocate or truncate, passing the
+ * whole new size as input, growing the allocation as needed (only
+ * truncate supports shrinking). We update by replacing the old
+ * allocation with the new.
+ */
+#if defined(HAVE_POSIX_FALLOCATE) && defined(__linux__)
+ /*
+ * posix_fallocate does not shrink allocations, adjust only on
+ * allocation increase.
+ */
+ if (request_size > *mapped_size)
+ {
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
+ pgstat_report_backend_mem_allocated_increase(request_size);
+ }
+#else
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
+ pgstat_report_backend_mem_allocated_increase(request_size);
+#endif
+ }
*mapped_address = address;
*mapped_size = request_size;
close(fd);
@@ -537,6 +575,14 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ if (op == DSM_OP_DESTROY)
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
*mapped_address = NULL;
*mapped_size = 0;
if (op == DSM_OP_DESTROY && shmctl(ident, IPC_RMID, NULL) < 0)
@@ -584,6 +630,13 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ pgstat_report_backend_mem_allocated_increase(request_size);
*mapped_address = address;
*mapped_size = request_size;
@@ -652,6 +705,13 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size,
return false;
}
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ if (op == DSM_OP_DESTROY)
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
*impl_private = NULL;
*mapped_address = NULL;
*mapped_size = 0;
@@ -768,6 +828,12 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size,
return false;
}
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ pgstat_report_backend_mem_allocated_increase(info.RegionSize);
*mapped_address = address;
*mapped_size = info.RegionSize;
*impl_private = hmap;
@@ -812,6 +878,13 @@ dsm_impl_mmap(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
*mapped_address = NULL;
*mapped_size = 0;
if (op == DSM_OP_DESTROY && unlink(name) != 0)
@@ -933,6 +1006,13 @@ dsm_impl_mmap(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ pgstat_report_backend_mem_allocated_increase(request_size);
*mapped_address = address;
*mapped_size = request_size;
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index c7ed1e6d7a..45da3af213 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -49,6 +49,8 @@ int pgstat_track_activity_query_size = 1024;
/* exposed so that backend_progress.c can access it */
PgBackendStatus *MyBEEntry = NULL;
+/* Memory allocated to this backend prior to pgstats initialization */
+uint64 backend_mem_allocated = 0;
static PgBackendStatus *BackendStatusArray = NULL;
static char *BackendAppnameBuffer = NULL;
@@ -400,6 +402,13 @@ pgstat_bestart(void)
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = UINT64CONST(0);
+ /*
+ * Move sum of memory allocated prior to pgstats initialization to pgstats
+ * and zero the local variable.
+ */
+ lbeentry.backend_mem_allocated = backend_mem_allocated;
+ backend_mem_allocated = 0;
+
/*
* we don't zero st_progress_param here to save cycles; nobody should
* examine it until st_progress_command has been set to something other
@@ -1148,3 +1157,99 @@ pgstat_clip_activity(const char *raw_activity)
return activity;
}
+
+/* --------
+ * pgstat_report_backend_mem_allocated_increase() -
+ *
+ * Called to report increase in memory allocated for this backend
+ * --------
+ */
+void
+pgstat_report_backend_mem_allocated_increase(uint64 allocation)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+
+ if (!beentry || !pgstat_track_activities)
+ {
+ /*
+ * Account for memory before pgstats is initialized. This will be
+ * migrated to pgstats on initialization.
+ */
+ backend_mem_allocated += allocation;
+
+ return;
+ }
+
+ /*
+ * Update my status entry, following the protocol of bumping
+ * st_changecount before and after. We use a volatile pointer here to
+ * ensure the compiler doesn't try to get cute.
+ */
+ PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
+ beentry->backend_mem_allocated += allocation;
+ PGSTAT_END_WRITE_ACTIVITY(beentry);
+}
+
+/* --------
+ * pgstat_report_backend_mem_allocated_decrease() -
+ *
+ * Called to report decrease in memory allocated for this backend
+ * --------
+ */
+void
+pgstat_report_backend_mem_allocated_decrease(uint64 deallocation)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+
+ /*
+ * Cases may occur where shared memory from a previous postmaster
+ * invocation still exist. These are cleaned up at startup by
+ * dsm_cleanup_using_control_segment. Limit decreasing memory allocated to
+ * zero in case no corresponding prior increase exists or decrease has
+ * already been accounted for.
+ */
+
+ if (!beentry || !pgstat_track_activities)
+ {
+ /*
+ * Account for memory before pgstats is initialized. This will be
+ * migrated to pgstats on initialization. Do not allow
+ * backend_mem_allocated to go below zero. If pgstats has not been
+ * initialized, we are in startup and we set backend_mem_allocated to
+ * zero in cases where it would go negative and skip generating an
+ * ereport.
+ */
+ if (deallocation > backend_mem_allocated)
+ backend_mem_allocated = 0;
+ else
+ backend_mem_allocated -= deallocation;
+
+ return;
+ }
+
+ /*
+ * Do not allow backend_mem_allocated to go below zero. ereport if we
+ * would have. There's no need for a lock around the read here as it's
+ * being referenced from the same backend which means that there shouldn't
+ * be concurrent writes. We want to generate an ereport in these cases.
+ */
+ if (deallocation > beentry->backend_mem_allocated)
+ {
+ ereport(LOG, errmsg("decrease reduces reported backend memory allocated below zero; setting reported to 0"));
+
+ /*
+ * Overwrite deallocation with current backend_mem_allocated so we end
+ * up at zero.
+ */
+ deallocation = beentry->backend_mem_allocated;
+ }
+
+ /*
+ * Update my status entry, following the protocol of bumping
+ * st_changecount before and after. We use a volatile pointer here to
+ * ensure the compiler doesn't try to get cute.
+ */
+ PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
+ beentry->backend_mem_allocated -= deallocation;
+ PGSTAT_END_WRITE_ACTIVITY(beentry);
+}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 4cca30aae7..1574aa8049 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -536,7 +536,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum
pg_stat_get_activity(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_ACTIVITY_COLS 30
+#define PG_STAT_GET_ACTIVITY_COLS 31
int num_backends = pgstat_fetch_stat_numbackends();
int curr_backend;
int pid = PG_ARGISNULL(0) ? -1 : PG_GETARG_INT32(0);
@@ -610,6 +610,8 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
else
nulls[16] = true;
+ values[30] = UInt64GetDatum(beentry->backend_mem_allocated);
+
/* Values only available to role member or pg_read_all_stats */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
diff --git a/src/backend/utils/mmgr/aset.c b/src/backend/utils/mmgr/aset.c
index b6eeb8abab..c91f8efa4d 100644
--- a/src/backend/utils/mmgr/aset.c
+++ b/src/backend/utils/mmgr/aset.c
@@ -47,6 +47,7 @@
#include "postgres.h"
#include "port/pg_bitutils.h"
+#include "utils/backend_status.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
#include "utils/memutils_memorychunk.h"
@@ -509,6 +510,7 @@ AllocSetContextCreateInternal(MemoryContext parent,
name);
((MemoryContext) set)->mem_allocated = firstBlockSize;
+ pgstat_report_backend_mem_allocated_increase(firstBlockSize);
return (MemoryContext) set;
}
@@ -532,6 +534,7 @@ AllocSetReset(MemoryContext context)
AllocBlock block;
Size keepersize PG_USED_FOR_ASSERTS_ONLY
= set->keeper->endptr - ((char *) set);
+ uint64 deallocation = 0;
AssertArg(AllocSetIsValid(set));
@@ -571,6 +574,7 @@ AllocSetReset(MemoryContext context)
{
/* Normal case, release the block */
context->mem_allocated -= block->endptr - ((char *) block);
+ deallocation += block->endptr - ((char *) block);
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
@@ -581,6 +585,7 @@ AllocSetReset(MemoryContext context)
}
Assert(context->mem_allocated == keepersize);
+ pgstat_report_backend_mem_allocated_decrease(deallocation);
/* Reset block size allocation sequence, too */
set->nextBlockSize = set->initBlockSize;
@@ -600,6 +605,7 @@ AllocSetDelete(MemoryContext context)
AllocBlock block = set->blocks;
Size keepersize PG_USED_FOR_ASSERTS_ONLY
= set->keeper->endptr - ((char *) set);
+ uint64 deallocation = 0;
AssertArg(AllocSetIsValid(set));
@@ -635,11 +641,13 @@ AllocSetDelete(MemoryContext context)
freelist->first_free = (AllocSetContext *) oldset->header.nextchild;
freelist->num_free--;
+ deallocation += oldset->header.mem_allocated;
/* All that remains is to free the header/initial block */
free(oldset);
}
Assert(freelist->num_free == 0);
+ pgstat_report_backend_mem_allocated_decrease(deallocation);
}
/* Now add the just-deleted context to the freelist. */
@@ -656,7 +664,10 @@ AllocSetDelete(MemoryContext context)
AllocBlock next = block->next;
if (block != set->keeper)
+ {
context->mem_allocated -= block->endptr - ((char *) block);
+ deallocation += block->endptr - ((char *) block);
+ }
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
@@ -669,6 +680,8 @@ AllocSetDelete(MemoryContext context)
}
Assert(context->mem_allocated == keepersize);
+ pgstat_report_backend_mem_allocated_decrease(deallocation +
+ context->mem_allocated);
/* Finally, free the context header, including the keeper block */
free(set);
@@ -712,6 +725,7 @@ AllocSetAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_backend_mem_allocated_increase(blksize);
block->aset = set;
block->freeptr = block->endptr = ((char *) block) + blksize;
@@ -916,6 +930,7 @@ AllocSetAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_backend_mem_allocated_increase(blksize);
block->aset = set;
block->freeptr = ((char *) block) + ALLOC_BLOCKHDRSZ;
@@ -1016,6 +1031,7 @@ AllocSetFree(void *pointer)
block->next->prev = block->prev;
set->header.mem_allocated -= block->endptr - ((char *) block);
+ pgstat_report_backend_mem_allocated_decrease(block->endptr - ((char *) block));
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
@@ -1127,7 +1143,9 @@ AllocSetRealloc(void *pointer, Size size)
/* updated separately, not to underflow when (oldblksize > blksize) */
set->header.mem_allocated -= oldblksize;
+ pgstat_report_backend_mem_allocated_decrease(oldblksize);
set->header.mem_allocated += blksize;
+ pgstat_report_backend_mem_allocated_increase(blksize);
block->freeptr = block->endptr = ((char *) block) + blksize;
diff --git a/src/backend/utils/mmgr/generation.c b/src/backend/utils/mmgr/generation.c
index b39894ec94..36e5b3f94d 100644
--- a/src/backend/utils/mmgr/generation.c
+++ b/src/backend/utils/mmgr/generation.c
@@ -37,6 +37,7 @@
#include "lib/ilist.h"
#include "port/pg_bitutils.h"
+#include "utils/backend_status.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
#include "utils/memutils_memorychunk.h"
@@ -258,6 +259,7 @@ GenerationContextCreate(MemoryContext parent,
name);
((MemoryContext) set)->mem_allocated = firstBlockSize;
+ pgstat_report_backend_mem_allocated_increase(firstBlockSize);
return (MemoryContext) set;
}
@@ -274,6 +276,7 @@ GenerationReset(MemoryContext context)
{
GenerationContext *set = (GenerationContext *) context;
dlist_mutable_iter miter;
+ uint64 deallocation = 0;
AssertArg(GenerationIsValid(set));
@@ -296,9 +299,14 @@ GenerationReset(MemoryContext context)
if (block == set->keeper)
GenerationBlockMarkEmpty(block);
else
+ {
+ deallocation += block->blksize;
GenerationBlockFree(set, block);
+ }
}
+ pgstat_report_backend_mem_allocated_decrease(deallocation);
+
/* set it so new allocations to make use of the keeper block */
set->block = set->keeper;
@@ -319,6 +327,9 @@ GenerationDelete(MemoryContext context)
{
/* Reset to release all releasable GenerationBlocks */
GenerationReset(context);
+
+ pgstat_report_backend_mem_allocated_decrease(context->mem_allocated);
+
/* And free the context header and keeper block */
free(context);
}
@@ -355,6 +366,7 @@ GenerationAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_backend_mem_allocated_increase(blksize);
/* block with a single (used) chunk */
block->context = set;
@@ -458,6 +470,7 @@ GenerationAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_backend_mem_allocated_increase(blksize);
/* initialize the new block */
GenerationBlockInit(set, block, blksize);
@@ -691,6 +704,8 @@ GenerationFree(void *pointer)
dlist_delete(&block->node);
set->header.mem_allocated -= block->blksize;
+ pgstat_report_backend_mem_allocated_decrease(block->blksize);
+
free(block);
}
diff --git a/src/backend/utils/mmgr/slab.c b/src/backend/utils/mmgr/slab.c
index 2d70adef09..e0e69b394e 100644
--- a/src/backend/utils/mmgr/slab.c
+++ b/src/backend/utils/mmgr/slab.c
@@ -53,6 +53,7 @@
#include "postgres.h"
#include "lib/ilist.h"
+#include "utils/backend_status.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
#include "utils/memutils_memorychunk.h"
@@ -218,6 +219,12 @@ SlabContextCreate(MemoryContext parent,
parent,
name);
+ /*
+ * If SlabContextCreate is updated to add headerSize to
+ * context->mem_allocated, then update here and SlabDelete appropriately
+ */
+ pgstat_report_backend_mem_allocated_increase(headerSize);
+
return (MemoryContext) slab;
}
@@ -233,6 +240,7 @@ SlabReset(MemoryContext context)
{
int i;
SlabContext *slab = castNode(SlabContext, context);
+ uint64 deallocation = 0;
Assert(slab);
@@ -258,9 +266,11 @@ SlabReset(MemoryContext context)
free(block);
slab->nblocks--;
context->mem_allocated -= slab->blockSize;
+ deallocation += slab->blockSize;
}
}
+ pgstat_report_backend_mem_allocated_decrease(deallocation);
slab->minFreeChunks = 0;
Assert(slab->nblocks == 0);
@@ -274,8 +284,17 @@ SlabReset(MemoryContext context)
void
SlabDelete(MemoryContext context)
{
+ /*
+ * Until header allocation is included in context->mem_allocated, cast to
+ * slab and decrement the headerSize
+ */
+ SlabContext *slab = castNode(SlabContext, context);
+
/* Reset to release all the SlabBlocks */
SlabReset(context);
+
+ pgstat_report_backend_mem_allocated_decrease(slab->headerSize);
+
/* And free the context header */
free(context);
}
@@ -344,6 +363,7 @@ SlabAlloc(MemoryContext context, Size size)
slab->minFreeChunks = slab->chunksPerBlock;
slab->nblocks += 1;
context->mem_allocated += slab->blockSize;
+ pgstat_report_backend_mem_allocated_increase(slab->blockSize);
}
/* grab the block from the freelist (even the new block is there) */
@@ -511,6 +531,7 @@ SlabFree(void *pointer)
free(block);
slab->nblocks--;
slab->header.mem_allocated -= slab->blockSize;
+ pgstat_report_backend_mem_allocated_decrease(slab->blockSize);
}
else
dlist_push_head(&slab->freelist[block->nfree], &block->node);
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index be47583122..e1bfb85b25 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5340,9 +5340,9 @@
proname => 'pg_stat_get_activity', prorows => '100', proisstrict => 'f',
proretset => 't', provolatile => 's', proparallel => 'r',
prorettype => 'record', proargtypes => 'int4',
- proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,int4,int8}',
- proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,leader_pid,query_id}',
+ proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,int4,int8,int8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,leader_pid,query_id,backend_mem_allocated}',
prosrc => 'pg_stat_get_activity' },
{ oid => '3318',
descr => 'statistics: information about progress of backends running maintenance command',
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index 7403bca25e..9bdc4197bd 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -168,6 +168,9 @@ typedef struct PgBackendStatus
/* query identifier, optionally computed using post_parse_analyze_hook */
uint64 st_query_id;
+
+ /* Current memory allocated to this backend */
+ uint64 backend_mem_allocated;
} PgBackendStatus;
@@ -305,7 +308,9 @@ extern const char *pgstat_get_backend_current_activity(int pid, bool checkUser);
extern const char *pgstat_get_crashed_backend_activity(int pid, char *buffer,
int buflen);
extern uint64 pgstat_get_my_query_id(void);
-
+extern void pgstat_report_backend_mem_allocated_increase(uint64 allocation);
+extern void pgstat_report_backend_mem_allocated_decrease(uint64 deallocation);
+extern uint64 pgstat_get_all_backend_memory_allocated(void);
/* ----------
* Support functions for the SQL-callable functions to
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 7ec3d2688f..674e5c6fe7 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1757,9 +1757,10 @@ pg_stat_activity| SELECT s.datid,
s.backend_xid,
s.backend_xmin,
s.query_id,
+ s.backend_mem_allocated,
s.query,
s.backend_type
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, backend_mem_allocated)
LEFT JOIN pg_database d ON ((s.datid = d.oid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_all_indexes| SELECT c.oid AS relid,
@@ -1871,7 +1872,7 @@ pg_stat_gssapi| SELECT s.pid,
s.gss_auth AS gss_authenticated,
s.gss_princ AS principal,
s.gss_enc AS encrypted
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, backend_mem_allocated)
WHERE (s.client_port IS NOT NULL);
pg_stat_progress_analyze| SELECT s.pid,
s.datid,
@@ -2052,7 +2053,7 @@ pg_stat_replication| SELECT s.pid,
w.sync_priority,
w.sync_state,
w.reply_time
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, backend_mem_allocated)
JOIN pg_stat_get_wal_senders() w(pid, state, sent_lsn, write_lsn, flush_lsn, replay_lsn, write_lag, flush_lag, replay_lag, sync_priority, sync_state, reply_time) ON ((s.pid = w.pid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_replication_slots| SELECT s.slot_name,
@@ -2086,7 +2087,7 @@ pg_stat_ssl| SELECT s.pid,
s.ssl_client_dn AS client_dn,
s.ssl_client_serial AS client_serial,
s.ssl_issuer_dn AS issuer_dn
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, backend_mem_allocated)
WHERE (s.client_port IS NOT NULL);
pg_stat_subscription| SELECT su.oid AS subid,
su.subname,
--
2.25.1
[text/x-patch] 0002-Add-the-ability-to-limit-the-amount-of-memory-that-c.patch (14.2K, ../../[email protected]/3-0002-Add-the-ability-to-limit-the-amount-of-memory-that-c.patch)
download | inline diff:
From 23d33838e3780ef02daad2bc737290d9905745a7 Mon Sep 17 00:00:00 2001
From: Reid Thompson <[email protected]>
Date: Sat, 4 Jun 2022 22:23:59 -0400
Subject: [PATCH 2/2] Add the ability to limit the amount of memory that can be
allocated to backends.
This builds on the work that adds backend memory allocated to pg_stat_activity.
Add GUC variable max_total_backend_memory.
Specifies a limit to the amount of memory (in MB) that may be allocated to
backends in total (i.e. this is not a per user or per backend limit). If unset,
or set to 0 it is disabled. It is intended as a resource to help avoid the OOM
killer on LINUX and manage resources in general. A backend request that would
push the total over the limit will be denied with an out of memory error causing
that backend's current query/transaction to fail. Due to the dynamic nature of
memory allocations, this limit is not exact. If within 1.5MB of the limit and
two backends request 1MB each at the same time both may be allocated, and exceed
the limit. Further requests will not be allocated until dropping below the
limit. Keep this in mind when setting this value. This limit does not affect
auxiliary backend processes. Backend memory allocations are displayed in the
pg_stat_activity view.
---
doc/src/sgml/config.sgml | 26 +++++
src/backend/storage/ipc/dsm_impl.c | 12 ++
src/backend/utils/activity/backend_status.c | 109 +++++++++++++++++-
src/backend/utils/misc/guc.c | 11 ++
src/backend/utils/misc/postgresql.conf.sample | 3 +
src/backend/utils/mmgr/aset.c | 17 +++
src/backend/utils/mmgr/generation.c | 9 ++
src/backend/utils/mmgr/slab.c | 8 ++
src/include/utils/backend_status.h | 2 +
9 files changed, 196 insertions(+), 1 deletion(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index a5cd4e44c7..e70ea71ba1 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -2079,6 +2079,32 @@ include_dir 'conf.d'
</listitem>
</varlistentry>
+ <varlistentry id="guc-max-total-backend-memory" xreflabel="max_total_backend_memory">
+ <term><varname>max_total_backend_memory</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>max_total_backend_memory</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies a limit to the amount of memory (MB) that may be allocated to
+ backends in total (i.e. this is not a per user or per backend limit).
+ If unset, or set to 0 it is disabled. A backend request that would
+ push the total over the limit will be denied with an out of memory
+ error causing that backend's current query/transaction to fail. Due to
+ the dynamic nature of memory allocations, this limit is not exact. If
+ within 1.5MB of the limit and two backends request 1MB each at the same
+ time both may be allocated, and exceed the limit. Further requests will
+ not be allocated until dropping below the limit. Keep this in mind when
+ setting this value. This limit does not affect auxiliary backend
+ processes <xref linkend="glossary-auxiliary-proc"/> . Backend memory
+ allocations (<varname>backend_mem_allocated</varname>) are displayed in
+ the <link linkend="monitoring-pg-stat-activity-view"><structname>pg_stat_activity</structname></link>
+ view.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/src/backend/storage/ipc/dsm_impl.c b/src/backend/storage/ipc/dsm_impl.c
index 3356bb65b5..cc061056a3 100644
--- a/src/backend/storage/ipc/dsm_impl.c
+++ b/src/backend/storage/ipc/dsm_impl.c
@@ -253,6 +253,10 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
return true;
}
+ /* Do not exceed maximum allowed memory allocation */
+ if (op == DSM_OP_CREATE && exceeds_max_total_bkend_mem(request_size))
+ return false;
+
/*
* Create new segment or open an existing one for attach.
*
@@ -524,6 +528,10 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size,
int flags = IPCProtection;
size_t segsize;
+ /* Do not exceed maximum allowed memory allocation */
+ if (op == DSM_OP_CREATE && exceeds_max_total_bkend_mem(request_size))
+ return false;
+
/*
* Allocate the memory BEFORE acquiring the resource, so that we don't
* leak the resource if memory allocation fails.
@@ -718,6 +726,10 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size,
return true;
}
+ /* Do not exceed maximum allowed memory allocation */
+ if (op == DSM_OP_CREATE && exceeds_max_total_bkend_mem(request_size))
+ return false;
+
/* Create new segment or open an existing one for attach. */
if (op == DSM_OP_CREATE)
{
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 45da3af213..9c2fc2f07c 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -44,6 +44,8 @@
*/
bool pgstat_track_activities = false;
int pgstat_track_activity_query_size = 1024;
+/* Max backend memory allocation allowed (MB). 0 = disabled */
+int max_total_bkend_mem = 0;
/* exposed so that backend_progress.c can access it */
@@ -1235,7 +1237,7 @@ pgstat_report_backend_mem_allocated_decrease(uint64 deallocation)
*/
if (deallocation > beentry->backend_mem_allocated)
{
- ereport(LOG, errmsg("decrease reduces reported backend memory allocated below zero; setting reported to 0"));
+ ereport(LOG, errmsg("deallocation would decrease backend memory below zero; setting reported to 0"));
/*
* Overwrite deallocation with current backend_mem_allocated so we end
@@ -1253,3 +1255,108 @@ pgstat_report_backend_mem_allocated_decrease(uint64 deallocation)
beentry->backend_mem_allocated -= deallocation;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
+
+/* ----------
+ * pgstat_get_all_backend_memory_allocated() -
+ *
+ * Return a uint64 representing the current shared memory allocated to all
+ * backends. This looks directly at the BackendStatusArray, and so will
+ * provide current information regardless of the age of our transaction's
+ * snapshot of the status array.
+ * In the future we will likely utilize additional values - perhaps limit
+ * backend allocation by user/role, etc.
+ * ----------
+ */
+uint64
+pgstat_get_all_backend_memory_allocated(void)
+{
+ PgBackendStatus *beentry;
+ int i;
+ uint64 all_backend_memory_allocated = 0;
+
+ beentry = BackendStatusArray;
+
+ /*
+ * We probably shouldn't get here before shared memory has been set up,
+ * but be safe.
+ */
+ if (beentry == NULL || BackendActivityBuffer == NULL)
+ return 0;
+
+ /*
+ * We include AUX procs in all backend memory calculation
+ */
+ for (i = 1; i <= NumBackendStatSlots; i++)
+ {
+ /*
+ * We use a volatile pointer here to ensure the compiler doesn't try to
+ * get cute.
+ */
+ volatile PgBackendStatus *vbeentry = beentry;
+ bool found;
+ uint64 backend_mem_allocated = 0;
+
+ for (;;)
+ {
+ int before_changecount;
+ int after_changecount;
+
+ pgstat_begin_read_activity(vbeentry, before_changecount);
+
+ /* Ignore invalid entries, which may contain invalid data.
+ * See pgstat_beshutdown_hook()
+ */
+ if (vbeentry->st_procpid > 0)
+ backend_mem_allocated = vbeentry->backend_mem_allocated;
+
+ pgstat_end_read_activity(vbeentry, after_changecount);
+
+ if ((found = pgstat_read_activity_complete(before_changecount,
+ after_changecount)))
+ break;
+
+ /* Make sure we can break out of loop if stuck... */
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ if (found)
+ all_backend_memory_allocated += backend_mem_allocated;
+
+ beentry++;
+ }
+
+ return all_backend_memory_allocated;
+}
+
+/*
+ * Determine if allocation request will exceed max backend memory allowed.
+ * Do not apply to auxiliary processes.
+ */
+bool
+exceeds_max_total_bkend_mem(uint64 allocation_request)
+{
+ bool result = false;
+
+ /* Exclude auxiliary processes from the check */
+ if (MyAuxProcType != NotAnAuxProcess)
+ return result;
+
+ /* Convert max_total_bkend_mem to bytes for comparison */
+ if (max_total_bkend_mem &&
+ pgstat_get_all_backend_memory_allocated() +
+ allocation_request > (uint64)max_total_bkend_mem * 1024 * 1024)
+ {
+ /*
+ * Explicitly identify the OOM being a result of this configuration
+ * parameter vs a system failure to allocate OOM.
+ */
+ ereport(WARNING,
+ errmsg("allocation would exceed max_total_backend_memory limit (%llu > %llu)",
+ (unsigned long long)pgstat_get_all_backend_memory_allocated() +
+ allocation_request, (unsigned long long)max_total_bkend_mem * 1024 * 1024));
+
+ result = true;
+ }
+
+ return result;
+}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 9fbbfb1be5..4e256a77d5 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -3664,6 +3664,17 @@ static struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_total_backend_memory", PGC_SU_BACKEND, RESOURCES_MEM,
+ gettext_noop("Restrict total backend memory allocations to this max."),
+ gettext_noop("0 turns this feature off."),
+ GUC_UNIT_MB
+ },
+ &max_total_bkend_mem,
+ 0, 0, INT_MAX,
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, 0, 0, 0, NULL, NULL, NULL
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 90bec0502c..8e944f6511 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -155,6 +155,9 @@
# mmap
# (change requires restart)
#min_dynamic_shared_memory = 0MB # (change requires restart)
+#max_total_backend_memory = 0MB # Restrict total backend memory allocations
+ # to this max (in MB). 0 turns this feature
+ # off.
# - Disk -
diff --git a/src/backend/utils/mmgr/aset.c b/src/backend/utils/mmgr/aset.c
index c91f8efa4d..ac9a1ced3f 100644
--- a/src/backend/utils/mmgr/aset.c
+++ b/src/backend/utils/mmgr/aset.c
@@ -428,6 +428,10 @@ AllocSetContextCreateInternal(MemoryContext parent,
else
firstBlockSize = Max(firstBlockSize, initBlockSize);
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(firstBlockSize))
+ return NULL;
+
/*
* Allocate the initial block. Unlike other aset.c blocks, it starts with
* the context header and its block header follows that.
@@ -720,6 +724,11 @@ AllocSetAlloc(MemoryContext context, Size size)
{
chunk_size = MAXALIGN(size);
blksize = chunk_size + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ;
+
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
block = (AllocBlock) malloc(blksize);
if (block == NULL)
return NULL;
@@ -911,6 +920,10 @@ AllocSetAlloc(MemoryContext context, Size size)
while (blksize < required_size)
blksize <<= 1;
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
/* Try to allocate it */
block = (AllocBlock) malloc(blksize);
@@ -1133,6 +1146,10 @@ AllocSetRealloc(void *pointer, Size size)
blksize = chksize + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ;
oldblksize = block->endptr - ((char *) block);
+ /* Do not exceed maximum allowed memory allocation */
+ if (blksize > oldblksize && exceeds_max_total_bkend_mem(blksize - oldblksize))
+ return NULL;
+
block = (AllocBlock) realloc(block, blksize);
if (block == NULL)
{
diff --git a/src/backend/utils/mmgr/generation.c b/src/backend/utils/mmgr/generation.c
index 36e5b3f94d..1d5720836c 100644
--- a/src/backend/utils/mmgr/generation.c
+++ b/src/backend/utils/mmgr/generation.c
@@ -192,6 +192,9 @@ GenerationContextCreate(MemoryContext parent,
else
allocSize = Max(allocSize, initBlockSize);
+ if (exceeds_max_total_bkend_mem(allocSize))
+ return NULL;
+
/*
* Allocate the initial block. Unlike other generation.c blocks, it
* starts with the context header and its block header follows that.
@@ -361,6 +364,9 @@ GenerationAlloc(MemoryContext context, Size size)
{
Size blksize = required_size + Generation_BLOCKHDRSZ;
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
block = (GenerationBlock *) malloc(blksize);
if (block == NULL)
return NULL;
@@ -464,6 +470,9 @@ GenerationAlloc(MemoryContext context, Size size)
if (blksize < required_size)
blksize = pg_nextpower2_size_t(required_size);
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
block = (GenerationBlock *) malloc(blksize);
if (block == NULL)
diff --git a/src/backend/utils/mmgr/slab.c b/src/backend/utils/mmgr/slab.c
index e0e69b394e..63c07120dd 100644
--- a/src/backend/utils/mmgr/slab.c
+++ b/src/backend/utils/mmgr/slab.c
@@ -177,6 +177,10 @@ SlabContextCreate(MemoryContext parent,
headerSize += chunksPerBlock * sizeof(bool);
#endif
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(headerSize))
+ return NULL;
+
slab = (SlabContext *) malloc(headerSize);
if (slab == NULL)
{
@@ -331,6 +335,10 @@ SlabAlloc(MemoryContext context, Size size)
*/
if (slab->minFreeChunks == 0)
{
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(slab->blockSize))
+ return NULL;
+
block = (SlabBlock *) malloc(slab->blockSize);
if (block == NULL)
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index 9bdc4197bd..3b940ff98e 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -270,6 +270,7 @@ typedef struct LocalPgBackendStatus
*/
extern PGDLLIMPORT bool pgstat_track_activities;
extern PGDLLIMPORT int pgstat_track_activity_query_size;
+extern PGDLLIMPORT int max_total_bkend_mem;
/* ----------
@@ -321,6 +322,7 @@ extern int pgstat_fetch_stat_numbackends(void);
extern PgBackendStatus *pgstat_fetch_stat_beentry(int beid);
extern LocalPgBackendStatus *pgstat_fetch_stat_local_beentry(int beid);
extern char *pgstat_clip_activity(const char *raw_activity);
+extern bool exceeds_max_total_bkend_mem(uint64 allocation_request);
#endif /* BACKEND_STATUS_H */
--
2.25.1
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: Add the ability to limit the amount of memory that can be allocated to backends.
2022-08-31 16:50 Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-08-31 17:34 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Justin Pryzby <[email protected]>
2022-09-04 03:40 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-09-09 17:14 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Justin Pryzby <[email protected]>
2022-09-12 16:25 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
@ 2022-09-15 08:07 ` Ibrar Ahmed <[email protected]>
2022-09-15 14:58 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
0 siblings, 1 reply; 22+ messages in thread
From: Ibrar Ahmed @ 2022-09-15 08:07 UTC (permalink / raw)
To: [email protected]; [email protected]; +Cc: Justin Pryzby <[email protected]>
On Mon, Sep 12, 2022 at 8:30 PM Reid Thompson <[email protected]>
wrote:
> On Fri, 2022-09-09 at 12:14 -0500, Justin Pryzby wrote:
> > On Sat, Sep 03, 2022 at 11:40:03PM -0400, Reid Thompson wrote:
> > > > > + 0, 0, INT_MAX,
> > > > > + NULL, NULL, NULL
> > > > I think this needs a maximum like INT_MAX/1024/1024
> > >
> > > Is this noting that we'd set a ceiling of 2048MB?
> >
> > The reason is that you're later multiplying it by 1024*1024, so you
> > need
> > to limit it to avoid overflowing. Compare with
> > min_dynamic_shared_memory, Log_RotationSize, maintenance_work_mem,
> > autovacuum_work_mem.
>
> What I originally attempted to implement is:
> GUC "max_total_backend_memory" max value as INT_MAX = 2147483647 MB
> (2251799812636672 bytes). And the other variables and comparisons as
> bytes represented as uint64 to avoid overflow.
>
> Is this invalid?
>
> > typo: Explicitely
>
> corrected
>
> > + errmsg("request will exceed postgresql.conf
> > defined max_total_backend_memory limit (%lu > %lu)",
> >
> > I wouldn't mention postgresql.conf - it could be in
> > postgresql.auto.conf, or an include file, or a -c parameter.
> > Suggest: allocation would exceed max_total_backend_memory limit...
> >
>
> updated
>
> >
> > + ereport(LOG, errmsg("decrease reduces reported
> > backend memory allocated below zero; setting reported to 0"));
> >
> > Suggest: deallocation would decrease backend memory below zero;
>
> updated
>
> > + {"max_total_backend_memory", PGC_SIGHUP,
> > RESOURCES_MEM,
> >
> >
> >
> > Should this be PGC_SU_BACKEND to allow a superuser to set a higher
> > limit (or no limit)?
>
> Sounds good to me. I'll update to that.
> Would PGC_SUSET be too open?
>
> > There's compilation warning under mingw cross compile due to
> > sizeof(long). See d914eb347 and other recent commits which I guess
> > is
> > the current way to handle this.
> > http://cfbot.cputube.org/reid-thompson.html
>
> updated %lu to %llu and changed cast from uint64 to
> unsigned long long in the ereport call
>
> > For performance test, you'd want to check what happens with a large
> > number of max_connections (and maybe a large number of clients). TPS
> > isn't the only thing that matters. For example, a utility command
> > might
> > sometimes do a lot of allocations (or deallocations), or a
> > "parameterized nested loop" may loop over over many outer tuples and
> > reset for each. There's also a lot of places that reset to a
> > "per-tuple" context. I started looking at its performance, but
> > nothing
> > to show yet.
>
> Thanks
>
> > Would you keep people copied on your replies ("reply all") ?
> > Otherwise
> > I (at least) may miss them. I think that's what's typical on these
> > lists (and the list tool is smart enough not to send duplicates to
> > people who are direct recipients).
>
> Ok - will do, thanks.
>
> --
> Reid Thompson
> Senior Software Engineer
> Crunchy Data, Inc.
>
> [email protected]
> www.crunchydata.com
>
>
> The patch does not apply; please rebase the patch.
patching file src/backend/utils/misc/guc.c
Hunk #1 FAILED at 3664.
1 out of 1 hunk FAILED -- saving rejects to file
src/backend/utils/misc/guc.c.rej
patching file src/backend/utils/misc/postgresql.conf.sample
--
Ibrar Ahmed
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: Add the ability to limit the amount of memory that can be allocated to backends.
2022-08-31 16:50 Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-08-31 17:34 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Justin Pryzby <[email protected]>
2022-09-04 03:40 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-09-09 17:14 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Justin Pryzby <[email protected]>
2022-09-12 16:25 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-09-15 08:07 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Ibrar Ahmed <[email protected]>
@ 2022-09-15 14:58 ` Reid Thompson <[email protected]>
2022-10-24 15:27 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Arne Roland <[email protected]>
0 siblings, 1 reply; 22+ messages in thread
From: Reid Thompson @ 2022-09-15 14:58 UTC (permalink / raw)
To: Ibrar Ahmed <[email protected]>; [email protected]; +Cc: [email protected]; Justin Pryzby <[email protected]>
On Thu, 2022-09-15 at 12:07 +0400, Ibrar Ahmed wrote:
>
> The patch does not apply; please rebase the patch.
>
> patching file src/backend/utils/misc/guc.c
> Hunk #1 FAILED at 3664.
> 1 out of 1 hunk FAILED -- saving rejects to file
> src/backend/utils/misc/guc.c.rej
>
> patching file src/backend/utils/misc/postgresql.conf.sample
>
rebased patches attached.
Thanks,
Reid
Attachments:
[text/x-patch] 0001-Add-tracking-of-backend-memory-allocated-to-pg_stat_.patch (28.5K, ../../[email protected]/2-0001-Add-tracking-of-backend-memory-allocated-to-pg_stat_.patch)
download | inline diff:
From 0e7010c53508d5a396edd16fd9166abe431f5dbe Mon Sep 17 00:00:00 2001
From: Reid Thompson <[email protected]>
Date: Thu, 11 Aug 2022 12:01:25 -0400
Subject: [PATCH 1/2] Add tracking of backend memory allocated to
pg_stat_activity
This new field displays the current bytes of memory allocated to the
backend process. It is updated as memory for the process is
malloc'd/free'd. Memory allocated to items on the freelist is included in
the displayed value. Dynamic shared memory allocations are included
only in the value displayed for the backend that created them, they are
not included in the value for backends that are attached to them to
avoid double counting. On occasion, orphaned memory segments may be
cleaned up on postmaster startup. This may result in decreasing the sum
without a prior increment. We limit the floor of backend_mem_allocated
to zero. Updated pg_stat_activity documentation for the new column.
---
doc/src/sgml/monitoring.sgml | 12 +++
src/backend/catalog/system_views.sql | 1 +
src/backend/storage/ipc/dsm_impl.c | 80 +++++++++++++++
src/backend/utils/activity/backend_status.c | 105 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 4 +-
src/backend/utils/mmgr/aset.c | 18 ++++
src/backend/utils/mmgr/generation.c | 15 +++
src/backend/utils/mmgr/slab.c | 21 ++++
src/include/catalog/pg_proc.dat | 6 +-
src/include/utils/backend_status.h | 7 +-
src/test/regress/expected/rules.out | 9 +-
11 files changed, 269 insertions(+), 9 deletions(-)
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 1d9509a2f6..40ae638f25 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -947,6 +947,18 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>backend_mem_allocated</structfield> <type>bigint</type>
+ </para>
+ <para>
+ The byte count of memory allocated to this backend. Dynamic shared memory
+ allocations are included only in the value displayed for the backend that
+ created them, they are not included in the value for backends that are
+ attached to them to avoid double counting.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>query</structfield> <type>text</type>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 55f7ec79e0..a78750ab12 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -862,6 +862,7 @@ CREATE VIEW pg_stat_activity AS
S.backend_xid,
s.backend_xmin,
S.query_id,
+ S.backend_mem_allocated,
S.query,
S.backend_type
FROM pg_stat_get_activity(NULL) AS S
diff --git a/src/backend/storage/ipc/dsm_impl.c b/src/backend/storage/ipc/dsm_impl.c
index e1b90c5de4..3356bb65b5 100644
--- a/src/backend/storage/ipc/dsm_impl.c
+++ b/src/backend/storage/ipc/dsm_impl.c
@@ -66,6 +66,7 @@
#include "postmaster/postmaster.h"
#include "storage/dsm_impl.h"
#include "storage/fd.h"
+#include "utils/backend_status.h"
#include "utils/guc.h"
#include "utils/memutils.h"
@@ -232,6 +233,13 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ if (op == DSM_OP_DESTROY)
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
*mapped_address = NULL;
*mapped_size = 0;
if (op == DSM_OP_DESTROY && shm_unlink(name) != 0)
@@ -332,6 +340,36 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ {
+ /*
+ * Posix creation calls dsm_impl_posix_resize implying that resizing
+ * occurs or may be added in the future. As implemented
+ * dsm_impl_posix_resize utilizes fallocate or truncate, passing the
+ * whole new size as input, growing the allocation as needed (only
+ * truncate supports shrinking). We update by replacing the old
+ * allocation with the new.
+ */
+#if defined(HAVE_POSIX_FALLOCATE) && defined(__linux__)
+ /*
+ * posix_fallocate does not shrink allocations, adjust only on
+ * allocation increase.
+ */
+ if (request_size > *mapped_size)
+ {
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
+ pgstat_report_backend_mem_allocated_increase(request_size);
+ }
+#else
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
+ pgstat_report_backend_mem_allocated_increase(request_size);
+#endif
+ }
*mapped_address = address;
*mapped_size = request_size;
close(fd);
@@ -537,6 +575,14 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ if (op == DSM_OP_DESTROY)
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
*mapped_address = NULL;
*mapped_size = 0;
if (op == DSM_OP_DESTROY && shmctl(ident, IPC_RMID, NULL) < 0)
@@ -584,6 +630,13 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ pgstat_report_backend_mem_allocated_increase(request_size);
*mapped_address = address;
*mapped_size = request_size;
@@ -652,6 +705,13 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size,
return false;
}
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ if (op == DSM_OP_DESTROY)
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
*impl_private = NULL;
*mapped_address = NULL;
*mapped_size = 0;
@@ -768,6 +828,12 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size,
return false;
}
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ pgstat_report_backend_mem_allocated_increase(info.RegionSize);
*mapped_address = address;
*mapped_size = info.RegionSize;
*impl_private = hmap;
@@ -812,6 +878,13 @@ dsm_impl_mmap(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
*mapped_address = NULL;
*mapped_size = 0;
if (op == DSM_OP_DESTROY && unlink(name) != 0)
@@ -933,6 +1006,13 @@ dsm_impl_mmap(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ pgstat_report_backend_mem_allocated_increase(request_size);
*mapped_address = address;
*mapped_size = request_size;
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index c7ed1e6d7a..45da3af213 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -49,6 +49,8 @@ int pgstat_track_activity_query_size = 1024;
/* exposed so that backend_progress.c can access it */
PgBackendStatus *MyBEEntry = NULL;
+/* Memory allocated to this backend prior to pgstats initialization */
+uint64 backend_mem_allocated = 0;
static PgBackendStatus *BackendStatusArray = NULL;
static char *BackendAppnameBuffer = NULL;
@@ -400,6 +402,13 @@ pgstat_bestart(void)
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = UINT64CONST(0);
+ /*
+ * Move sum of memory allocated prior to pgstats initialization to pgstats
+ * and zero the local variable.
+ */
+ lbeentry.backend_mem_allocated = backend_mem_allocated;
+ backend_mem_allocated = 0;
+
/*
* we don't zero st_progress_param here to save cycles; nobody should
* examine it until st_progress_command has been set to something other
@@ -1148,3 +1157,99 @@ pgstat_clip_activity(const char *raw_activity)
return activity;
}
+
+/* --------
+ * pgstat_report_backend_mem_allocated_increase() -
+ *
+ * Called to report increase in memory allocated for this backend
+ * --------
+ */
+void
+pgstat_report_backend_mem_allocated_increase(uint64 allocation)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+
+ if (!beentry || !pgstat_track_activities)
+ {
+ /*
+ * Account for memory before pgstats is initialized. This will be
+ * migrated to pgstats on initialization.
+ */
+ backend_mem_allocated += allocation;
+
+ return;
+ }
+
+ /*
+ * Update my status entry, following the protocol of bumping
+ * st_changecount before and after. We use a volatile pointer here to
+ * ensure the compiler doesn't try to get cute.
+ */
+ PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
+ beentry->backend_mem_allocated += allocation;
+ PGSTAT_END_WRITE_ACTIVITY(beentry);
+}
+
+/* --------
+ * pgstat_report_backend_mem_allocated_decrease() -
+ *
+ * Called to report decrease in memory allocated for this backend
+ * --------
+ */
+void
+pgstat_report_backend_mem_allocated_decrease(uint64 deallocation)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+
+ /*
+ * Cases may occur where shared memory from a previous postmaster
+ * invocation still exist. These are cleaned up at startup by
+ * dsm_cleanup_using_control_segment. Limit decreasing memory allocated to
+ * zero in case no corresponding prior increase exists or decrease has
+ * already been accounted for.
+ */
+
+ if (!beentry || !pgstat_track_activities)
+ {
+ /*
+ * Account for memory before pgstats is initialized. This will be
+ * migrated to pgstats on initialization. Do not allow
+ * backend_mem_allocated to go below zero. If pgstats has not been
+ * initialized, we are in startup and we set backend_mem_allocated to
+ * zero in cases where it would go negative and skip generating an
+ * ereport.
+ */
+ if (deallocation > backend_mem_allocated)
+ backend_mem_allocated = 0;
+ else
+ backend_mem_allocated -= deallocation;
+
+ return;
+ }
+
+ /*
+ * Do not allow backend_mem_allocated to go below zero. ereport if we
+ * would have. There's no need for a lock around the read here as it's
+ * being referenced from the same backend which means that there shouldn't
+ * be concurrent writes. We want to generate an ereport in these cases.
+ */
+ if (deallocation > beentry->backend_mem_allocated)
+ {
+ ereport(LOG, errmsg("decrease reduces reported backend memory allocated below zero; setting reported to 0"));
+
+ /*
+ * Overwrite deallocation with current backend_mem_allocated so we end
+ * up at zero.
+ */
+ deallocation = beentry->backend_mem_allocated;
+ }
+
+ /*
+ * Update my status entry, following the protocol of bumping
+ * st_changecount before and after. We use a volatile pointer here to
+ * ensure the compiler doesn't try to get cute.
+ */
+ PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
+ beentry->backend_mem_allocated -= deallocation;
+ PGSTAT_END_WRITE_ACTIVITY(beentry);
+}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index be15b4b2e5..35d497a12e 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -536,7 +536,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum
pg_stat_get_activity(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_ACTIVITY_COLS 30
+#define PG_STAT_GET_ACTIVITY_COLS 31
int num_backends = pgstat_fetch_stat_numbackends();
int curr_backend;
int pid = PG_ARGISNULL(0) ? -1 : PG_GETARG_INT32(0);
@@ -610,6 +610,8 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
else
nulls[16] = true;
+ values[30] = UInt64GetDatum(beentry->backend_mem_allocated);
+
/* Values only available to role member or pg_read_all_stats */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
diff --git a/src/backend/utils/mmgr/aset.c b/src/backend/utils/mmgr/aset.c
index ec423375ae..2ac6f10e12 100644
--- a/src/backend/utils/mmgr/aset.c
+++ b/src/backend/utils/mmgr/aset.c
@@ -47,6 +47,7 @@
#include "postgres.h"
#include "port/pg_bitutils.h"
+#include "utils/backend_status.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
#include "utils/memutils_memorychunk.h"
@@ -509,6 +510,7 @@ AllocSetContextCreateInternal(MemoryContext parent,
name);
((MemoryContext) set)->mem_allocated = firstBlockSize;
+ pgstat_report_backend_mem_allocated_increase(firstBlockSize);
return (MemoryContext) set;
}
@@ -532,6 +534,7 @@ AllocSetReset(MemoryContext context)
AllocBlock block;
Size keepersize PG_USED_FOR_ASSERTS_ONLY
= set->keeper->endptr - ((char *) set);
+ uint64 deallocation = 0;
AssertArg(AllocSetIsValid(set));
@@ -571,6 +574,7 @@ AllocSetReset(MemoryContext context)
{
/* Normal case, release the block */
context->mem_allocated -= block->endptr - ((char *) block);
+ deallocation += block->endptr - ((char *) block);
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
@@ -581,6 +585,7 @@ AllocSetReset(MemoryContext context)
}
Assert(context->mem_allocated == keepersize);
+ pgstat_report_backend_mem_allocated_decrease(deallocation);
/* Reset block size allocation sequence, too */
set->nextBlockSize = set->initBlockSize;
@@ -600,6 +605,7 @@ AllocSetDelete(MemoryContext context)
AllocBlock block = set->blocks;
Size keepersize PG_USED_FOR_ASSERTS_ONLY
= set->keeper->endptr - ((char *) set);
+ uint64 deallocation = 0;
AssertArg(AllocSetIsValid(set));
@@ -635,11 +641,13 @@ AllocSetDelete(MemoryContext context)
freelist->first_free = (AllocSetContext *) oldset->header.nextchild;
freelist->num_free--;
+ deallocation += oldset->header.mem_allocated;
/* All that remains is to free the header/initial block */
free(oldset);
}
Assert(freelist->num_free == 0);
+ pgstat_report_backend_mem_allocated_decrease(deallocation);
}
/* Now add the just-deleted context to the freelist. */
@@ -656,7 +664,10 @@ AllocSetDelete(MemoryContext context)
AllocBlock next = block->next;
if (block != set->keeper)
+ {
context->mem_allocated -= block->endptr - ((char *) block);
+ deallocation += block->endptr - ((char *) block);
+ }
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
@@ -669,6 +680,8 @@ AllocSetDelete(MemoryContext context)
}
Assert(context->mem_allocated == keepersize);
+ pgstat_report_backend_mem_allocated_decrease(deallocation +
+ context->mem_allocated);
/* Finally, free the context header, including the keeper block */
free(set);
@@ -718,6 +731,7 @@ AllocSetAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_backend_mem_allocated_increase(blksize);
block->aset = set;
block->freeptr = block->endptr = ((char *) block) + blksize;
@@ -928,6 +942,7 @@ AllocSetAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_backend_mem_allocated_increase(blksize);
block->aset = set;
block->freeptr = ((char *) block) + ALLOC_BLOCKHDRSZ;
@@ -1028,6 +1043,7 @@ AllocSetFree(void *pointer)
block->next->prev = block->prev;
set->header.mem_allocated -= block->endptr - ((char *) block);
+ pgstat_report_backend_mem_allocated_decrease(block->endptr - ((char *) block));
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
@@ -1144,7 +1160,9 @@ AllocSetRealloc(void *pointer, Size size)
/* updated separately, not to underflow when (oldblksize > blksize) */
set->header.mem_allocated -= oldblksize;
+ pgstat_report_backend_mem_allocated_decrease(oldblksize);
set->header.mem_allocated += blksize;
+ pgstat_report_backend_mem_allocated_increase(blksize);
block->freeptr = block->endptr = ((char *) block) + blksize;
diff --git a/src/backend/utils/mmgr/generation.c b/src/backend/utils/mmgr/generation.c
index c743b24fa7..34b11392ff 100644
--- a/src/backend/utils/mmgr/generation.c
+++ b/src/backend/utils/mmgr/generation.c
@@ -37,6 +37,7 @@
#include "lib/ilist.h"
#include "port/pg_bitutils.h"
+#include "utils/backend_status.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
#include "utils/memutils_memorychunk.h"
@@ -258,6 +259,7 @@ GenerationContextCreate(MemoryContext parent,
name);
((MemoryContext) set)->mem_allocated = firstBlockSize;
+ pgstat_report_backend_mem_allocated_increase(firstBlockSize);
return (MemoryContext) set;
}
@@ -274,6 +276,7 @@ GenerationReset(MemoryContext context)
{
GenerationContext *set = (GenerationContext *) context;
dlist_mutable_iter miter;
+ uint64 deallocation = 0;
AssertArg(GenerationIsValid(set));
@@ -296,9 +299,14 @@ GenerationReset(MemoryContext context)
if (block == set->keeper)
GenerationBlockMarkEmpty(block);
else
+ {
+ deallocation += block->blksize;
GenerationBlockFree(set, block);
+ }
}
+ pgstat_report_backend_mem_allocated_decrease(deallocation);
+
/* set it so new allocations to make use of the keeper block */
set->block = set->keeper;
@@ -319,6 +327,9 @@ GenerationDelete(MemoryContext context)
{
/* Reset to release all releasable GenerationBlocks */
GenerationReset(context);
+
+ pgstat_report_backend_mem_allocated_decrease(context->mem_allocated);
+
/* And free the context header and keeper block */
free(context);
}
@@ -363,6 +374,7 @@ GenerationAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_backend_mem_allocated_increase(blksize);
/* block with a single (used) chunk */
block->context = set;
@@ -466,6 +478,7 @@ GenerationAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_backend_mem_allocated_increase(blksize);
/* initialize the new block */
GenerationBlockInit(set, block, blksize);
@@ -699,6 +712,8 @@ GenerationFree(void *pointer)
dlist_delete(&block->node);
set->header.mem_allocated -= block->blksize;
+ pgstat_report_backend_mem_allocated_decrease(block->blksize);
+
free(block);
}
diff --git a/src/backend/utils/mmgr/slab.c b/src/backend/utils/mmgr/slab.c
index 9149aaafcb..72376da82e 100644
--- a/src/backend/utils/mmgr/slab.c
+++ b/src/backend/utils/mmgr/slab.c
@@ -53,6 +53,7 @@
#include "postgres.h"
#include "lib/ilist.h"
+#include "utils/backend_status.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
#include "utils/memutils_memorychunk.h"
@@ -223,6 +224,12 @@ SlabContextCreate(MemoryContext parent,
parent,
name);
+ /*
+ * If SlabContextCreate is updated to add headerSize to
+ * context->mem_allocated, then update here and SlabDelete appropriately
+ */
+ pgstat_report_backend_mem_allocated_increase(headerSize);
+
return (MemoryContext) slab;
}
@@ -238,6 +245,7 @@ SlabReset(MemoryContext context)
{
int i;
SlabContext *slab = castNode(SlabContext, context);
+ uint64 deallocation = 0;
Assert(slab);
@@ -263,9 +271,11 @@ SlabReset(MemoryContext context)
free(block);
slab->nblocks--;
context->mem_allocated -= slab->blockSize;
+ deallocation += slab->blockSize;
}
}
+ pgstat_report_backend_mem_allocated_decrease(deallocation);
slab->minFreeChunks = 0;
Assert(slab->nblocks == 0);
@@ -279,8 +289,17 @@ SlabReset(MemoryContext context)
void
SlabDelete(MemoryContext context)
{
+ /*
+ * Until header allocation is included in context->mem_allocated, cast to
+ * slab and decrement the headerSize
+ */
+ SlabContext *slab = castNode(SlabContext, context);
+
/* Reset to release all the SlabBlocks */
SlabReset(context);
+
+ pgstat_report_backend_mem_allocated_decrease(slab->headerSize);
+
/* And free the context header */
free(context);
}
@@ -349,6 +368,7 @@ SlabAlloc(MemoryContext context, Size size)
slab->minFreeChunks = slab->chunksPerBlock;
slab->nblocks += 1;
context->mem_allocated += slab->blockSize;
+ pgstat_report_backend_mem_allocated_increase(slab->blockSize);
}
/* grab the block from the freelist (even the new block is there) */
@@ -514,6 +534,7 @@ SlabFree(void *pointer)
free(block);
slab->nblocks--;
slab->header.mem_allocated -= slab->blockSize;
+ pgstat_report_backend_mem_allocated_decrease(slab->blockSize);
}
else
dlist_push_head(&slab->freelist[block->nfree], &block->node);
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index a07e737a33..363d92e9f2 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5340,9 +5340,9 @@
proname => 'pg_stat_get_activity', prorows => '100', proisstrict => 'f',
proretset => 't', provolatile => 's', proparallel => 'r',
prorettype => 'record', proargtypes => 'int4',
- proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,int4,int8}',
- proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,leader_pid,query_id}',
+ proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,int4,int8,int8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,leader_pid,query_id,backend_mem_allocated}',
prosrc => 'pg_stat_get_activity' },
{ oid => '3318',
descr => 'statistics: information about progress of backends running maintenance command',
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index 7403bca25e..9bdc4197bd 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -168,6 +168,9 @@ typedef struct PgBackendStatus
/* query identifier, optionally computed using post_parse_analyze_hook */
uint64 st_query_id;
+
+ /* Current memory allocated to this backend */
+ uint64 backend_mem_allocated;
} PgBackendStatus;
@@ -305,7 +308,9 @@ extern const char *pgstat_get_backend_current_activity(int pid, bool checkUser);
extern const char *pgstat_get_crashed_backend_activity(int pid, char *buffer,
int buflen);
extern uint64 pgstat_get_my_query_id(void);
-
+extern void pgstat_report_backend_mem_allocated_increase(uint64 allocation);
+extern void pgstat_report_backend_mem_allocated_decrease(uint64 deallocation);
+extern uint64 pgstat_get_all_backend_memory_allocated(void);
/* ----------
* Support functions for the SQL-callable functions to
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 9dd137415e..4588d71c8a 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1752,9 +1752,10 @@ pg_stat_activity| SELECT s.datid,
s.backend_xid,
s.backend_xmin,
s.query_id,
+ s.backend_mem_allocated,
s.query,
s.backend_type
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, backend_mem_allocated)
LEFT JOIN pg_database d ON ((s.datid = d.oid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_all_indexes| SELECT c.oid AS relid,
@@ -1866,7 +1867,7 @@ pg_stat_gssapi| SELECT s.pid,
s.gss_auth AS gss_authenticated,
s.gss_princ AS principal,
s.gss_enc AS encrypted
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, backend_mem_allocated)
WHERE (s.client_port IS NOT NULL);
pg_stat_progress_analyze| SELECT s.pid,
s.datid,
@@ -2047,7 +2048,7 @@ pg_stat_replication| SELECT s.pid,
w.sync_priority,
w.sync_state,
w.reply_time
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, backend_mem_allocated)
JOIN pg_stat_get_wal_senders() w(pid, state, sent_lsn, write_lsn, flush_lsn, replay_lsn, write_lag, flush_lag, replay_lag, sync_priority, sync_state, reply_time) ON ((s.pid = w.pid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_replication_slots| SELECT s.slot_name,
@@ -2081,7 +2082,7 @@ pg_stat_ssl| SELECT s.pid,
s.ssl_client_dn AS client_dn,
s.ssl_client_serial AS client_serial,
s.ssl_issuer_dn AS issuer_dn
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, backend_mem_allocated)
WHERE (s.client_port IS NOT NULL);
pg_stat_subscription| SELECT su.oid AS subid,
su.subname,
--
2.25.1
[text/x-patch] 0002-Add-the-ability-to-limit-the-amount-of-memory-that-c.patch (14.5K, ../../[email protected]/3-0002-Add-the-ability-to-limit-the-amount-of-memory-that-c.patch)
download | inline diff:
From bb8e1a2bbb9425eb7667a97dc49beebe8f9bf327 Mon Sep 17 00:00:00 2001
From: Reid Thompson <[email protected]>
Date: Sat, 4 Jun 2022 22:23:59 -0400
Subject: [PATCH 2/2] Add the ability to limit the amount of memory that can be
allocated to backends.
This builds on the work that adds backend memory allocated to pg_stat_activity.
Add GUC variable max_total_backend_memory.
Specifies a limit to the amount of memory (in MB) that may be allocated to
backends in total (i.e. this is not a per user or per backend limit). If unset,
or set to 0 it is disabled. It is intended as a resource to help avoid the OOM
killer on LINUX and manage resources in general. A backend request that would
push the total over the limit will be denied with an out of memory error causing
that backend's current query/transaction to fail. Due to the dynamic nature of
memory allocations, this limit is not exact. If within 1.5MB of the limit and
two backends request 1MB each at the same time both may be allocated, and exceed
the limit. Further requests will not be allocated until dropping below the
limit. Keep this in mind when setting this value. This limit does not affect
auxiliary backend processes. Backend memory allocations are displayed in the
pg_stat_activity view.
---
doc/src/sgml/config.sgml | 26 ++++
src/backend/storage/ipc/dsm_impl.c | 12 ++
src/backend/utils/activity/backend_status.c | 111 +++++++++++++++++-
src/backend/utils/misc/guc_tables.c | 11 ++
src/backend/utils/misc/postgresql.conf.sample | 3 +
src/backend/utils/mmgr/aset.c | 17 +++
src/backend/utils/mmgr/generation.c | 11 +-
src/backend/utils/mmgr/slab.c | 8 ++
src/include/utils/backend_status.h | 2 +
9 files changed, 199 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 700914684d..ce8e35daee 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -2079,6 +2079,32 @@ include_dir 'conf.d'
</listitem>
</varlistentry>
+ <varlistentry id="guc-max-total-backend-memory" xreflabel="max_total_backend_memory">
+ <term><varname>max_total_backend_memory</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>max_total_backend_memory</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies a limit to the amount of memory (MB) that may be allocated to
+ backends in total (i.e. this is not a per user or per backend limit).
+ If unset, or set to 0 it is disabled. A backend request that would
+ push the total over the limit will be denied with an out of memory
+ error causing that backend's current query/transaction to fail. Due to
+ the dynamic nature of memory allocations, this limit is not exact. If
+ within 1.5MB of the limit and two backends request 1MB each at the same
+ time both may be allocated, and exceed the limit. Further requests will
+ not be allocated until dropping below the limit. Keep this in mind when
+ setting this value. This limit does not affect auxiliary backend
+ processes <xref linkend="glossary-auxiliary-proc"/> . Backend memory
+ allocations (<varname>backend_mem_allocated</varname>) are displayed in
+ the <link linkend="monitoring-pg-stat-activity-view"><structname>pg_stat_activity</structname></link>
+ view.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/src/backend/storage/ipc/dsm_impl.c b/src/backend/storage/ipc/dsm_impl.c
index 3356bb65b5..cc061056a3 100644
--- a/src/backend/storage/ipc/dsm_impl.c
+++ b/src/backend/storage/ipc/dsm_impl.c
@@ -253,6 +253,10 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
return true;
}
+ /* Do not exceed maximum allowed memory allocation */
+ if (op == DSM_OP_CREATE && exceeds_max_total_bkend_mem(request_size))
+ return false;
+
/*
* Create new segment or open an existing one for attach.
*
@@ -524,6 +528,10 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size,
int flags = IPCProtection;
size_t segsize;
+ /* Do not exceed maximum allowed memory allocation */
+ if (op == DSM_OP_CREATE && exceeds_max_total_bkend_mem(request_size))
+ return false;
+
/*
* Allocate the memory BEFORE acquiring the resource, so that we don't
* leak the resource if memory allocation fails.
@@ -718,6 +726,10 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size,
return true;
}
+ /* Do not exceed maximum allowed memory allocation */
+ if (op == DSM_OP_CREATE && exceeds_max_total_bkend_mem(request_size))
+ return false;
+
/* Create new segment or open an existing one for attach. */
if (op == DSM_OP_CREATE)
{
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 45da3af213..7820c55489 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -45,6 +45,9 @@
bool pgstat_track_activities = false;
int pgstat_track_activity_query_size = 1024;
+/* Max backend memory allocation allowed (MB). 0 = disabled */
+int max_total_bkend_mem = 0;
+
/* exposed so that backend_progress.c can access it */
PgBackendStatus *MyBEEntry = NULL;
@@ -1235,7 +1238,7 @@ pgstat_report_backend_mem_allocated_decrease(uint64 deallocation)
*/
if (deallocation > beentry->backend_mem_allocated)
{
- ereport(LOG, errmsg("decrease reduces reported backend memory allocated below zero; setting reported to 0"));
+ ereport(LOG, errmsg("deallocation would decrease backend memory below zero; setting reported to 0"));
/*
* Overwrite deallocation with current backend_mem_allocated so we end
@@ -1253,3 +1256,109 @@ pgstat_report_backend_mem_allocated_decrease(uint64 deallocation)
beentry->backend_mem_allocated -= deallocation;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
+
+/* ----------
+ * pgstat_get_all_backend_memory_allocated() -
+ *
+ * Return a uint64 representing the current shared memory allocated to all
+ * backends. This looks directly at the BackendStatusArray, and so will
+ * provide current information regardless of the age of our transaction's
+ * snapshot of the status array.
+ * In the future we will likely utilize additional values - perhaps limit
+ * backend allocation by user/role, etc.
+ * ----------
+ */
+uint64
+pgstat_get_all_backend_memory_allocated(void)
+{
+ PgBackendStatus *beentry;
+ int i;
+ uint64 all_backend_memory_allocated = 0;
+
+ beentry = BackendStatusArray;
+
+ /*
+ * We probably shouldn't get here before shared memory has been set up,
+ * but be safe.
+ */
+ if (beentry == NULL || BackendActivityBuffer == NULL)
+ return 0;
+
+ /*
+ * We include AUX procs in all backend memory calculation
+ */
+ for (i = 1; i <= NumBackendStatSlots; i++)
+ {
+ /*
+ * We use a volatile pointer here to ensure the compiler doesn't try
+ * to get cute.
+ */
+ volatile PgBackendStatus *vbeentry = beentry;
+ bool found;
+ uint64 backend_mem_allocated = 0;
+
+ for (;;)
+ {
+ int before_changecount;
+ int after_changecount;
+
+ pgstat_begin_read_activity(vbeentry, before_changecount);
+
+ /*
+ * Ignore invalid entries, which may contain invalid data.
+ * See pgstat_beshutdown_hook()
+ */
+ if (vbeentry->st_procpid > 0)
+ backend_mem_allocated = vbeentry->backend_mem_allocated;
+
+ pgstat_end_read_activity(vbeentry, after_changecount);
+
+ if ((found = pgstat_read_activity_complete(before_changecount,
+ after_changecount)))
+ break;
+
+ /* Make sure we can break out of loop if stuck... */
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ if (found)
+ all_backend_memory_allocated += backend_mem_allocated;
+
+ beentry++;
+ }
+
+ return all_backend_memory_allocated;
+}
+
+/*
+ * Determine if allocation request will exceed max backend memory allowed.
+ * Do not apply to auxiliary processes.
+ */
+bool
+exceeds_max_total_bkend_mem(uint64 allocation_request)
+{
+ bool result = false;
+
+ /* Exclude auxiliary processes from the check */
+ if (MyAuxProcType != NotAnAuxProcess)
+ return result;
+
+ /* Convert max_total_bkend_mem to bytes for comparison */
+ if (max_total_bkend_mem &&
+ pgstat_get_all_backend_memory_allocated() +
+ allocation_request > (uint64) max_total_bkend_mem * 1024 * 1024)
+ {
+ /*
+ * Explicitly identify the OOM being a result of this configuration
+ * parameter vs a system failure to allocate OOM.
+ */
+ ereport(WARNING,
+ errmsg("allocation would exceed max_total_backend_memory limit (%llu > %llu)",
+ (unsigned long long) pgstat_get_all_backend_memory_allocated() +
+ allocation_request, (unsigned long long) max_total_bkend_mem * 1024 * 1024));
+
+ result = true;
+ }
+
+ return result;
+}
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 550e95056c..7d4dc8677a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -3415,6 +3415,17 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_total_backend_memory", PGC_SU_BACKEND, RESOURCES_MEM,
+ gettext_noop("Restrict total backend memory allocations to this max."),
+ gettext_noop("0 turns this feature off."),
+ GUC_UNIT_MB
+ },
+ &max_total_bkend_mem,
+ 0, 0, INT_MAX,
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, 0, 0, 0, NULL, NULL, NULL
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 2ae76e5cfb..8a0b383eb7 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -155,6 +155,9 @@
# mmap
# (change requires restart)
#min_dynamic_shared_memory = 0MB # (change requires restart)
+#max_total_backend_memory = 0MB # Restrict total backend memory allocations
+ # to this max (in MB). 0 turns this feature
+ # off.
# - Disk -
diff --git a/src/backend/utils/mmgr/aset.c b/src/backend/utils/mmgr/aset.c
index 2ac6f10e12..509b93faa1 100644
--- a/src/backend/utils/mmgr/aset.c
+++ b/src/backend/utils/mmgr/aset.c
@@ -428,6 +428,10 @@ AllocSetContextCreateInternal(MemoryContext parent,
else
firstBlockSize = Max(firstBlockSize, initBlockSize);
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(firstBlockSize))
+ return NULL;
+
/*
* Allocate the initial block. Unlike other aset.c blocks, it starts with
* the context header and its block header follows that.
@@ -726,6 +730,11 @@ AllocSetAlloc(MemoryContext context, Size size)
#endif
blksize = chunk_size + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ;
+
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
block = (AllocBlock) malloc(blksize);
if (block == NULL)
return NULL;
@@ -923,6 +932,10 @@ AllocSetAlloc(MemoryContext context, Size size)
while (blksize < required_size)
blksize <<= 1;
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
/* Try to allocate it */
block = (AllocBlock) malloc(blksize);
@@ -1150,6 +1163,10 @@ AllocSetRealloc(void *pointer, Size size)
blksize = chksize + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ;
oldblksize = block->endptr - ((char *) block);
+ /* Do not exceed maximum allowed memory allocation */
+ if (blksize > oldblksize && exceeds_max_total_bkend_mem(blksize - oldblksize))
+ return NULL;
+
block = (AllocBlock) realloc(block, blksize);
if (block == NULL)
{
diff --git a/src/backend/utils/mmgr/generation.c b/src/backend/utils/mmgr/generation.c
index 34b11392ff..6256e84d48 100644
--- a/src/backend/utils/mmgr/generation.c
+++ b/src/backend/utils/mmgr/generation.c
@@ -192,6 +192,9 @@ GenerationContextCreate(MemoryContext parent,
else
allocSize = Max(allocSize, initBlockSize);
+ if (exceeds_max_total_bkend_mem(allocSize))
+ return NULL;
+
/*
* Allocate the initial block. Unlike other generation.c blocks, it
* starts with the context header and its block header follows that.
@@ -276,7 +279,7 @@ GenerationReset(MemoryContext context)
{
GenerationContext *set = (GenerationContext *) context;
dlist_mutable_iter miter;
- uint64 deallocation = 0;
+ uint64 deallocation = 0;
AssertArg(GenerationIsValid(set));
@@ -369,6 +372,9 @@ GenerationAlloc(MemoryContext context, Size size)
{
Size blksize = required_size + Generation_BLOCKHDRSZ;
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
block = (GenerationBlock *) malloc(blksize);
if (block == NULL)
return NULL;
@@ -472,6 +478,9 @@ GenerationAlloc(MemoryContext context, Size size)
if (blksize < required_size)
blksize = pg_nextpower2_size_t(required_size);
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
block = (GenerationBlock *) malloc(blksize);
if (block == NULL)
diff --git a/src/backend/utils/mmgr/slab.c b/src/backend/utils/mmgr/slab.c
index 72376da82e..364c9eb795 100644
--- a/src/backend/utils/mmgr/slab.c
+++ b/src/backend/utils/mmgr/slab.c
@@ -182,6 +182,10 @@ SlabContextCreate(MemoryContext parent,
headerSize += chunksPerBlock * sizeof(bool);
#endif
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(headerSize))
+ return NULL;
+
slab = (SlabContext *) malloc(headerSize);
if (slab == NULL)
{
@@ -336,6 +340,10 @@ SlabAlloc(MemoryContext context, Size size)
*/
if (slab->minFreeChunks == 0)
{
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(slab->blockSize))
+ return NULL;
+
block = (SlabBlock *) malloc(slab->blockSize);
if (block == NULL)
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index 9bdc4197bd..3b940ff98e 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -270,6 +270,7 @@ typedef struct LocalPgBackendStatus
*/
extern PGDLLIMPORT bool pgstat_track_activities;
extern PGDLLIMPORT int pgstat_track_activity_query_size;
+extern PGDLLIMPORT int max_total_bkend_mem;
/* ----------
@@ -321,6 +322,7 @@ extern int pgstat_fetch_stat_numbackends(void);
extern PgBackendStatus *pgstat_fetch_stat_beentry(int beid);
extern LocalPgBackendStatus *pgstat_fetch_stat_local_beentry(int beid);
extern char *pgstat_clip_activity(const char *raw_activity);
+extern bool exceeds_max_total_bkend_mem(uint64 allocation_request);
#endif /* BACKEND_STATUS_H */
--
2.25.1
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: Add the ability to limit the amount of memory that can be allocated to backends.
2022-08-31 16:50 Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-08-31 17:34 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Justin Pryzby <[email protected]>
2022-09-04 03:40 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-09-09 17:14 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Justin Pryzby <[email protected]>
2022-09-12 16:25 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-09-15 08:07 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Ibrar Ahmed <[email protected]>
2022-09-15 14:58 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
@ 2022-10-24 15:27 ` Arne Roland <[email protected]>
2022-10-25 15:49 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
0 siblings, 1 reply; 22+ messages in thread
From: Arne Roland @ 2022-10-24 15:27 UTC (permalink / raw)
To: [email protected] <[email protected]>; [email protected] <[email protected]>; +Cc: Justin Pryzby <[email protected]>; Ibrar Ahmed <[email protected]>
Hello Reid,
could you rebase the patch again? It doesn't apply currently (http://cfbot.cputube.org/patch_40_3867.log). Thanks!
You mention, that you want to prevent the compiler from getting cute.
I don't think this comments are exactly helpful in the current state. I think probably fine to just omit them.
I don't understand the purpose of the result variable in exceeds_max_total_bkend_mem. What purpose does it serve?
I really like the simplicity of the suggestion here to prevent oom.
I intent to play around with a lot of backends, once I get a rebased patch.
Regards
Arne
________________________________
From: Reid Thompson <[email protected]>
Sent: Thursday, September 15, 2022 4:58:19 PM
To: Ibrar Ahmed; [email protected]
Cc: [email protected]; Justin Pryzby
Subject: Re: Add the ability to limit the amount of memory that can be allocated to backends.
On Thu, 2022-09-15 at 12:07 +0400, Ibrar Ahmed wrote:
>
> The patch does not apply; please rebase the patch.
>
> patching file src/backend/utils/misc/guc.c
> Hunk #1 FAILED at 3664.
> 1 out of 1 hunk FAILED -- saving rejects to file
> src/backend/utils/misc/guc.c.rej
>
> patching file src/backend/utils/misc/postgresql.conf.sample
>
rebased patches attached.
Thanks,
Reid
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: Add the ability to limit the amount of memory that can be allocated to backends.
2022-08-31 16:50 Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-08-31 17:34 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Justin Pryzby <[email protected]>
2022-09-04 03:40 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-09-09 17:14 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Justin Pryzby <[email protected]>
2022-09-12 16:25 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-09-15 08:07 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Ibrar Ahmed <[email protected]>
2022-09-15 14:58 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-10-24 15:27 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Arne Roland <[email protected]>
@ 2022-10-25 15:49 ` Reid Thompson <[email protected]>
2022-11-03 15:48 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
0 siblings, 1 reply; 22+ messages in thread
From: Reid Thompson @ 2022-10-25 15:49 UTC (permalink / raw)
To: Arne Roland <[email protected]>; [email protected] <[email protected]>; +Cc: [email protected]; Justin Pryzby <[email protected]>; Ibrar Ahmed <[email protected]>
Hi Arne,
On Mon, 2022-10-24 at 15:27 +0000, Arne Roland wrote:
> Hello Reid,
>
> could you rebase the patch again? It doesn't apply currently
> (http://cfbot.cputube.org/patch_40_3867.log). Thanks!
rebased patches attached.
> You mention, that you want to prevent the compiler from getting
> cute.I don't think this comments are exactly helpful in the current
> state. I think probably fine to just omit them.
I attempted to follow previous convention when adding code and these
comments have been consistently applied throughout backend_status.c
where a volatile pointer is being used.
> I don't understand the purpose of the result variable in
> exceeds_max_total_bkend_mem. What purpose does it serve?
>
> I really like the simplicity of the suggestion here to prevent oom.
If max_total_backend_memory is configured, exceeds_max_total_bkend_mem()
will return true if an allocation request will push total backend memory
allocated over the configured value.
exceeds_max_total_bkend_mem() is implemented in the various allocators
along the lines of
...snip...
/* Do not exceed maximum allowed memory allocation */
if (exceeds_max_total_bkend_mem('new request size'))
return NULL;
...snip...
Do not allocate the memory requested, return NULL instead. PG already
had code in place to handle NULL returns from allocation requests.
The allocation code in aset.c, slab.c, generation.c, dsm_impl.c utilizes
exceeds_max_total_bkend_mem()
max_total_backend_memory (integer)
Specifies a limit to the amount of memory (MB) that may be allocated
to backends in total (i.e. this is not a per user or per backend limit).
If unset, or set to 0 it is disabled. A backend request that would push
the total over the limit will be denied with an out of memory error
causing that backend's current query/transaction to fail. Due to the
dynamic nature of memory allocations, this limit is not exact. If within
1.5MB of the limit and two backends request 1MB each at the same time
both may be allocated, and exceed the limit. Further requests will not
be allocated until dropping below the limit. Keep this in mind when
setting this value. This limit does not affect auxiliary backend
processes Auxiliary process . Backend memory allocations
(backend_mem_allocated) are displayed in the pg_stat_activity view.
> I intent to play around with a lot of backends, once I get a rebased
> patch.
>
> Regards
> Arne
Attachments:
[text/x-patch] 0001-Add-tracking-of-backend-memory-allocated-to-pg_stat_.patch (28.5K, ../../[email protected]/2-0001-Add-tracking-of-backend-memory-allocated-to-pg_stat_.patch)
download | inline diff:
From ab654a48ec7bfbc3bc377c5757a04f1756e72e79 Mon Sep 17 00:00:00 2001
From: Reid Thompson <[email protected]>
Date: Thu, 11 Aug 2022 12:01:25 -0400
Subject: [PATCH 1/2] Add tracking of backend memory allocated to
pg_stat_activity
This new field displays the current bytes of memory allocated to the
backend process. It is updated as memory for the process is
malloc'd/free'd. Memory allocated to items on the freelist is included in
the displayed value. Dynamic shared memory allocations are included
only in the value displayed for the backend that created them, they are
not included in the value for backends that are attached to them to
avoid double counting. On occasion, orphaned memory segments may be
cleaned up on postmaster startup. This may result in decreasing the sum
without a prior increment. We limit the floor of backend_mem_allocated
to zero. Updated pg_stat_activity documentation for the new column.
---
doc/src/sgml/monitoring.sgml | 12 +++
src/backend/catalog/system_views.sql | 1 +
src/backend/storage/ipc/dsm_impl.c | 80 +++++++++++++++
src/backend/utils/activity/backend_status.c | 105 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 4 +-
src/backend/utils/mmgr/aset.c | 18 ++++
src/backend/utils/mmgr/generation.c | 15 +++
src/backend/utils/mmgr/slab.c | 21 ++++
src/include/catalog/pg_proc.dat | 6 +-
src/include/utils/backend_status.h | 7 +-
src/test/regress/expected/rules.out | 9 +-
11 files changed, 269 insertions(+), 9 deletions(-)
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index e5d622d514..4983bbc814 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -947,6 +947,18 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>backend_mem_allocated</structfield> <type>bigint</type>
+ </para>
+ <para>
+ The byte count of memory allocated to this backend. Dynamic shared memory
+ allocations are included only in the value displayed for the backend that
+ created them, they are not included in the value for backends that are
+ attached to them to avoid double counting.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>query</structfield> <type>text</type>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 2d8104b090..cbf804625c 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -865,6 +865,7 @@ CREATE VIEW pg_stat_activity AS
S.backend_xid,
s.backend_xmin,
S.query_id,
+ S.backend_mem_allocated,
S.query,
S.backend_type
FROM pg_stat_get_activity(NULL) AS S
diff --git a/src/backend/storage/ipc/dsm_impl.c b/src/backend/storage/ipc/dsm_impl.c
index e1b90c5de4..3356bb65b5 100644
--- a/src/backend/storage/ipc/dsm_impl.c
+++ b/src/backend/storage/ipc/dsm_impl.c
@@ -66,6 +66,7 @@
#include "postmaster/postmaster.h"
#include "storage/dsm_impl.h"
#include "storage/fd.h"
+#include "utils/backend_status.h"
#include "utils/guc.h"
#include "utils/memutils.h"
@@ -232,6 +233,13 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ if (op == DSM_OP_DESTROY)
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
*mapped_address = NULL;
*mapped_size = 0;
if (op == DSM_OP_DESTROY && shm_unlink(name) != 0)
@@ -332,6 +340,36 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ {
+ /*
+ * Posix creation calls dsm_impl_posix_resize implying that resizing
+ * occurs or may be added in the future. As implemented
+ * dsm_impl_posix_resize utilizes fallocate or truncate, passing the
+ * whole new size as input, growing the allocation as needed (only
+ * truncate supports shrinking). We update by replacing the old
+ * allocation with the new.
+ */
+#if defined(HAVE_POSIX_FALLOCATE) && defined(__linux__)
+ /*
+ * posix_fallocate does not shrink allocations, adjust only on
+ * allocation increase.
+ */
+ if (request_size > *mapped_size)
+ {
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
+ pgstat_report_backend_mem_allocated_increase(request_size);
+ }
+#else
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
+ pgstat_report_backend_mem_allocated_increase(request_size);
+#endif
+ }
*mapped_address = address;
*mapped_size = request_size;
close(fd);
@@ -537,6 +575,14 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ if (op == DSM_OP_DESTROY)
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
*mapped_address = NULL;
*mapped_size = 0;
if (op == DSM_OP_DESTROY && shmctl(ident, IPC_RMID, NULL) < 0)
@@ -584,6 +630,13 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ pgstat_report_backend_mem_allocated_increase(request_size);
*mapped_address = address;
*mapped_size = request_size;
@@ -652,6 +705,13 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size,
return false;
}
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ if (op == DSM_OP_DESTROY)
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
*impl_private = NULL;
*mapped_address = NULL;
*mapped_size = 0;
@@ -768,6 +828,12 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size,
return false;
}
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ pgstat_report_backend_mem_allocated_increase(info.RegionSize);
*mapped_address = address;
*mapped_size = info.RegionSize;
*impl_private = hmap;
@@ -812,6 +878,13 @@ dsm_impl_mmap(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ pgstat_report_backend_mem_allocated_decrease(*mapped_size);
*mapped_address = NULL;
*mapped_size = 0;
if (op == DSM_OP_DESTROY && unlink(name) != 0)
@@ -933,6 +1006,13 @@ dsm_impl_mmap(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ pgstat_report_backend_mem_allocated_increase(request_size);
*mapped_address = address;
*mapped_size = request_size;
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 1146a6c33c..5c33824eb8 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -49,6 +49,8 @@ int pgstat_track_activity_query_size = 1024;
/* exposed so that backend_progress.c can access it */
PgBackendStatus *MyBEEntry = NULL;
+/* Memory allocated to this backend prior to pgstats initialization */
+uint64 backend_mem_allocated = 0;
static PgBackendStatus *BackendStatusArray = NULL;
static char *BackendAppnameBuffer = NULL;
@@ -400,6 +402,13 @@ pgstat_bestart(void)
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = UINT64CONST(0);
+ /*
+ * Move sum of memory allocated prior to pgstats initialization to pgstats
+ * and zero the local variable.
+ */
+ lbeentry.backend_mem_allocated = backend_mem_allocated;
+ backend_mem_allocated = 0;
+
/*
* we don't zero st_progress_param here to save cycles; nobody should
* examine it until st_progress_command has been set to something other
@@ -1191,3 +1200,99 @@ pgstat_clip_activity(const char *raw_activity)
return activity;
}
+
+/* --------
+ * pgstat_report_backend_mem_allocated_increase() -
+ *
+ * Called to report increase in memory allocated for this backend
+ * --------
+ */
+void
+pgstat_report_backend_mem_allocated_increase(uint64 allocation)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+
+ if (!beentry || !pgstat_track_activities)
+ {
+ /*
+ * Account for memory before pgstats is initialized. This will be
+ * migrated to pgstats on initialization.
+ */
+ backend_mem_allocated += allocation;
+
+ return;
+ }
+
+ /*
+ * Update my status entry, following the protocol of bumping
+ * st_changecount before and after. We use a volatile pointer here to
+ * ensure the compiler doesn't try to get cute.
+ */
+ PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
+ beentry->backend_mem_allocated += allocation;
+ PGSTAT_END_WRITE_ACTIVITY(beentry);
+}
+
+/* --------
+ * pgstat_report_backend_mem_allocated_decrease() -
+ *
+ * Called to report decrease in memory allocated for this backend
+ * --------
+ */
+void
+pgstat_report_backend_mem_allocated_decrease(uint64 deallocation)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+
+ /*
+ * Cases may occur where shared memory from a previous postmaster
+ * invocation still exist. These are cleaned up at startup by
+ * dsm_cleanup_using_control_segment. Limit decreasing memory allocated to
+ * zero in case no corresponding prior increase exists or decrease has
+ * already been accounted for.
+ */
+
+ if (!beentry || !pgstat_track_activities)
+ {
+ /*
+ * Account for memory before pgstats is initialized. This will be
+ * migrated to pgstats on initialization. Do not allow
+ * backend_mem_allocated to go below zero. If pgstats has not been
+ * initialized, we are in startup and we set backend_mem_allocated to
+ * zero in cases where it would go negative and skip generating an
+ * ereport.
+ */
+ if (deallocation > backend_mem_allocated)
+ backend_mem_allocated = 0;
+ else
+ backend_mem_allocated -= deallocation;
+
+ return;
+ }
+
+ /*
+ * Do not allow backend_mem_allocated to go below zero. ereport if we
+ * would have. There's no need for a lock around the read here as it's
+ * being referenced from the same backend which means that there shouldn't
+ * be concurrent writes. We want to generate an ereport in these cases.
+ */
+ if (deallocation > beentry->backend_mem_allocated)
+ {
+ ereport(LOG, errmsg("decrease reduces reported backend memory allocated below zero; setting reported to 0"));
+
+ /*
+ * Overwrite deallocation with current backend_mem_allocated so we end
+ * up at zero.
+ */
+ deallocation = beentry->backend_mem_allocated;
+ }
+
+ /*
+ * Update my status entry, following the protocol of bumping
+ * st_changecount before and after. We use a volatile pointer here to
+ * ensure the compiler doesn't try to get cute.
+ */
+ PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
+ beentry->backend_mem_allocated -= deallocation;
+ PGSTAT_END_WRITE_ACTIVITY(beentry);
+}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 96bffc0f2a..692ed1df18 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -553,7 +553,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum
pg_stat_get_activity(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_ACTIVITY_COLS 30
+#define PG_STAT_GET_ACTIVITY_COLS 31
int num_backends = pgstat_fetch_stat_numbackends();
int curr_backend;
int pid = PG_ARGISNULL(0) ? -1 : PG_GETARG_INT32(0);
@@ -609,6 +609,8 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
else
nulls[16] = true;
+ values[30] = UInt64GetDatum(beentry->backend_mem_allocated);
+
/* Values only available to role member or pg_read_all_stats */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
diff --git a/src/backend/utils/mmgr/aset.c b/src/backend/utils/mmgr/aset.c
index db402e3a41..f6d1333f3d 100644
--- a/src/backend/utils/mmgr/aset.c
+++ b/src/backend/utils/mmgr/aset.c
@@ -47,6 +47,7 @@
#include "postgres.h"
#include "port/pg_bitutils.h"
+#include "utils/backend_status.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
#include "utils/memutils_memorychunk.h"
@@ -521,6 +522,7 @@ AllocSetContextCreateInternal(MemoryContext parent,
name);
((MemoryContext) set)->mem_allocated = firstBlockSize;
+ pgstat_report_backend_mem_allocated_increase(firstBlockSize);
return (MemoryContext) set;
}
@@ -543,6 +545,7 @@ AllocSetReset(MemoryContext context)
AllocSet set = (AllocSet) context;
AllocBlock block;
Size keepersize PG_USED_FOR_ASSERTS_ONLY;
+ uint64 deallocation = 0;
AssertArg(AllocSetIsValid(set));
@@ -585,6 +588,7 @@ AllocSetReset(MemoryContext context)
{
/* Normal case, release the block */
context->mem_allocated -= block->endptr - ((char *) block);
+ deallocation += block->endptr - ((char *) block);
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
@@ -595,6 +599,7 @@ AllocSetReset(MemoryContext context)
}
Assert(context->mem_allocated == keepersize);
+ pgstat_report_backend_mem_allocated_decrease(deallocation);
/* Reset block size allocation sequence, too */
set->nextBlockSize = set->initBlockSize;
@@ -613,6 +618,7 @@ AllocSetDelete(MemoryContext context)
AllocSet set = (AllocSet) context;
AllocBlock block = set->blocks;
Size keepersize PG_USED_FOR_ASSERTS_ONLY;
+ uint64 deallocation = 0;
AssertArg(AllocSetIsValid(set));
@@ -651,11 +657,13 @@ AllocSetDelete(MemoryContext context)
freelist->first_free = (AllocSetContext *) oldset->header.nextchild;
freelist->num_free--;
+ deallocation += oldset->header.mem_allocated;
/* All that remains is to free the header/initial block */
free(oldset);
}
Assert(freelist->num_free == 0);
+ pgstat_report_backend_mem_allocated_decrease(deallocation);
}
/* Now add the just-deleted context to the freelist. */
@@ -672,7 +680,10 @@ AllocSetDelete(MemoryContext context)
AllocBlock next = block->next;
if (block != set->keeper)
+ {
context->mem_allocated -= block->endptr - ((char *) block);
+ deallocation += block->endptr - ((char *) block);
+ }
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
@@ -685,6 +696,8 @@ AllocSetDelete(MemoryContext context)
}
Assert(context->mem_allocated == keepersize);
+ pgstat_report_backend_mem_allocated_decrease(deallocation +
+ context->mem_allocated);
/* Finally, free the context header, including the keeper block */
free(set);
@@ -734,6 +747,7 @@ AllocSetAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_backend_mem_allocated_increase(blksize);
block->aset = set;
block->freeptr = block->endptr = ((char *) block) + blksize;
@@ -944,6 +958,7 @@ AllocSetAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_backend_mem_allocated_increase(blksize);
block->aset = set;
block->freeptr = ((char *) block) + ALLOC_BLOCKHDRSZ;
@@ -1043,6 +1058,7 @@ AllocSetFree(void *pointer)
block->next->prev = block->prev;
set->header.mem_allocated -= block->endptr - ((char *) block);
+ pgstat_report_backend_mem_allocated_decrease(block->endptr - ((char *) block));
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
@@ -1173,7 +1189,9 @@ AllocSetRealloc(void *pointer, Size size)
/* updated separately, not to underflow when (oldblksize > blksize) */
set->header.mem_allocated -= oldblksize;
+ pgstat_report_backend_mem_allocated_decrease(oldblksize);
set->header.mem_allocated += blksize;
+ pgstat_report_backend_mem_allocated_increase(blksize);
block->freeptr = block->endptr = ((char *) block) + blksize;
diff --git a/src/backend/utils/mmgr/generation.c b/src/backend/utils/mmgr/generation.c
index 4cb75f493f..7bb9175f6d 100644
--- a/src/backend/utils/mmgr/generation.c
+++ b/src/backend/utils/mmgr/generation.c
@@ -37,6 +37,7 @@
#include "lib/ilist.h"
#include "port/pg_bitutils.h"
+#include "utils/backend_status.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
#include "utils/memutils_memorychunk.h"
@@ -267,6 +268,7 @@ GenerationContextCreate(MemoryContext parent,
name);
((MemoryContext) set)->mem_allocated = firstBlockSize;
+ pgstat_report_backend_mem_allocated_increase(firstBlockSize);
return (MemoryContext) set;
}
@@ -283,6 +285,7 @@ GenerationReset(MemoryContext context)
{
GenerationContext *set = (GenerationContext *) context;
dlist_mutable_iter miter;
+ uint64 deallocation = 0;
AssertArg(GenerationIsValid(set));
@@ -305,9 +308,14 @@ GenerationReset(MemoryContext context)
if (block == set->keeper)
GenerationBlockMarkEmpty(block);
else
+ {
+ deallocation += block->blksize;
GenerationBlockFree(set, block);
+ }
}
+ pgstat_report_backend_mem_allocated_decrease(deallocation);
+
/* set it so new allocations to make use of the keeper block */
set->block = set->keeper;
@@ -328,6 +336,9 @@ GenerationDelete(MemoryContext context)
{
/* Reset to release all releasable GenerationBlocks */
GenerationReset(context);
+
+ pgstat_report_backend_mem_allocated_decrease(context->mem_allocated);
+
/* And free the context header and keeper block */
free(context);
}
@@ -374,6 +385,7 @@ GenerationAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_backend_mem_allocated_increase(blksize);
/* block with a single (used) chunk */
block->context = set;
@@ -477,6 +489,7 @@ GenerationAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_backend_mem_allocated_increase(blksize);
/* initialize the new block */
GenerationBlockInit(set, block, blksize);
@@ -726,6 +739,8 @@ GenerationFree(void *pointer)
dlist_delete(&block->node);
set->header.mem_allocated -= block->blksize;
+ pgstat_report_backend_mem_allocated_decrease(block->blksize);
+
free(block);
}
diff --git a/src/backend/utils/mmgr/slab.c b/src/backend/utils/mmgr/slab.c
index 1a0b28f9ea..efdc2736c1 100644
--- a/src/backend/utils/mmgr/slab.c
+++ b/src/backend/utils/mmgr/slab.c
@@ -53,6 +53,7 @@
#include "postgres.h"
#include "lib/ilist.h"
+#include "utils/backend_status.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
#include "utils/memutils_memorychunk.h"
@@ -238,6 +239,12 @@ SlabContextCreate(MemoryContext parent,
parent,
name);
+ /*
+ * If SlabContextCreate is updated to add headerSize to
+ * context->mem_allocated, then update here and SlabDelete appropriately
+ */
+ pgstat_report_backend_mem_allocated_increase(headerSize);
+
return (MemoryContext) slab;
}
@@ -253,6 +260,7 @@ SlabReset(MemoryContext context)
{
SlabContext *slab = (SlabContext *) context;
int i;
+ uint64 deallocation = 0;
AssertArg(SlabIsValid(slab));
@@ -278,9 +286,11 @@ SlabReset(MemoryContext context)
free(block);
slab->nblocks--;
context->mem_allocated -= slab->blockSize;
+ deallocation += slab->blockSize;
}
}
+ pgstat_report_backend_mem_allocated_decrease(deallocation);
slab->minFreeChunks = 0;
Assert(slab->nblocks == 0);
@@ -294,8 +304,17 @@ SlabReset(MemoryContext context)
void
SlabDelete(MemoryContext context)
{
+ /*
+ * Until header allocation is included in context->mem_allocated, cast to
+ * slab and decrement the headerSize
+ */
+ SlabContext *slab = castNode(SlabContext, context);
+
/* Reset to release all the SlabBlocks */
SlabReset(context);
+
+ pgstat_report_backend_mem_allocated_decrease(slab->headerSize);
+
/* And free the context header */
free(context);
}
@@ -364,6 +383,7 @@ SlabAlloc(MemoryContext context, Size size)
slab->minFreeChunks = slab->chunksPerBlock;
slab->nblocks += 1;
context->mem_allocated += slab->blockSize;
+ pgstat_report_backend_mem_allocated_increase(slab->blockSize);
}
/* grab the block from the freelist (even the new block is there) */
@@ -537,6 +557,7 @@ SlabFree(void *pointer)
free(block);
slab->nblocks--;
slab->header.mem_allocated -= slab->blockSize;
+ pgstat_report_backend_mem_allocated_decrease(slab->blockSize);
}
else
dlist_push_head(&slab->freelist[block->nfree], &block->node);
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 62a5b8e655..737a7c5034 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5347,9 +5347,9 @@
proname => 'pg_stat_get_activity', prorows => '100', proisstrict => 'f',
proretset => 't', provolatile => 's', proparallel => 'r',
prorettype => 'record', proargtypes => 'int4',
- proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,int4,int8}',
- proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,leader_pid,query_id}',
+ proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,int4,int8,int8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,leader_pid,query_id,backend_mem_allocated}',
prosrc => 'pg_stat_get_activity' },
{ oid => '3318',
descr => 'statistics: information about progress of backends running maintenance command',
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index b582b46e9f..d59a1d50f6 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,9 @@ typedef struct PgBackendStatus
/* query identifier, optionally computed using post_parse_analyze_hook */
uint64 st_query_id;
+
+ /* Current memory allocated to this backend */
+ uint64 backend_mem_allocated;
} PgBackendStatus;
@@ -313,7 +316,9 @@ extern const char *pgstat_get_backend_current_activity(int pid, bool checkUser);
extern const char *pgstat_get_crashed_backend_activity(int pid, char *buffer,
int buflen);
extern uint64 pgstat_get_my_query_id(void);
-
+extern void pgstat_report_backend_mem_allocated_increase(uint64 allocation);
+extern void pgstat_report_backend_mem_allocated_decrease(uint64 deallocation);
+extern uint64 pgstat_get_all_backend_memory_allocated(void);
/* ----------
* Support functions for the SQL-callable functions to
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index bfcd8ac9a0..36947c636d 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1752,9 +1752,10 @@ pg_stat_activity| SELECT s.datid,
s.backend_xid,
s.backend_xmin,
s.query_id,
+ s.backend_mem_allocated,
s.query,
s.backend_type
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, backend_mem_allocated)
LEFT JOIN pg_database d ON ((s.datid = d.oid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_all_indexes| SELECT c.oid AS relid,
@@ -1869,7 +1870,7 @@ pg_stat_gssapi| SELECT s.pid,
s.gss_auth AS gss_authenticated,
s.gss_princ AS principal,
s.gss_enc AS encrypted
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, backend_mem_allocated)
WHERE (s.client_port IS NOT NULL);
pg_stat_progress_analyze| SELECT s.pid,
s.datid,
@@ -2050,7 +2051,7 @@ pg_stat_replication| SELECT s.pid,
w.sync_priority,
w.sync_state,
w.reply_time
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, backend_mem_allocated)
JOIN pg_stat_get_wal_senders() w(pid, state, sent_lsn, write_lsn, flush_lsn, replay_lsn, write_lag, flush_lag, replay_lag, sync_priority, sync_state, reply_time) ON ((s.pid = w.pid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_replication_slots| SELECT s.slot_name,
@@ -2084,7 +2085,7 @@ pg_stat_ssl| SELECT s.pid,
s.ssl_client_dn AS client_dn,
s.ssl_client_serial AS client_serial,
s.ssl_issuer_dn AS issuer_dn
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, backend_mem_allocated)
WHERE (s.client_port IS NOT NULL);
pg_stat_subscription| SELECT su.oid AS subid,
su.subname,
--
2.25.1
[text/x-patch] 0002-Add-the-ability-to-limit-the-amount-of-memory-that-c.patch (14.5K, ../../[email protected]/3-0002-Add-the-ability-to-limit-the-amount-of-memory-that-c.patch)
download | inline diff:
From 7c6735a34d22fde1a5103bd962d0a45f1322e87d Mon Sep 17 00:00:00 2001
From: Reid Thompson <[email protected]>
Date: Sat, 4 Jun 2022 22:23:59 -0400
Subject: [PATCH 2/2] Add the ability to limit the amount of memory that can be
allocated to backends.
This builds on the work that adds backend memory allocated to pg_stat_activity.
Add GUC variable max_total_backend_memory.
Specifies a limit to the amount of memory (in MB) that may be allocated to
backends in total (i.e. this is not a per user or per backend limit). If unset,
or set to 0 it is disabled. It is intended as a resource to help avoid the OOM
killer on LINUX and manage resources in general. A backend request that would
push the total over the limit will be denied with an out of memory error causing
that backend's current query/transaction to fail. Due to the dynamic nature of
memory allocations, this limit is not exact. If within 1.5MB of the limit and
two backends request 1MB each at the same time both may be allocated, and exceed
the limit. Further requests will not be allocated until dropping below the
limit. Keep this in mind when setting this value. This limit does not affect
auxiliary backend processes. Backend memory allocations are displayed in the
pg_stat_activity view.
---
doc/src/sgml/config.sgml | 26 ++++
src/backend/storage/ipc/dsm_impl.c | 12 ++
src/backend/utils/activity/backend_status.c | 111 +++++++++++++++++-
src/backend/utils/misc/guc_tables.c | 11 ++
src/backend/utils/misc/postgresql.conf.sample | 3 +
src/backend/utils/mmgr/aset.c | 17 +++
src/backend/utils/mmgr/generation.c | 11 +-
src/backend/utils/mmgr/slab.c | 8 ++
src/include/utils/backend_status.h | 2 +
9 files changed, 199 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 559eb898a9..4d22491a7e 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -2079,6 +2079,32 @@ include_dir 'conf.d'
</listitem>
</varlistentry>
+ <varlistentry id="guc-max-total-backend-memory" xreflabel="max_total_backend_memory">
+ <term><varname>max_total_backend_memory</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>max_total_backend_memory</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies a limit to the amount of memory (MB) that may be allocated to
+ backends in total (i.e. this is not a per user or per backend limit).
+ If unset, or set to 0 it is disabled. A backend request that would
+ push the total over the limit will be denied with an out of memory
+ error causing that backend's current query/transaction to fail. Due to
+ the dynamic nature of memory allocations, this limit is not exact. If
+ within 1.5MB of the limit and two backends request 1MB each at the same
+ time both may be allocated, and exceed the limit. Further requests will
+ not be allocated until dropping below the limit. Keep this in mind when
+ setting this value. This limit does not affect auxiliary backend
+ processes <xref linkend="glossary-auxiliary-proc"/> . Backend memory
+ allocations (<varname>backend_mem_allocated</varname>) are displayed in
+ the <link linkend="monitoring-pg-stat-activity-view"><structname>pg_stat_activity</structname></link>
+ view.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/src/backend/storage/ipc/dsm_impl.c b/src/backend/storage/ipc/dsm_impl.c
index 3356bb65b5..cc061056a3 100644
--- a/src/backend/storage/ipc/dsm_impl.c
+++ b/src/backend/storage/ipc/dsm_impl.c
@@ -253,6 +253,10 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
return true;
}
+ /* Do not exceed maximum allowed memory allocation */
+ if (op == DSM_OP_CREATE && exceeds_max_total_bkend_mem(request_size))
+ return false;
+
/*
* Create new segment or open an existing one for attach.
*
@@ -524,6 +528,10 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size,
int flags = IPCProtection;
size_t segsize;
+ /* Do not exceed maximum allowed memory allocation */
+ if (op == DSM_OP_CREATE && exceeds_max_total_bkend_mem(request_size))
+ return false;
+
/*
* Allocate the memory BEFORE acquiring the resource, so that we don't
* leak the resource if memory allocation fails.
@@ -718,6 +726,10 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size,
return true;
}
+ /* Do not exceed maximum allowed memory allocation */
+ if (op == DSM_OP_CREATE && exceeds_max_total_bkend_mem(request_size))
+ return false;
+
/* Create new segment or open an existing one for attach. */
if (op == DSM_OP_CREATE)
{
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 5c33824eb8..872fe66188 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -45,6 +45,9 @@
bool pgstat_track_activities = false;
int pgstat_track_activity_query_size = 1024;
+/* Max backend memory allocation allowed (MB). 0 = disabled */
+int max_total_bkend_mem = 0;
+
/* exposed so that backend_progress.c can access it */
PgBackendStatus *MyBEEntry = NULL;
@@ -1278,7 +1281,7 @@ pgstat_report_backend_mem_allocated_decrease(uint64 deallocation)
*/
if (deallocation > beentry->backend_mem_allocated)
{
- ereport(LOG, errmsg("decrease reduces reported backend memory allocated below zero; setting reported to 0"));
+ ereport(LOG, errmsg("deallocation would decrease backend memory below zero; setting reported to 0"));
/*
* Overwrite deallocation with current backend_mem_allocated so we end
@@ -1296,3 +1299,109 @@ pgstat_report_backend_mem_allocated_decrease(uint64 deallocation)
beentry->backend_mem_allocated -= deallocation;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
+
+/* ----------
+ * pgstat_get_all_backend_memory_allocated() -
+ *
+ * Return a uint64 representing the current shared memory allocated to all
+ * backends. This looks directly at the BackendStatusArray, and so will
+ * provide current information regardless of the age of our transaction's
+ * snapshot of the status array.
+ * In the future we will likely utilize additional values - perhaps limit
+ * backend allocation by user/role, etc.
+ * ----------
+ */
+uint64
+pgstat_get_all_backend_memory_allocated(void)
+{
+ PgBackendStatus *beentry;
+ int i;
+ uint64 all_backend_memory_allocated = 0;
+
+ beentry = BackendStatusArray;
+
+ /*
+ * We probably shouldn't get here before shared memory has been set up,
+ * but be safe.
+ */
+ if (beentry == NULL || BackendActivityBuffer == NULL)
+ return 0;
+
+ /*
+ * We include AUX procs in all backend memory calculation
+ */
+ for (i = 1; i <= NumBackendStatSlots; i++)
+ {
+ /*
+ * We use a volatile pointer here to ensure the compiler doesn't try
+ * to get cute.
+ */
+ volatile PgBackendStatus *vbeentry = beentry;
+ bool found;
+ uint64 backend_mem_allocated = 0;
+
+ for (;;)
+ {
+ int before_changecount;
+ int after_changecount;
+
+ pgstat_begin_read_activity(vbeentry, before_changecount);
+
+ /*
+ * Ignore invalid entries, which may contain invalid data.
+ * See pgstat_beshutdown_hook()
+ */
+ if (vbeentry->st_procpid > 0)
+ backend_mem_allocated = vbeentry->backend_mem_allocated;
+
+ pgstat_end_read_activity(vbeentry, after_changecount);
+
+ if ((found = pgstat_read_activity_complete(before_changecount,
+ after_changecount)))
+ break;
+
+ /* Make sure we can break out of loop if stuck... */
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ if (found)
+ all_backend_memory_allocated += backend_mem_allocated;
+
+ beentry++;
+ }
+
+ return all_backend_memory_allocated;
+}
+
+/*
+ * Determine if allocation request will exceed max backend memory allowed.
+ * Do not apply to auxiliary processes.
+ */
+bool
+exceeds_max_total_bkend_mem(uint64 allocation_request)
+{
+ bool result = false;
+
+ /* Exclude auxiliary processes from the check */
+ if (MyAuxProcType != NotAnAuxProcess)
+ return result;
+
+ /* Convert max_total_bkend_mem to bytes for comparison */
+ if (max_total_bkend_mem &&
+ pgstat_get_all_backend_memory_allocated() +
+ allocation_request > (uint64) max_total_bkend_mem * 1024 * 1024)
+ {
+ /*
+ * Explicitly identify the OOM being a result of this configuration
+ * parameter vs a system failure to allocate OOM.
+ */
+ ereport(WARNING,
+ errmsg("allocation would exceed max_total_backend_memory limit (%llu > %llu)",
+ (unsigned long long) pgstat_get_all_backend_memory_allocated() +
+ allocation_request, (unsigned long long) max_total_bkend_mem * 1024 * 1024));
+
+ result = true;
+ }
+
+ return result;
+}
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 05ab087934..745cc2ca9c 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -3405,6 +3405,17 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_total_backend_memory", PGC_SU_BACKEND, RESOURCES_MEM,
+ gettext_noop("Restrict total backend memory allocations to this max."),
+ gettext_noop("0 turns this feature off."),
+ GUC_UNIT_MB
+ },
+ &max_total_bkend_mem,
+ 0, 0, INT_MAX,
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, 0, 0, 0, NULL, NULL, NULL
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 868d21c351..1ce0dee6d0 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -155,6 +155,9 @@
# mmap
# (change requires restart)
#min_dynamic_shared_memory = 0MB # (change requires restart)
+#max_total_backend_memory = 0MB # Restrict total backend memory allocations
+ # to this max (in MB). 0 turns this feature
+ # off.
# - Disk -
diff --git a/src/backend/utils/mmgr/aset.c b/src/backend/utils/mmgr/aset.c
index f6d1333f3d..5c659dc0fc 100644
--- a/src/backend/utils/mmgr/aset.c
+++ b/src/backend/utils/mmgr/aset.c
@@ -440,6 +440,10 @@ AllocSetContextCreateInternal(MemoryContext parent,
else
firstBlockSize = Max(firstBlockSize, initBlockSize);
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(firstBlockSize))
+ return NULL;
+
/*
* Allocate the initial block. Unlike other aset.c blocks, it starts with
* the context header and its block header follows that.
@@ -742,6 +746,11 @@ AllocSetAlloc(MemoryContext context, Size size)
#endif
blksize = chunk_size + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ;
+
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
block = (AllocBlock) malloc(blksize);
if (block == NULL)
return NULL;
@@ -939,6 +948,10 @@ AllocSetAlloc(MemoryContext context, Size size)
while (blksize < required_size)
blksize <<= 1;
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
/* Try to allocate it */
block = (AllocBlock) malloc(blksize);
@@ -1179,6 +1192,10 @@ AllocSetRealloc(void *pointer, Size size)
blksize = chksize + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ;
oldblksize = block->endptr - ((char *) block);
+ /* Do not exceed maximum allowed memory allocation */
+ if (blksize > oldblksize && exceeds_max_total_bkend_mem(blksize - oldblksize))
+ return NULL;
+
block = (AllocBlock) realloc(block, blksize);
if (block == NULL)
{
diff --git a/src/backend/utils/mmgr/generation.c b/src/backend/utils/mmgr/generation.c
index 7bb9175f6d..ef2e11aefc 100644
--- a/src/backend/utils/mmgr/generation.c
+++ b/src/backend/utils/mmgr/generation.c
@@ -201,6 +201,9 @@ GenerationContextCreate(MemoryContext parent,
else
allocSize = Max(allocSize, initBlockSize);
+ if (exceeds_max_total_bkend_mem(allocSize))
+ return NULL;
+
/*
* Allocate the initial block. Unlike other generation.c blocks, it
* starts with the context header and its block header follows that.
@@ -285,7 +288,7 @@ GenerationReset(MemoryContext context)
{
GenerationContext *set = (GenerationContext *) context;
dlist_mutable_iter miter;
- uint64 deallocation = 0;
+ uint64 deallocation = 0;
AssertArg(GenerationIsValid(set));
@@ -380,6 +383,9 @@ GenerationAlloc(MemoryContext context, Size size)
{
Size blksize = required_size + Generation_BLOCKHDRSZ;
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
block = (GenerationBlock *) malloc(blksize);
if (block == NULL)
return NULL;
@@ -483,6 +489,9 @@ GenerationAlloc(MemoryContext context, Size size)
if (blksize < required_size)
blksize = pg_nextpower2_size_t(required_size);
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
block = (GenerationBlock *) malloc(blksize);
if (block == NULL)
diff --git a/src/backend/utils/mmgr/slab.c b/src/backend/utils/mmgr/slab.c
index efdc2736c1..6ec512defd 100644
--- a/src/backend/utils/mmgr/slab.c
+++ b/src/backend/utils/mmgr/slab.c
@@ -197,6 +197,10 @@ SlabContextCreate(MemoryContext parent,
headerSize += chunksPerBlock * sizeof(bool);
#endif
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(headerSize))
+ return NULL;
+
slab = (SlabContext *) malloc(headerSize);
if (slab == NULL)
{
@@ -351,6 +355,10 @@ SlabAlloc(MemoryContext context, Size size)
*/
if (slab->minFreeChunks == 0)
{
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(slab->blockSize))
+ return NULL;
+
block = (SlabBlock *) malloc(slab->blockSize);
if (block == NULL)
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index d59a1d50f6..0403a24eef 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -278,6 +278,7 @@ typedef struct LocalPgBackendStatus
*/
extern PGDLLIMPORT bool pgstat_track_activities;
extern PGDLLIMPORT int pgstat_track_activity_query_size;
+extern PGDLLIMPORT int max_total_bkend_mem;
/* ----------
@@ -329,6 +330,7 @@ extern int pgstat_fetch_stat_numbackends(void);
extern PgBackendStatus *pgstat_fetch_stat_beentry(BackendId beid);
extern LocalPgBackendStatus *pgstat_fetch_stat_local_beentry(int beid);
extern char *pgstat_clip_activity(const char *raw_activity);
+extern bool exceeds_max_total_bkend_mem(uint64 allocation_request);
#endif /* BACKEND_STATUS_H */
--
2.25.1
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: Add the ability to limit the amount of memory that can be allocated to backends.
2022-08-31 16:50 Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-08-31 17:34 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Justin Pryzby <[email protected]>
2022-09-04 03:40 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-09-09 17:14 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Justin Pryzby <[email protected]>
2022-09-12 16:25 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-09-15 08:07 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Ibrar Ahmed <[email protected]>
2022-09-15 14:58 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-10-24 15:27 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Arne Roland <[email protected]>
2022-10-25 15:49 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
@ 2022-11-03 15:48 ` Reid Thompson <[email protected]>
2022-11-27 03:22 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
0 siblings, 1 reply; 22+ messages in thread
From: Reid Thompson @ 2022-11-03 15:48 UTC (permalink / raw)
To: Arne Roland <[email protected]>; [email protected] <[email protected]>; +Cc: [email protected]; Justin Pryzby <[email protected]>; Ibrar Ahmed <[email protected]>
On Tue, 2022-10-25 at 11:49 -0400, Reid Thompson wrote:
> Hi Arne,
>
> On Mon, 2022-10-24 at 15:27 +0000, Arne Roland wrote:
> > Hello Reid,
> >
> > could you rebase the patch again? It doesn't apply currently
> > (http://cfbot.cputube.org/patch_40_3867.log). Thanks!
>
> rebased patches attached.
Rebased to current. Add a couple changes per conversation with D
Christensen (include units in field name, group field with backend_xid
and backend_xmin fields in pg_stat_activity view, rather than between
query_id and query)
--
Reid Thompson
Senior Software Engineer
Crunchy Data, Inc.
[email protected]
www.crunchydata.com
Attachments:
[text/x-patch] 0002-Add-the-ability-to-limit-the-amount-of-memory-that-c.patch (14.3K, ../../[email protected]/2-0002-Add-the-ability-to-limit-the-amount-of-memory-that-c.patch)
download | inline diff:
From 9cf35c79be107feedb63f6f674ac9d2347d1875e Mon Sep 17 00:00:00 2001
From: Reid Thompson <[email protected]>
Date: Sat, 4 Jun 2022 22:23:59 -0400
Subject: [PATCH 2/2] Add the ability to limit the amount of memory that can be
allocated to backends.
This builds on the work that adds backend memory allocated to pg_stat_activity.
Add GUC variable max_total_backend_memory.
Specifies a limit to the amount of memory (in MB) that may be allocated to
backends in total (i.e. this is not a per user or per backend limit). If unset,
or set to 0 it is disabled. It is intended as a resource to help avoid the OOM
killer on LINUX and manage resources in general. A backend request that would
push the total over the limit will be denied with an out of memory error causing
that backend's current query/transaction to fail. Due to the dynamic nature of
memory allocations, this limit is not exact. If within 1.5MB of the limit and
two backends request 1MB each at the same time both may be allocated, and exceed
the limit. Further requests will not be allocated until dropping below the
limit. Keep this in mind when setting this value. This limit does not affect
auxiliary backend processes. Backend memory allocations are displayed in the
pg_stat_activity view.
---
doc/src/sgml/config.sgml | 26 ++++
src/backend/storage/ipc/dsm_impl.c | 12 ++
src/backend/utils/activity/backend_status.c | 111 +++++++++++++++++-
src/backend/utils/misc/guc_tables.c | 11 ++
src/backend/utils/misc/postgresql.conf.sample | 3 +
src/backend/utils/mmgr/aset.c | 17 +++
src/backend/utils/mmgr/generation.c | 9 ++
src/backend/utils/mmgr/slab.c | 8 ++
src/include/utils/backend_status.h | 2 +
9 files changed, 198 insertions(+), 1 deletion(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 559eb898a9..5762999fa5 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -2079,6 +2079,32 @@ include_dir 'conf.d'
</listitem>
</varlistentry>
+ <varlistentry id="guc-max-total-backend-memory" xreflabel="max_total_backend_memory">
+ <term><varname>max_total_backend_memory</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>max_total_backend_memory</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies a limit to the amount of memory (MB) that may be allocated to
+ backends in total (i.e. this is not a per user or per backend limit).
+ If unset, or set to 0 it is disabled. A backend request that would
+ push the total over the limit will be denied with an out of memory
+ error causing that backend's current query/transaction to fail. Due to
+ the dynamic nature of memory allocations, this limit is not exact. If
+ within 1.5MB of the limit and two backends request 1MB each at the same
+ time both may be allocated, and exceed the limit. Further requests will
+ not be allocated until dropping below the limit. Keep this in mind when
+ setting this value. This limit does not affect auxiliary backend
+ processes <xref linkend="glossary-auxiliary-proc"/> . Backend memory
+ allocations (<varname>backend_allocated_bytes</varname>) are displayed in
+ the <link linkend="monitoring-pg-stat-activity-view"><structname>pg_stat_activity</structname></link>
+ view.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/src/backend/storage/ipc/dsm_impl.c b/src/backend/storage/ipc/dsm_impl.c
index eae03159c3..aaf74e9486 100644
--- a/src/backend/storage/ipc/dsm_impl.c
+++ b/src/backend/storage/ipc/dsm_impl.c
@@ -254,6 +254,10 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
return true;
}
+ /* Do not exceed maximum allowed memory allocation */
+ if (op == DSM_OP_CREATE && exceeds_max_total_bkend_mem(request_size))
+ return false;
+
/*
* Create new segment or open an existing one for attach.
*
@@ -525,6 +529,10 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size,
int flags = IPCProtection;
size_t segsize;
+ /* Do not exceed maximum allowed memory allocation */
+ if (op == DSM_OP_CREATE && exceeds_max_total_bkend_mem(request_size))
+ return false;
+
/*
* Allocate the memory BEFORE acquiring the resource, so that we don't
* leak the resource if memory allocation fails.
@@ -719,6 +727,10 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size,
return true;
}
+ /* Do not exceed maximum allowed memory allocation */
+ if (op == DSM_OP_CREATE && exceeds_max_total_bkend_mem(request_size))
+ return false;
+
/* Create new segment or open an existing one for attach. */
if (op == DSM_OP_CREATE)
{
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 30a89e899a..5500ed4f37 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -45,6 +45,9 @@
bool pgstat_track_activities = false;
int pgstat_track_activity_query_size = 1024;
+/* Max backend memory allocation allowed (MB). 0 = disabled */
+int max_total_bkend_mem = 0;
+
/* exposed so that backend_progress.c can access it */
PgBackendStatus *MyBEEntry = NULL;
@@ -1278,7 +1281,7 @@ pgstat_report_backend_allocated_bytes_decrease(uint64 deallocation)
*/
if (deallocation > beentry->backend_allocated_bytes)
{
- ereport(LOG, errmsg("decrease reduces reported backend memory allocated below zero; setting reported to 0"));
+ ereport(LOG, errmsg("deallocation would decrease backend memory below zero; setting reported to 0"));
/*
* Overwrite deallocation with current backend_allocated_bytes so we
@@ -1296,3 +1299,109 @@ pgstat_report_backend_allocated_bytes_decrease(uint64 deallocation)
beentry->backend_allocated_bytes -= deallocation;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
+
+/* ----------
+ * pgstat_get_all_backend_memory_allocated() -
+ *
+ * Return a uint64 representing the current shared memory allocated to all
+ * backends. This looks directly at the BackendStatusArray, and so will
+ * provide current information regardless of the age of our transaction's
+ * snapshot of the status array.
+ * In the future we will likely utilize additional values - perhaps limit
+ * backend allocation by user/role, etc.
+ * ----------
+ */
+uint64
+pgstat_get_all_backend_memory_allocated(void)
+{
+ PgBackendStatus *beentry;
+ int i;
+ uint64 all_backend_memory_allocated = 0;
+
+ beentry = BackendStatusArray;
+
+ /*
+ * We probably shouldn't get here before shared memory has been set up,
+ * but be safe.
+ */
+ if (beentry == NULL || BackendActivityBuffer == NULL)
+ return 0;
+
+ /*
+ * We include AUX procs in all backend memory calculation
+ */
+ for (i = 1; i <= NumBackendStatSlots; i++)
+ {
+ /*
+ * We use a volatile pointer here to ensure the compiler doesn't try
+ * to get cute.
+ */
+ volatile PgBackendStatus *vbeentry = beentry;
+ bool found;
+ uint64 backend_allocated_bytes = 0;
+
+ for (;;)
+ {
+ int before_changecount;
+ int after_changecount;
+
+ pgstat_begin_read_activity(vbeentry, before_changecount);
+
+ /*
+ * Ignore invalid entries, which may contain invalid data.
+ * See pgstat_beshutdown_hook()
+ */
+ if (vbeentry->st_procpid > 0)
+ backend_allocated_bytes = vbeentry->backend_allocated_bytes;
+
+ pgstat_end_read_activity(vbeentry, after_changecount);
+
+ if ((found = pgstat_read_activity_complete(before_changecount,
+ after_changecount)))
+ break;
+
+ /* Make sure we can break out of loop if stuck... */
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ if (found)
+ all_backend_memory_allocated += backend_allocated_bytes;
+
+ beentry++;
+ }
+
+ return all_backend_memory_allocated;
+}
+
+/*
+ * Determine if allocation request will exceed max backend memory allowed.
+ * Do not apply to auxiliary processes.
+ */
+bool
+exceeds_max_total_bkend_mem(uint64 allocation_request)
+{
+ bool result = false;
+
+ /* Exclude auxiliary processes from the check */
+ if (MyAuxProcType != NotAnAuxProcess)
+ return result;
+
+ /* Convert max_total_bkend_mem to bytes for comparison */
+ if (max_total_bkend_mem &&
+ pgstat_get_all_backend_memory_allocated() +
+ allocation_request > (uint64) max_total_bkend_mem * 1024 * 1024)
+ {
+ /*
+ * Explicitly identify the OOM being a result of this configuration
+ * parameter vs a system failure to allocate OOM.
+ */
+ ereport(WARNING,
+ errmsg("allocation would exceed max_total_backend_memory limit (%llu > %llu)",
+ (unsigned long long) pgstat_get_all_backend_memory_allocated() +
+ allocation_request, (unsigned long long) max_total_bkend_mem * 1024 * 1024));
+
+ result = true;
+ }
+
+ return result;
+}
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 836b49484a..0e09766949 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -3403,6 +3403,17 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_total_backend_memory", PGC_SU_BACKEND, RESOURCES_MEM,
+ gettext_noop("Restrict total backend memory allocations to this max."),
+ gettext_noop("0 turns this feature off."),
+ GUC_UNIT_MB
+ },
+ &max_total_bkend_mem,
+ 0, 0, INT_MAX,
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, 0, 0, 0, NULL, NULL, NULL
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 868d21c351..1ce0dee6d0 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -155,6 +155,9 @@
# mmap
# (change requires restart)
#min_dynamic_shared_memory = 0MB # (change requires restart)
+#max_total_backend_memory = 0MB # Restrict total backend memory allocations
+ # to this max (in MB). 0 turns this feature
+ # off.
# - Disk -
diff --git a/src/backend/utils/mmgr/aset.c b/src/backend/utils/mmgr/aset.c
index 81e62e4981..cc865a89e0 100644
--- a/src/backend/utils/mmgr/aset.c
+++ b/src/backend/utils/mmgr/aset.c
@@ -440,6 +440,10 @@ AllocSetContextCreateInternal(MemoryContext parent,
else
firstBlockSize = Max(firstBlockSize, initBlockSize);
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(firstBlockSize))
+ return NULL;
+
/*
* Allocate the initial block. Unlike other aset.c blocks, it starts with
* the context header and its block header follows that.
@@ -742,6 +746,11 @@ AllocSetAlloc(MemoryContext context, Size size)
#endif
blksize = chunk_size + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ;
+
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
block = (AllocBlock) malloc(blksize);
if (block == NULL)
return NULL;
@@ -939,6 +948,10 @@ AllocSetAlloc(MemoryContext context, Size size)
while (blksize < required_size)
blksize <<= 1;
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
/* Try to allocate it */
block = (AllocBlock) malloc(blksize);
@@ -1179,6 +1192,10 @@ AllocSetRealloc(void *pointer, Size size)
blksize = chksize + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ;
oldblksize = block->endptr - ((char *) block);
+ /* Do not exceed maximum allowed memory allocation */
+ if (blksize > oldblksize && exceeds_max_total_bkend_mem(blksize - oldblksize))
+ return NULL;
+
block = (AllocBlock) realloc(block, blksize);
if (block == NULL)
{
diff --git a/src/backend/utils/mmgr/generation.c b/src/backend/utils/mmgr/generation.c
index df3007edfb..495665d3b0 100644
--- a/src/backend/utils/mmgr/generation.c
+++ b/src/backend/utils/mmgr/generation.c
@@ -201,6 +201,9 @@ GenerationContextCreate(MemoryContext parent,
else
allocSize = Max(allocSize, initBlockSize);
+ if (exceeds_max_total_bkend_mem(allocSize))
+ return NULL;
+
/*
* Allocate the initial block. Unlike other generation.c blocks, it
* starts with the context header and its block header follows that.
@@ -380,6 +383,9 @@ GenerationAlloc(MemoryContext context, Size size)
{
Size blksize = required_size + Generation_BLOCKHDRSZ;
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
block = (GenerationBlock *) malloc(blksize);
if (block == NULL)
return NULL;
@@ -483,6 +489,9 @@ GenerationAlloc(MemoryContext context, Size size)
if (blksize < required_size)
blksize = pg_nextpower2_size_t(required_size);
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
block = (GenerationBlock *) malloc(blksize);
if (block == NULL)
diff --git a/src/backend/utils/mmgr/slab.c b/src/backend/utils/mmgr/slab.c
index 532c038973..5b98176654 100644
--- a/src/backend/utils/mmgr/slab.c
+++ b/src/backend/utils/mmgr/slab.c
@@ -197,6 +197,10 @@ SlabContextCreate(MemoryContext parent,
headerSize += chunksPerBlock * sizeof(bool);
#endif
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(headerSize))
+ return NULL;
+
slab = (SlabContext *) malloc(headerSize);
if (slab == NULL)
{
@@ -351,6 +355,10 @@ SlabAlloc(MemoryContext context, Size size)
*/
if (slab->minFreeChunks == 0)
{
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(slab->blockSize))
+ return NULL;
+
block = (SlabBlock *) malloc(slab->blockSize);
if (block == NULL)
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index 75d87e8308..c8beb116b8 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -278,6 +278,7 @@ typedef struct LocalPgBackendStatus
*/
extern PGDLLIMPORT bool pgstat_track_activities;
extern PGDLLIMPORT int pgstat_track_activity_query_size;
+extern PGDLLIMPORT int max_total_bkend_mem;
/* ----------
@@ -329,6 +330,7 @@ extern int pgstat_fetch_stat_numbackends(void);
extern PgBackendStatus *pgstat_fetch_stat_beentry(BackendId beid);
extern LocalPgBackendStatus *pgstat_fetch_stat_local_beentry(int beid);
extern char *pgstat_clip_activity(const char *raw_activity);
+extern bool exceeds_max_total_bkend_mem(uint64 allocation_request);
#endif /* BACKEND_STATUS_H */
--
2.25.1
[text/x-patch] 0001-Add-tracking-of-backend-memory-allocated-to-pg_stat_.patch (28.6K, ../../[email protected]/3-0001-Add-tracking-of-backend-memory-allocated-to-pg_stat_.patch)
download | inline diff:
From 8f729c59c3aa1a02d008795159a748e1592a9916 Mon Sep 17 00:00:00 2001
From: Reid Thompson <[email protected]>
Date: Thu, 11 Aug 2022 12:01:25 -0400
Subject: [PATCH 1/2] Add tracking of backend memory allocated to
pg_stat_activity
This new field displays the current bytes of memory allocated to the
backend process. It is updated as memory for the process is
malloc'd/free'd. Memory allocated to items on the freelist is included in
the displayed value. Dynamic shared memory allocations are included
only in the value displayed for the backend that created them, they are
not included in the value for backends that are attached to them to
avoid double counting. On occasion, orphaned memory segments may be
cleaned up on postmaster startup. This may result in decreasing the sum
without a prior increment. We limit the floor of backend_mem_allocated
to zero. Updated pg_stat_activity documentation for the new column.
---
doc/src/sgml/monitoring.sgml | 12 +++
src/backend/catalog/system_views.sql | 1 +
src/backend/storage/ipc/dsm_impl.c | 81 +++++++++++++++
src/backend/utils/activity/backend_status.c | 105 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 4 +-
src/backend/utils/mmgr/aset.c | 18 ++++
src/backend/utils/mmgr/generation.c | 15 +++
src/backend/utils/mmgr/slab.c | 21 ++++
src/include/catalog/pg_proc.dat | 6 +-
src/include/utils/backend_status.h | 7 +-
src/test/regress/expected/rules.out | 9 +-
11 files changed, 270 insertions(+), 9 deletions(-)
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index e5d622d514..972805b85a 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -947,6 +947,18 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>backend_allocated_bytes</structfield> <type>bigint</type>
+ </para>
+ <para>
+ The byte count of memory allocated to this backend. Dynamic shared memory
+ allocations are included only in the value displayed for the backend that
+ created them, they are not included in the value for backends that are
+ attached to them to avoid double counting.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>query</structfield> <type>text</type>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 2d8104b090..84d462aa97 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -864,6 +864,7 @@ CREATE VIEW pg_stat_activity AS
S.state,
S.backend_xid,
s.backend_xmin,
+ S.backend_allocated_bytes,
S.query_id,
S.query,
S.backend_type
diff --git a/src/backend/storage/ipc/dsm_impl.c b/src/backend/storage/ipc/dsm_impl.c
index 6ddd46a4e7..eae03159c3 100644
--- a/src/backend/storage/ipc/dsm_impl.c
+++ b/src/backend/storage/ipc/dsm_impl.c
@@ -66,6 +66,7 @@
#include "postmaster/postmaster.h"
#include "storage/dsm_impl.h"
#include "storage/fd.h"
+#include "utils/backend_status.h"
#include "utils/guc.h"
#include "utils/memutils.h"
@@ -232,6 +233,14 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ if (op == DSM_OP_DESTROY)
+ pgstat_report_backend_allocated_bytes_decrease(*mapped_size);
*mapped_address = NULL;
*mapped_size = 0;
if (op == DSM_OP_DESTROY && shm_unlink(name) != 0)
@@ -332,6 +341,36 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ {
+ /*
+ * Posix creation calls dsm_impl_posix_resize implying that resizing
+ * occurs or may be added in the future. As implemented
+ * dsm_impl_posix_resize utilizes fallocate or truncate, passing the
+ * whole new size as input, growing the allocation as needed (only
+ * truncate supports shrinking). We update by replacing the old
+ * allocation with the new.
+ */
+#if defined(HAVE_POSIX_FALLOCATE) && defined(__linux__)
+ /*
+ * posix_fallocate does not shrink allocations, adjust only on
+ * allocation increase.
+ */
+ if (request_size > *mapped_size)
+ {
+ pgstat_report_backend_allocated_bytes_decrease(*mapped_size);
+ pgstat_report_backend_allocated_bytes_increase(request_size);
+ }
+#else
+ pgstat_report_backend_allocated_bytes_decrease(*mapped_size);
+ pgstat_report_backend_allocated_bytes_increase(request_size);
+#endif
+ }
*mapped_address = address;
*mapped_size = request_size;
close(fd);
@@ -537,6 +576,14 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ if (op == DSM_OP_DESTROY)
+ pgstat_report_backend_allocated_bytes_decrease(*mapped_size);
*mapped_address = NULL;
*mapped_size = 0;
if (op == DSM_OP_DESTROY && shmctl(ident, IPC_RMID, NULL) < 0)
@@ -584,6 +631,13 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ pgstat_report_backend_allocated_bytes_increase(request_size);
*mapped_address = address;
*mapped_size = request_size;
@@ -652,6 +706,13 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size,
return false;
}
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ if (op == DSM_OP_DESTROY)
+ pgstat_report_backend_allocated_bytes_decrease(*mapped_size);
*impl_private = NULL;
*mapped_address = NULL;
*mapped_size = 0;
@@ -768,6 +829,12 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size,
return false;
}
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ pgstat_report_backend_allocated_bytes_increase(info.RegionSize);
*mapped_address = address;
*mapped_size = info.RegionSize;
*impl_private = hmap;
@@ -812,6 +879,13 @@ dsm_impl_mmap(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ pgstat_report_backend_allocated_bytes_decrease(*mapped_size);
*mapped_address = NULL;
*mapped_size = 0;
if (op == DSM_OP_DESTROY && unlink(name) != 0)
@@ -933,6 +1007,13 @@ dsm_impl_mmap(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ pgstat_report_backend_allocated_bytes_increase(request_size);
*mapped_address = address;
*mapped_size = request_size;
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 1146a6c33c..30a89e899a 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -49,6 +49,8 @@ int pgstat_track_activity_query_size = 1024;
/* exposed so that backend_progress.c can access it */
PgBackendStatus *MyBEEntry = NULL;
+/* Memory allocated to this backend prior to pgstats initialization */
+uint64 backend_allocated_bytes = 0;
static PgBackendStatus *BackendStatusArray = NULL;
static char *BackendAppnameBuffer = NULL;
@@ -400,6 +402,13 @@ pgstat_bestart(void)
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = UINT64CONST(0);
+ /*
+ * Move sum of memory allocated prior to pgstats initialization to pgstats
+ * and zero the local variable.
+ */
+ lbeentry.backend_allocated_bytes = backend_allocated_bytes;
+ backend_allocated_bytes = 0;
+
/*
* we don't zero st_progress_param here to save cycles; nobody should
* examine it until st_progress_command has been set to something other
@@ -1191,3 +1200,99 @@ pgstat_clip_activity(const char *raw_activity)
return activity;
}
+
+/* --------
+ * pgstat_report_backend_allocated_bytes_increase() -
+ *
+ * Called to report increase in memory allocated for this backend
+ * --------
+ */
+void
+pgstat_report_backend_allocated_bytes_increase(uint64 allocation)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+
+ if (!beentry || !pgstat_track_activities)
+ {
+ /*
+ * Account for memory before pgstats is initialized. This will be
+ * migrated to pgstats on initialization.
+ */
+ backend_allocated_bytes += allocation;
+
+ return;
+ }
+
+ /*
+ * Update my status entry, following the protocol of bumping
+ * st_changecount before and after. We use a volatile pointer here to
+ * ensure the compiler doesn't try to get cute.
+ */
+ PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
+ beentry->backend_allocated_bytes += allocation;
+ PGSTAT_END_WRITE_ACTIVITY(beentry);
+}
+
+/* --------
+ * pgstat_report_backend_allocated_bytes_decrease() -
+ *
+ * Called to report decrease in memory allocated for this backend
+ * --------
+ */
+void
+pgstat_report_backend_allocated_bytes_decrease(uint64 deallocation)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+
+ /*
+ * Cases may occur where shared memory from a previous postmaster
+ * invocation still exist. These are cleaned up at startup by
+ * dsm_cleanup_using_control_segment. Limit decreasing memory allocated to
+ * zero in case no corresponding prior increase exists or decrease has
+ * already been accounted for.
+ */
+
+ if (!beentry || !pgstat_track_activities)
+ {
+ /*
+ * Account for memory before pgstats is initialized. This will be
+ * migrated to pgstats on initialization. Do not allow
+ * backend_allocated_bytes to go below zero. If pgstats has not been
+ * initialized, we are in startup and we set backend_allocated_bytes
+ * to zero in cases where it would go negative and skip generating an
+ * ereport.
+ */
+ if (deallocation > backend_allocated_bytes)
+ backend_allocated_bytes = 0;
+ else
+ backend_allocated_bytes -= deallocation;
+
+ return;
+ }
+
+ /*
+ * Do not allow backend_allocated_bytes to go below zero. ereport if we
+ * would have. There's no need for a lock around the read here as it's
+ * being referenced from the same backend which means that there shouldn't
+ * be concurrent writes. We want to generate an ereport in these cases.
+ */
+ if (deallocation > beentry->backend_allocated_bytes)
+ {
+ ereport(LOG, errmsg("decrease reduces reported backend memory allocated below zero; setting reported to 0"));
+
+ /*
+ * Overwrite deallocation with current backend_allocated_bytes so we
+ * end up at zero.
+ */
+ deallocation = beentry->backend_allocated_bytes;
+ }
+
+ /*
+ * Update my status entry, following the protocol of bumping
+ * st_changecount before and after. We use a volatile pointer here to
+ * ensure the compiler doesn't try to get cute.
+ */
+ PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
+ beentry->backend_allocated_bytes -= deallocation;
+ PGSTAT_END_WRITE_ACTIVITY(beentry);
+}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 96bffc0f2a..b6d135ad2f 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -553,7 +553,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum
pg_stat_get_activity(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_ACTIVITY_COLS 30
+#define PG_STAT_GET_ACTIVITY_COLS 31
int num_backends = pgstat_fetch_stat_numbackends();
int curr_backend;
int pid = PG_ARGISNULL(0) ? -1 : PG_GETARG_INT32(0);
@@ -609,6 +609,8 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
else
nulls[16] = true;
+ values[30] = UInt64GetDatum(beentry->backend_allocated_bytes);
+
/* Values only available to role member or pg_read_all_stats */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
diff --git a/src/backend/utils/mmgr/aset.c b/src/backend/utils/mmgr/aset.c
index b6a8bbcd59..81e62e4981 100644
--- a/src/backend/utils/mmgr/aset.c
+++ b/src/backend/utils/mmgr/aset.c
@@ -47,6 +47,7 @@
#include "postgres.h"
#include "port/pg_bitutils.h"
+#include "utils/backend_status.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
#include "utils/memutils_memorychunk.h"
@@ -521,6 +522,7 @@ AllocSetContextCreateInternal(MemoryContext parent,
name);
((MemoryContext) set)->mem_allocated = firstBlockSize;
+ pgstat_report_backend_allocated_bytes_increase(firstBlockSize);
return (MemoryContext) set;
}
@@ -543,6 +545,7 @@ AllocSetReset(MemoryContext context)
AllocSet set = (AllocSet) context;
AllocBlock block;
Size keepersize PG_USED_FOR_ASSERTS_ONLY;
+ uint64 deallocation = 0;
Assert(AllocSetIsValid(set));
@@ -585,6 +588,7 @@ AllocSetReset(MemoryContext context)
{
/* Normal case, release the block */
context->mem_allocated -= block->endptr - ((char *) block);
+ deallocation += block->endptr - ((char *) block);
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
@@ -595,6 +599,7 @@ AllocSetReset(MemoryContext context)
}
Assert(context->mem_allocated == keepersize);
+ pgstat_report_backend_allocated_bytes_decrease(deallocation);
/* Reset block size allocation sequence, too */
set->nextBlockSize = set->initBlockSize;
@@ -613,6 +618,7 @@ AllocSetDelete(MemoryContext context)
AllocSet set = (AllocSet) context;
AllocBlock block = set->blocks;
Size keepersize PG_USED_FOR_ASSERTS_ONLY;
+ uint64 deallocation = 0;
Assert(AllocSetIsValid(set));
@@ -651,11 +657,13 @@ AllocSetDelete(MemoryContext context)
freelist->first_free = (AllocSetContext *) oldset->header.nextchild;
freelist->num_free--;
+ deallocation += oldset->header.mem_allocated;
/* All that remains is to free the header/initial block */
free(oldset);
}
Assert(freelist->num_free == 0);
+ pgstat_report_backend_allocated_bytes_decrease(deallocation);
}
/* Now add the just-deleted context to the freelist. */
@@ -672,7 +680,10 @@ AllocSetDelete(MemoryContext context)
AllocBlock next = block->next;
if (block != set->keeper)
+ {
context->mem_allocated -= block->endptr - ((char *) block);
+ deallocation += block->endptr - ((char *) block);
+ }
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
@@ -685,6 +696,8 @@ AllocSetDelete(MemoryContext context)
}
Assert(context->mem_allocated == keepersize);
+ pgstat_report_backend_allocated_bytes_decrease(deallocation +
+ context->mem_allocated);
/* Finally, free the context header, including the keeper block */
free(set);
@@ -734,6 +747,7 @@ AllocSetAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_backend_allocated_bytes_increase(blksize);
block->aset = set;
block->freeptr = block->endptr = ((char *) block) + blksize;
@@ -944,6 +958,7 @@ AllocSetAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_backend_allocated_bytes_increase(blksize);
block->aset = set;
block->freeptr = ((char *) block) + ALLOC_BLOCKHDRSZ;
@@ -1043,6 +1058,7 @@ AllocSetFree(void *pointer)
block->next->prev = block->prev;
set->header.mem_allocated -= block->endptr - ((char *) block);
+ pgstat_report_backend_allocated_bytes_decrease(block->endptr - ((char *) block));
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
@@ -1173,7 +1189,9 @@ AllocSetRealloc(void *pointer, Size size)
/* updated separately, not to underflow when (oldblksize > blksize) */
set->header.mem_allocated -= oldblksize;
+ pgstat_report_backend_allocated_bytes_decrease(oldblksize);
set->header.mem_allocated += blksize;
+ pgstat_report_backend_allocated_bytes_increase(blksize);
block->freeptr = block->endptr = ((char *) block) + blksize;
diff --git a/src/backend/utils/mmgr/generation.c b/src/backend/utils/mmgr/generation.c
index b432a92be3..df3007edfb 100644
--- a/src/backend/utils/mmgr/generation.c
+++ b/src/backend/utils/mmgr/generation.c
@@ -37,6 +37,7 @@
#include "lib/ilist.h"
#include "port/pg_bitutils.h"
+#include "utils/backend_status.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
#include "utils/memutils_memorychunk.h"
@@ -267,6 +268,7 @@ GenerationContextCreate(MemoryContext parent,
name);
((MemoryContext) set)->mem_allocated = firstBlockSize;
+ pgstat_report_backend_allocated_bytes_increase(firstBlockSize);
return (MemoryContext) set;
}
@@ -283,6 +285,7 @@ GenerationReset(MemoryContext context)
{
GenerationContext *set = (GenerationContext *) context;
dlist_mutable_iter miter;
+ uint64 deallocation = 0;
Assert(GenerationIsValid(set));
@@ -305,9 +308,14 @@ GenerationReset(MemoryContext context)
if (block == set->keeper)
GenerationBlockMarkEmpty(block);
else
+ {
+ deallocation += block->blksize;
GenerationBlockFree(set, block);
+ }
}
+ pgstat_report_backend_allocated_bytes_decrease(deallocation);
+
/* set it so new allocations to make use of the keeper block */
set->block = set->keeper;
@@ -328,6 +336,9 @@ GenerationDelete(MemoryContext context)
{
/* Reset to release all releasable GenerationBlocks */
GenerationReset(context);
+
+ pgstat_report_backend_allocated_bytes_decrease(context->mem_allocated);
+
/* And free the context header and keeper block */
free(context);
}
@@ -374,6 +385,7 @@ GenerationAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_backend_allocated_bytes_increase(blksize);
/* block with a single (used) chunk */
block->context = set;
@@ -477,6 +489,7 @@ GenerationAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_backend_allocated_bytes_increase(blksize);
/* initialize the new block */
GenerationBlockInit(set, block, blksize);
@@ -726,6 +739,8 @@ GenerationFree(void *pointer)
dlist_delete(&block->node);
set->header.mem_allocated -= block->blksize;
+ pgstat_report_backend_allocated_bytes_decrease(block->blksize);
+
free(block);
}
diff --git a/src/backend/utils/mmgr/slab.c b/src/backend/utils/mmgr/slab.c
index 6df0839b6a..532c038973 100644
--- a/src/backend/utils/mmgr/slab.c
+++ b/src/backend/utils/mmgr/slab.c
@@ -53,6 +53,7 @@
#include "postgres.h"
#include "lib/ilist.h"
+#include "utils/backend_status.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
#include "utils/memutils_memorychunk.h"
@@ -238,6 +239,12 @@ SlabContextCreate(MemoryContext parent,
parent,
name);
+ /*
+ * If SlabContextCreate is updated to add headerSize to
+ * context->mem_allocated, then update here and SlabDelete appropriately
+ */
+ pgstat_report_backend_allocated_bytes_increase(headerSize);
+
return (MemoryContext) slab;
}
@@ -253,6 +260,7 @@ SlabReset(MemoryContext context)
{
SlabContext *slab = (SlabContext *) context;
int i;
+ uint64 deallocation = 0;
Assert(SlabIsValid(slab));
@@ -278,9 +286,11 @@ SlabReset(MemoryContext context)
free(block);
slab->nblocks--;
context->mem_allocated -= slab->blockSize;
+ deallocation += slab->blockSize;
}
}
+ pgstat_report_backend_allocated_bytes_decrease(deallocation);
slab->minFreeChunks = 0;
Assert(slab->nblocks == 0);
@@ -294,8 +304,17 @@ SlabReset(MemoryContext context)
void
SlabDelete(MemoryContext context)
{
+ /*
+ * Until header allocation is included in context->mem_allocated, cast to
+ * slab and decrement the headerSize
+ */
+ SlabContext *slab = castNode(SlabContext, context);
+
/* Reset to release all the SlabBlocks */
SlabReset(context);
+
+ pgstat_report_backend_allocated_bytes_decrease(slab->headerSize);
+
/* And free the context header */
free(context);
}
@@ -364,6 +383,7 @@ SlabAlloc(MemoryContext context, Size size)
slab->minFreeChunks = slab->chunksPerBlock;
slab->nblocks += 1;
context->mem_allocated += slab->blockSize;
+ pgstat_report_backend_allocated_bytes_increase(slab->blockSize);
}
/* grab the block from the freelist (even the new block is there) */
@@ -537,6 +557,7 @@ SlabFree(void *pointer)
free(block);
slab->nblocks--;
slab->header.mem_allocated -= slab->blockSize;
+ pgstat_report_backend_allocated_bytes_decrease(slab->blockSize);
}
else
dlist_push_head(&slab->freelist[block->nfree], &block->node);
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 20f5aa56ea..1c37f7db5d 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5347,9 +5347,9 @@
proname => 'pg_stat_get_activity', prorows => '100', proisstrict => 'f',
proretset => 't', provolatile => 's', proparallel => 'r',
prorettype => 'record', proargtypes => 'int4',
- proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,int4,int8}',
- proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,leader_pid,query_id}',
+ proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,int4,int8,int8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,leader_pid,query_id,backend_allocated_bytes}',
prosrc => 'pg_stat_get_activity' },
{ oid => '3318',
descr => 'statistics: information about progress of backends running maintenance command',
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index b582b46e9f..75d87e8308 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,9 @@ typedef struct PgBackendStatus
/* query identifier, optionally computed using post_parse_analyze_hook */
uint64 st_query_id;
+
+ /* Current memory allocated to this backend */
+ uint64 backend_allocated_bytes;
} PgBackendStatus;
@@ -313,7 +316,9 @@ extern const char *pgstat_get_backend_current_activity(int pid, bool checkUser);
extern const char *pgstat_get_crashed_backend_activity(int pid, char *buffer,
int buflen);
extern uint64 pgstat_get_my_query_id(void);
-
+extern void pgstat_report_backend_allocated_bytes_increase(uint64 allocation);
+extern void pgstat_report_backend_allocated_bytes_decrease(uint64 deallocation);
+extern uint64 pgstat_get_all_backend_memory_allocated(void);
/* ----------
* Support functions for the SQL-callable functions to
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 624d0e5aae..ba9f494806 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1753,10 +1753,11 @@ pg_stat_activity| SELECT s.datid,
s.state,
s.backend_xid,
s.backend_xmin,
+ s.backend_allocated_bytes,
s.query_id,
s.query,
s.backend_type
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, backend_allocated_bytes)
LEFT JOIN pg_database d ON ((s.datid = d.oid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_all_indexes| SELECT c.oid AS relid,
@@ -1871,7 +1872,7 @@ pg_stat_gssapi| SELECT s.pid,
s.gss_auth AS gss_authenticated,
s.gss_princ AS principal,
s.gss_enc AS encrypted
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, backend_allocated_bytes)
WHERE (s.client_port IS NOT NULL);
pg_stat_progress_analyze| SELECT s.pid,
s.datid,
@@ -2052,7 +2053,7 @@ pg_stat_replication| SELECT s.pid,
w.sync_priority,
w.sync_state,
w.reply_time
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, backend_allocated_bytes)
JOIN pg_stat_get_wal_senders() w(pid, state, sent_lsn, write_lsn, flush_lsn, replay_lsn, write_lag, flush_lag, replay_lag, sync_priority, sync_state, reply_time) ON ((s.pid = w.pid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_replication_slots| SELECT s.slot_name,
@@ -2086,7 +2087,7 @@ pg_stat_ssl| SELECT s.pid,
s.ssl_client_dn AS client_dn,
s.ssl_client_serial AS client_serial,
s.ssl_issuer_dn AS issuer_dn
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, backend_allocated_bytes)
WHERE (s.client_port IS NOT NULL);
pg_stat_subscription| SELECT su.oid AS subid,
su.subname,
--
2.25.1
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: Add the ability to limit the amount of memory that can be allocated to backends.
2022-08-31 16:50 Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-08-31 17:34 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Justin Pryzby <[email protected]>
2022-09-04 03:40 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-09-09 17:14 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Justin Pryzby <[email protected]>
2022-09-12 16:25 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-09-15 08:07 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Ibrar Ahmed <[email protected]>
2022-09-15 14:58 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-10-24 15:27 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Arne Roland <[email protected]>
2022-10-25 15:49 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-11-03 15:48 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
@ 2022-11-27 03:22 ` Reid Thompson <[email protected]>
2022-12-06 18:32 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Andres Freund <[email protected]>
0 siblings, 1 reply; 22+ messages in thread
From: Reid Thompson @ 2022-11-27 03:22 UTC (permalink / raw)
To: Arne Roland <[email protected]>; [email protected] <[email protected]>; +Cc: [email protected]; Justin Pryzby <[email protected]>; Ibrar Ahmed <[email protected]>
On Thu, 2022-11-03 at 11:48 -0400, Reid Thompson wrote:
> On Tue, 2022-10-25 at 11:49 -0400, Reid Thompson wrote:
>
> Rebased to current. Add a couple changes per conversation with D
> Christensen (include units in field name, group field with
> backend_xid
> and backend_xmin fields in pg_stat_activity view, rather than between
> query_id and query)
>
rebased/patched to current master && current pg-stat-activity-backend-memory-allocated
--
Reid Thompson
Senior Software Engineer
Crunchy Data, Inc.
[email protected]
www.crunchydata.com
Attachments:
[text/x-patch] 0002-Add-the-ability-to-limit-the-amount-of-memory-that-c.patch (14.1K, ../../[email protected]/2-0002-Add-the-ability-to-limit-the-amount-of-memory-that-c.patch)
download | inline diff:
From 1470f45e086bef0757cc262d10e08904e46b9a88 Mon Sep 17 00:00:00 2001
From: Reid Thompson <[email protected]>
Date: Sat, 4 Jun 2022 22:23:59 -0400
Subject: [PATCH 2/2] Add the ability to limit the amount of memory that can be
allocated to backends.
This builds on the work that adds backend memory allocated to pg_stat_activity.
Add GUC variable max_total_backend_memory.
Specifies a limit to the amount of memory (in MB) that may be allocated to
backends in total (i.e. this is not a per user or per backend limit). If unset,
or set to 0 it is disabled. It is intended as a resource to help avoid the OOM
killer on LINUX and manage resources in general. A backend request that would
push the total over the limit will be denied with an out of memory error causing
that backend's current query/transaction to fail. Due to the dynamic nature of
memory allocations, this limit is not exact. If within 1.5MB of the limit and
two backends request 1MB each at the same time both may be allocated, and exceed
the limit. Further requests will not be allocated until dropping below the
limit. Keep this in mind when setting this value. This limit does not affect
auxiliary backend processes. Backend memory allocations are displayed in the
pg_stat_activity view.
---
doc/src/sgml/config.sgml | 26 +++++
src/backend/storage/ipc/dsm_impl.c | 12 ++
src/backend/utils/activity/backend_status.c | 108 ++++++++++++++++++
src/backend/utils/misc/guc_tables.c | 11 ++
src/backend/utils/misc/postgresql.conf.sample | 3 +
src/backend/utils/mmgr/aset.c | 17 +++
src/backend/utils/mmgr/generation.c | 9 ++
src/backend/utils/mmgr/slab.c | 8 ++
src/include/utils/backend_status.h | 3 +
9 files changed, 197 insertions(+)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 24b1624bad..c2db3ace7a 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -2079,6 +2079,32 @@ include_dir 'conf.d'
</listitem>
</varlistentry>
+ <varlistentry id="guc-max-total-backend-memory" xreflabel="max_total_backend_memory">
+ <term><varname>max_total_backend_memory</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>max_total_backend_memory</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies a limit to the amount of memory (MB) that may be allocated to
+ backends in total (i.e. this is not a per user or per backend limit).
+ If unset, or set to 0 it is disabled. A backend request that would
+ push the total over the limit will be denied with an out of memory
+ error causing that backend's current query/transaction to fail. Due to
+ the dynamic nature of memory allocations, this limit is not exact. If
+ within 1.5MB of the limit and two backends request 1MB each at the same
+ time both may be allocated, and exceed the limit. Further requests will
+ not be allocated until dropping below the limit. Keep this in mind when
+ setting this value. This limit does not affect auxiliary backend
+ processes <xref linkend="glossary-auxiliary-proc"/> . Backend memory
+ allocations (<varname>allocated_bytes</varname>) are displayed in the
+ <link linkend="monitoring-pg-stat-activity-view"><structname>pg_stat_activity</structname></link>
+ view.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/src/backend/storage/ipc/dsm_impl.c b/src/backend/storage/ipc/dsm_impl.c
index 65d59fc43e..8d9df676af 100644
--- a/src/backend/storage/ipc/dsm_impl.c
+++ b/src/backend/storage/ipc/dsm_impl.c
@@ -254,6 +254,10 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
return true;
}
+ /* Do not exceed maximum allowed memory allocation */
+ if (op == DSM_OP_CREATE && exceeds_max_total_bkend_mem(request_size))
+ return false;
+
/*
* Create new segment or open an existing one for attach.
*
@@ -525,6 +529,10 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size,
int flags = IPCProtection;
size_t segsize;
+ /* Do not exceed maximum allowed memory allocation */
+ if (op == DSM_OP_CREATE && exceeds_max_total_bkend_mem(request_size))
+ return false;
+
/*
* Allocate the memory BEFORE acquiring the resource, so that we don't
* leak the resource if memory allocation fails.
@@ -719,6 +727,10 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size,
return true;
}
+ /* Do not exceed maximum allowed memory allocation */
+ if (op == DSM_OP_CREATE && exceeds_max_total_bkend_mem(request_size))
+ return false;
+
/* Create new segment or open an existing one for attach. */
if (op == DSM_OP_CREATE)
{
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 3785e8af53..07dfd8f490 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -45,6 +45,9 @@
bool pgstat_track_activities = false;
int pgstat_track_activity_query_size = 1024;
+/* Max backend memory allocation allowed (MB). 0 = disabled */
+int max_total_bkend_mem = 0;
+
/* exposed so that backend_progress.c can access it */
PgBackendStatus *MyBEEntry = NULL;
@@ -1236,3 +1239,108 @@ pgstat_reset_allocated_bytes_storage(void)
my_allocated_bytes = &local_my_allocated_bytes;
}
+/* ----------
+ * pgstat_get_all_memory_allocated() -
+ *
+ * Return a uint64 representing the current shared memory allocated to all
+ * backends. This looks directly at the BackendStatusArray, and so will
+ * provide current information regardless of the age of our transaction's
+ * snapshot of the status array.
+ * In the future we will likely utilize additional values - perhaps limit
+ * backend allocation by user/role, etc.
+ * ----------
+ */
+uint64
+pgstat_get_all_backend_memory_allocated(void)
+{
+ PgBackendStatus *beentry;
+ int i;
+ uint64 all_memory_allocated = 0;
+
+ beentry = BackendStatusArray;
+
+ /*
+ * We probably shouldn't get here before shared memory has been set up,
+ * but be safe.
+ */
+ if (beentry == NULL || BackendActivityBuffer == NULL)
+ return 0;
+
+ /*
+ * We include AUX procs in all backend memory calculation
+ */
+ for (i = 1; i <= NumBackendStatSlots; i++)
+ {
+ /*
+ * We use a volatile pointer here to ensure the compiler doesn't try
+ * to get cute.
+ */
+ volatile PgBackendStatus *vbeentry = beentry;
+ bool found;
+ uint64 allocated_bytes = 0;
+
+ for (;;)
+ {
+ int before_changecount;
+ int after_changecount;
+
+ pgstat_begin_read_activity(vbeentry, before_changecount);
+
+ /*
+ * Ignore invalid entries, which may contain invalid data.
+ * See pgstat_beshutdown_hook()
+ */
+ if (vbeentry->st_procpid > 0)
+ allocated_bytes = vbeentry->allocated_bytes;
+
+ pgstat_end_read_activity(vbeentry, after_changecount);
+
+ if ((found = pgstat_read_activity_complete(before_changecount,
+ after_changecount)))
+ break;
+
+ /* Make sure we can break out of loop if stuck... */
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ if (found)
+ all_memory_allocated += allocated_bytes;
+
+ beentry++;
+ }
+
+ return all_memory_allocated;
+}
+
+/*
+ * Determine if allocation request will exceed max backend memory allowed.
+ * Do not apply to auxiliary processes.
+ */
+bool
+exceeds_max_total_bkend_mem(uint64 allocation_request)
+{
+ bool result = false;
+
+ /* Exclude auxiliary processes from the check */
+ if (MyAuxProcType != NotAnAuxProcess)
+ return result;
+
+ /* Convert max_total_bkend_mem to bytes for comparison */
+ if (max_total_bkend_mem &&
+ pgstat_get_all_backend_memory_allocated() +
+ allocation_request > (uint64) max_total_bkend_mem * 1024 * 1024)
+ {
+ /*
+ * Explicitly identify the OOM being a result of this configuration
+ * parameter vs a system failure to allocate OOM.
+ */
+ ereport(WARNING,
+ errmsg("allocation would exceed max_total_memory limit (%llu > %llu)",
+ (unsigned long long) pgstat_get_all_backend_memory_allocated() +
+ allocation_request, (unsigned long long) max_total_bkend_mem * 1024 * 1024));
+
+ result = true;
+ }
+
+ return result;
+}
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 349dd6a537..c20a656310 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -3423,6 +3423,17 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_total_backend_memory", PGC_SU_BACKEND, RESOURCES_MEM,
+ gettext_noop("Restrict total backend memory allocations to this max."),
+ gettext_noop("0 turns this feature off."),
+ GUC_UNIT_MB
+ },
+ &max_total_bkend_mem,
+ 0, 0, INT_MAX,
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, 0, 0, 0, NULL, NULL, NULL
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 868d21c351..1ce0dee6d0 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -155,6 +155,9 @@
# mmap
# (change requires restart)
#min_dynamic_shared_memory = 0MB # (change requires restart)
+#max_total_backend_memory = 0MB # Restrict total backend memory allocations
+ # to this max (in MB). 0 turns this feature
+ # off.
# - Disk -
diff --git a/src/backend/utils/mmgr/aset.c b/src/backend/utils/mmgr/aset.c
index b202e115b6..596f1db408 100644
--- a/src/backend/utils/mmgr/aset.c
+++ b/src/backend/utils/mmgr/aset.c
@@ -440,6 +440,10 @@ AllocSetContextCreateInternal(MemoryContext parent,
else
firstBlockSize = Max(firstBlockSize, initBlockSize);
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(firstBlockSize))
+ return NULL;
+
/*
* Allocate the initial block. Unlike other aset.c blocks, it starts with
* the context header and its block header follows that.
@@ -741,6 +745,11 @@ AllocSetAlloc(MemoryContext context, Size size)
#endif
blksize = chunk_size + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ;
+
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
block = (AllocBlock) malloc(blksize);
if (block == NULL)
return NULL;
@@ -938,6 +947,10 @@ AllocSetAlloc(MemoryContext context, Size size)
while (blksize < required_size)
blksize <<= 1;
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
/* Try to allocate it */
block = (AllocBlock) malloc(blksize);
@@ -1178,6 +1191,10 @@ AllocSetRealloc(void *pointer, Size size)
blksize = chksize + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ;
oldblksize = block->endptr - ((char *) block);
+ /* Do not exceed maximum allowed memory allocation */
+ if (blksize > oldblksize && exceeds_max_total_bkend_mem(blksize - oldblksize))
+ return NULL;
+
block = (AllocBlock) realloc(block, blksize);
if (block == NULL)
{
diff --git a/src/backend/utils/mmgr/generation.c b/src/backend/utils/mmgr/generation.c
index 459eb985d6..145409cf21 100644
--- a/src/backend/utils/mmgr/generation.c
+++ b/src/backend/utils/mmgr/generation.c
@@ -201,6 +201,9 @@ GenerationContextCreate(MemoryContext parent,
else
allocSize = Max(allocSize, initBlockSize);
+ if (exceeds_max_total_bkend_mem(allocSize))
+ return NULL;
+
/*
* Allocate the initial block. Unlike other generation.c blocks, it
* starts with the context header and its block header follows that.
@@ -380,6 +383,9 @@ GenerationAlloc(MemoryContext context, Size size)
{
Size blksize = required_size + Generation_BLOCKHDRSZ;
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
block = (GenerationBlock *) malloc(blksize);
if (block == NULL)
return NULL;
@@ -483,6 +489,9 @@ GenerationAlloc(MemoryContext context, Size size)
if (blksize < required_size)
blksize = pg_nextpower2_size_t(required_size);
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
block = (GenerationBlock *) malloc(blksize);
if (block == NULL)
diff --git a/src/backend/utils/mmgr/slab.c b/src/backend/utils/mmgr/slab.c
index f38256f6f3..9304e13638 100644
--- a/src/backend/utils/mmgr/slab.c
+++ b/src/backend/utils/mmgr/slab.c
@@ -197,6 +197,10 @@ SlabContextCreate(MemoryContext parent,
headerSize += chunksPerBlock * sizeof(bool);
#endif
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(headerSize))
+ return NULL;
+
slab = (SlabContext *) malloc(headerSize);
if (slab == NULL)
{
@@ -351,6 +355,10 @@ SlabAlloc(MemoryContext context, Size size)
*/
if (slab->minFreeChunks == 0)
{
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(slab->blockSize))
+ return NULL;
+
block = (SlabBlock *) malloc(slab->blockSize);
if (block == NULL)
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index e5aa90b101..94528aa650 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -286,6 +286,7 @@ typedef struct LocalPgBackendStatus
*/
extern PGDLLIMPORT bool pgstat_track_activities;
extern PGDLLIMPORT int pgstat_track_activity_query_size;
+extern PGDLLIMPORT int max_total_bkend_mem;
/* ----------
@@ -325,6 +326,7 @@ extern const char *pgstat_get_backend_current_activity(int pid, bool checkUser);
extern const char *pgstat_get_crashed_backend_activity(int pid, char *buffer,
int buflen);
extern uint64 pgstat_get_my_query_id(void);
+extern uint64 pgstat_get_all_backend_memory_allocated(void);
extern void pgstat_set_allocated_bytes_storage(uint64 *allocated_bytes);
extern void pgstat_reset_allocated_bytes_storage(void);
@@ -337,6 +339,7 @@ extern int pgstat_fetch_stat_numbackends(void);
extern PgBackendStatus *pgstat_fetch_stat_beentry(BackendId beid);
extern LocalPgBackendStatus *pgstat_fetch_stat_local_beentry(int beid);
extern char *pgstat_clip_activity(const char *raw_activity);
+extern bool exceeds_max_total_bkend_mem(uint64 allocation_request);
/* ----------
* pgstat_report_allocated_bytes() -
--
2.25.1
[text/x-patch] 0001-Add-tracking-of-backend-memory-allocated-to-pg_stat_.patch (32.8K, ../../[email protected]/3-0001-Add-tracking-of-backend-memory-allocated-to-pg_stat_.patch)
download | inline diff:
From 3d772d8620faba4bd4e091d6618c63557fbf6749 Mon Sep 17 00:00:00 2001
From: Reid Thompson <[email protected]>
Date: Thu, 11 Aug 2022 12:01:25 -0400
Subject: [PATCH 1/2] Add tracking of backend memory allocated to
pg_stat_activity
This new field displays the current bytes of memory allocated to the
backend process. It is updated as memory for the process is
malloc'd/free'd. Memory allocated to items on the freelist is included in
the displayed value. Dynamic shared memory allocations are included
only in the value displayed for the backend that created them, they are
not included in the value for backends that are attached to them to
avoid double counting. On occasion, orphaned memory segments may be
cleaned up on postmaster startup. This may result in decreasing the sum
without a prior increment. We limit the floor of backend_mem_allocated
to zero. Updated pg_stat_activity documentation for the new column.
---
doc/src/sgml/monitoring.sgml | 15 ++++
src/backend/catalog/system_views.sql | 1 +
src/backend/postmaster/autovacuum.c | 6 ++
src/backend/postmaster/postmaster.c | 13 ++++
src/backend/postmaster/syslogger.c | 3 +
src/backend/storage/ipc/dsm_impl.c | 81 +++++++++++++++++++++
src/backend/utils/activity/backend_status.c | 45 ++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 4 +-
src/backend/utils/mmgr/aset.c | 17 +++++
src/backend/utils/mmgr/generation.c | 15 ++++
src/backend/utils/mmgr/slab.c | 21 ++++++
src/include/catalog/pg_proc.dat | 6 +-
src/include/utils/backend_status.h | 59 ++++++++++++++-
src/test/regress/expected/rules.out | 9 ++-
src/test/regress/expected/stats.out | 11 +++
src/test/regress/sql/stats.sql | 3 +
16 files changed, 300 insertions(+), 9 deletions(-)
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 5579b8b9e0..ffe7d2566c 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -947,6 +947,21 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>allocated_bytes</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Memory currently allocated to this backend in bytes. This is the balance
+ of bytes allocated and freed by this backend. Dynamic shared memory
+ allocations are included only in the value displayed for the backend that
+ created them, they are not included in the value for backends that are
+ attached to them to avoid double counting. Use <function>pg_size_pretty</function>
+ described in <xref linkend="functions-admin-dbsize"/> to make this value
+ more easily readable.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>query</structfield> <type>text</type>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 2d8104b090..9ea8f78c95 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -864,6 +864,7 @@ CREATE VIEW pg_stat_activity AS
S.state,
S.backend_xid,
s.backend_xmin,
+ S.allocated_bytes,
S.query_id,
S.query,
S.backend_type
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 601834d4b4..f54606104d 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -407,6 +407,9 @@ StartAutoVacLauncher(void)
#ifndef EXEC_BACKEND
case 0:
+ /* Zero allocated bytes to avoid double counting parent allocation */
+ pgstat_zero_my_allocated_bytes();
+
/* in postmaster child ... */
InitPostmasterChild();
@@ -1485,6 +1488,9 @@ StartAutoVacWorker(void)
#ifndef EXEC_BACKEND
case 0:
+ /* Zero allocated bytes to avoid double counting parent allocation */
+ pgstat_zero_my_allocated_bytes();
+
/* in postmaster child ... */
InitPostmasterChild();
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index a8a246921f..89a6caec78 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -4102,6 +4102,9 @@ BackendStartup(Port *port)
{
free(bn);
+ /* Zero allocated bytes to avoid double counting parent allocation */
+ pgstat_zero_my_allocated_bytes();
+
/* Detangle from postmaster */
InitPostmasterChild();
@@ -5307,6 +5310,11 @@ StartChildProcess(AuxProcType type)
MemoryContextDelete(PostmasterContext);
PostmasterContext = NULL;
+ /* Zero allocated bytes to avoid double counting parent allocation.
+ * Needs to be after the MemoryContextDelete(PostmasterContext) above.
+ */
+ pgstat_zero_my_allocated_bytes();
+
AuxiliaryProcessMain(type); /* does not return */
}
#endif /* EXEC_BACKEND */
@@ -5700,6 +5708,11 @@ do_start_bgworker(RegisteredBgWorker *rw)
MemoryContextDelete(PostmasterContext);
PostmasterContext = NULL;
+ /* Zero allocated bytes to avoid double counting parent allocation.
+ * Needs to be after the MemoryContextDelete(PostmasterContext) above.
+ */
+ pgstat_zero_my_allocated_bytes();
+
StartBackgroundWorker();
exit(1); /* should not get here */
diff --git a/src/backend/postmaster/syslogger.c b/src/backend/postmaster/syslogger.c
index d6d02e3c63..4cbc59cda5 100644
--- a/src/backend/postmaster/syslogger.c
+++ b/src/backend/postmaster/syslogger.c
@@ -679,6 +679,9 @@ SysLogger_Start(void)
#ifndef EXEC_BACKEND
case 0:
+ /* Zero allocated bytes to avoid double counting parent allocation */
+ pgstat_zero_my_allocated_bytes();
+
/* in postmaster child ... */
InitPostmasterChild();
diff --git a/src/backend/storage/ipc/dsm_impl.c b/src/backend/storage/ipc/dsm_impl.c
index 6ddd46a4e7..65d59fc43e 100644
--- a/src/backend/storage/ipc/dsm_impl.c
+++ b/src/backend/storage/ipc/dsm_impl.c
@@ -66,6 +66,7 @@
#include "postmaster/postmaster.h"
#include "storage/dsm_impl.h"
#include "storage/fd.h"
+#include "utils/backend_status.h"
#include "utils/guc.h"
#include "utils/memutils.h"
@@ -232,6 +233,14 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ if (op == DSM_OP_DESTROY)
+ pgstat_report_allocated_bytes(*mapped_size, DECREASE);
*mapped_address = NULL;
*mapped_size = 0;
if (op == DSM_OP_DESTROY && shm_unlink(name) != 0)
@@ -332,6 +341,36 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ {
+ /*
+ * Posix creation calls dsm_impl_posix_resize implying that resizing
+ * occurs or may be added in the future. As implemented
+ * dsm_impl_posix_resize utilizes fallocate or truncate, passing the
+ * whole new size as input, growing the allocation as needed (only
+ * truncate supports shrinking). We update by replacing the old
+ * allocation with the new.
+ */
+#if defined(HAVE_POSIX_FALLOCATE) && defined(__linux__)
+ /*
+ * posix_fallocate does not shrink allocations, adjust only on
+ * allocation increase.
+ */
+ if (request_size > *mapped_size)
+ {
+ pgstat_report_allocated_bytes(*mapped_size, DECREASE);
+ pgstat_report_allocated_bytes(request_size, INCREASE);
+ }
+#else
+ pgstat_report_allocated_bytes(*mapped_size, DECREASE);
+ pgstat_report_allocated_bytes(request_size, INCREASE);
+#endif
+ }
*mapped_address = address;
*mapped_size = request_size;
close(fd);
@@ -537,6 +576,14 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ if (op == DSM_OP_DESTROY)
+ pgstat_report_allocated_bytes(*mapped_size, DECREASE);
*mapped_address = NULL;
*mapped_size = 0;
if (op == DSM_OP_DESTROY && shmctl(ident, IPC_RMID, NULL) < 0)
@@ -584,6 +631,13 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ pgstat_report_allocated_bytes(request_size, INCREASE);
*mapped_address = address;
*mapped_size = request_size;
@@ -652,6 +706,13 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size,
return false;
}
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ if (op == DSM_OP_DESTROY)
+ pgstat_report_allocated_bytes(*mapped_size, DECREASE);
*impl_private = NULL;
*mapped_address = NULL;
*mapped_size = 0;
@@ -768,6 +829,12 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size,
return false;
}
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ pgstat_report_allocated_bytes(info.RegionSize, INCREASE);
*mapped_address = address;
*mapped_size = info.RegionSize;
*impl_private = hmap;
@@ -812,6 +879,13 @@ dsm_impl_mmap(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ pgstat_report_allocated_bytes(*mapped_size, DECREASE);
*mapped_address = NULL;
*mapped_size = 0;
if (op == DSM_OP_DESTROY && unlink(name) != 0)
@@ -933,6 +1007,13 @@ dsm_impl_mmap(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ pgstat_report_allocated_bytes(request_size, INCREASE);
*mapped_address = address;
*mapped_size = request_size;
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 1146a6c33c..3785e8af53 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -49,6 +49,9 @@ int pgstat_track_activity_query_size = 1024;
/* exposed so that backend_progress.c can access it */
PgBackendStatus *MyBEEntry = NULL;
+/* Memory allocated to this backend prior to pgstats initialization */
+uint64 local_my_allocated_bytes = 0;
+uint64 *my_allocated_bytes = &local_my_allocated_bytes;
static PgBackendStatus *BackendStatusArray = NULL;
static char *BackendAppnameBuffer = NULL;
@@ -400,6 +403,15 @@ pgstat_bestart(void)
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = UINT64CONST(0);
+ /* Alter allocation reporting from local_my_allocated_bytes to shared memory */
+ pgstat_set_allocated_bytes_storage(&MyBEEntry->allocated_bytes);
+
+ /* Populate sum of memory allocated prior to pgstats initialization to pgstats
+ * and zero the local variable.
+ */
+ lbeentry.allocated_bytes += local_my_allocated_bytes;
+ local_my_allocated_bytes = 0;
+
/*
* we don't zero st_progress_param here to save cycles; nobody should
* examine it until st_progress_command has been set to something other
@@ -459,6 +471,11 @@ pgstat_beshutdown_hook(int code, Datum arg)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ /*
+ * Stop reporting memory allocation changes to &MyBEEntry->allocated_bytes
+ */
+ pgstat_reset_allocated_bytes_storage();
+
/*
* Clear my status entry, following the protocol of bumping st_changecount
* before and after. We use a volatile pointer here to ensure the
@@ -1191,3 +1208,31 @@ pgstat_clip_activity(const char *raw_activity)
return activity;
}
+
+/*
+ * Configure bytes allocated reporting to report allocated bytes to
+ * *allocated_bytes. *allocated_bytes needs to be valid until
+ * pgstat_set_allocated_bytes_storage() is called.
+ *
+ * Expected to be called during backend startup (in pgstat_bestart), to point
+ * my_allocated_bytes into shared memory.
+ */
+void
+pgstat_set_allocated_bytes_storage(uint64 *new_allocated_bytes)
+{
+ my_allocated_bytes = new_allocated_bytes;
+ *new_allocated_bytes = local_my_allocated_bytes;
+}
+
+/*
+ * Reset allocated bytes storage location.
+ *
+ * Expected to be called during backend shutdown, before the location set up
+ * by pgstat_set_allocated_bytes_storage() becomes invalid.
+ */
+void
+pgstat_reset_allocated_bytes_storage(void)
+{
+ my_allocated_bytes = &local_my_allocated_bytes;
+}
+
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index ae3365d917..170d60df8b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -559,7 +559,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum
pg_stat_get_activity(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_ACTIVITY_COLS 30
+#define PG_STAT_GET_ACTIVITY_COLS 31
int num_backends = pgstat_fetch_stat_numbackends();
int curr_backend;
int pid = PG_ARGISNULL(0) ? -1 : PG_GETARG_INT32(0);
@@ -615,6 +615,8 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
else
nulls[16] = true;
+ values[30] = UInt64GetDatum(beentry->allocated_bytes);
+
/* Values only available to role member or pg_read_all_stats */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
diff --git a/src/backend/utils/mmgr/aset.c b/src/backend/utils/mmgr/aset.c
index b6a8bbcd59..b202e115b6 100644
--- a/src/backend/utils/mmgr/aset.c
+++ b/src/backend/utils/mmgr/aset.c
@@ -47,6 +47,7 @@
#include "postgres.h"
#include "port/pg_bitutils.h"
+#include "utils/backend_status.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
#include "utils/memutils_memorychunk.h"
@@ -521,6 +522,7 @@ AllocSetContextCreateInternal(MemoryContext parent,
name);
((MemoryContext) set)->mem_allocated = firstBlockSize;
+ pgstat_report_allocated_bytes(firstBlockSize, INCREASE);
return (MemoryContext) set;
}
@@ -543,6 +545,7 @@ AllocSetReset(MemoryContext context)
AllocSet set = (AllocSet) context;
AllocBlock block;
Size keepersize PG_USED_FOR_ASSERTS_ONLY;
+ uint64 deallocation = 0;
Assert(AllocSetIsValid(set));
@@ -585,6 +588,7 @@ AllocSetReset(MemoryContext context)
{
/* Normal case, release the block */
context->mem_allocated -= block->endptr - ((char *) block);
+ deallocation += block->endptr - ((char *) block);
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
@@ -595,6 +599,7 @@ AllocSetReset(MemoryContext context)
}
Assert(context->mem_allocated == keepersize);
+ pgstat_report_allocated_bytes(deallocation, DECREASE);
/* Reset block size allocation sequence, too */
set->nextBlockSize = set->initBlockSize;
@@ -613,6 +618,7 @@ AllocSetDelete(MemoryContext context)
AllocSet set = (AllocSet) context;
AllocBlock block = set->blocks;
Size keepersize PG_USED_FOR_ASSERTS_ONLY;
+ uint64 deallocation = 0;
Assert(AllocSetIsValid(set));
@@ -651,11 +657,13 @@ AllocSetDelete(MemoryContext context)
freelist->first_free = (AllocSetContext *) oldset->header.nextchild;
freelist->num_free--;
+ deallocation += oldset->header.mem_allocated;
/* All that remains is to free the header/initial block */
free(oldset);
}
Assert(freelist->num_free == 0);
+ pgstat_report_allocated_bytes(deallocation, DECREASE);
}
/* Now add the just-deleted context to the freelist. */
@@ -672,7 +680,10 @@ AllocSetDelete(MemoryContext context)
AllocBlock next = block->next;
if (block != set->keeper)
+ {
context->mem_allocated -= block->endptr - ((char *) block);
+ deallocation += block->endptr - ((char *) block);
+ }
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
@@ -685,6 +696,7 @@ AllocSetDelete(MemoryContext context)
}
Assert(context->mem_allocated == keepersize);
+ pgstat_report_allocated_bytes(deallocation + context->mem_allocated, DECREASE);
/* Finally, free the context header, including the keeper block */
free(set);
@@ -734,6 +746,7 @@ AllocSetAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_allocated_bytes(blksize, INCREASE);
block->aset = set;
block->freeptr = block->endptr = ((char *) block) + blksize;
@@ -944,6 +957,7 @@ AllocSetAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_allocated_bytes(blksize, INCREASE);
block->aset = set;
block->freeptr = ((char *) block) + ALLOC_BLOCKHDRSZ;
@@ -1043,6 +1057,7 @@ AllocSetFree(void *pointer)
block->next->prev = block->prev;
set->header.mem_allocated -= block->endptr - ((char *) block);
+ pgstat_report_allocated_bytes(block->endptr - ((char *) block), DECREASE);
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
@@ -1173,7 +1188,9 @@ AllocSetRealloc(void *pointer, Size size)
/* updated separately, not to underflow when (oldblksize > blksize) */
set->header.mem_allocated -= oldblksize;
+ pgstat_report_allocated_bytes(oldblksize, DECREASE);
set->header.mem_allocated += blksize;
+ pgstat_report_allocated_bytes(blksize, INCREASE);
block->freeptr = block->endptr = ((char *) block) + blksize;
diff --git a/src/backend/utils/mmgr/generation.c b/src/backend/utils/mmgr/generation.c
index b432a92be3..459eb985d6 100644
--- a/src/backend/utils/mmgr/generation.c
+++ b/src/backend/utils/mmgr/generation.c
@@ -37,6 +37,7 @@
#include "lib/ilist.h"
#include "port/pg_bitutils.h"
+#include "utils/backend_status.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
#include "utils/memutils_memorychunk.h"
@@ -267,6 +268,7 @@ GenerationContextCreate(MemoryContext parent,
name);
((MemoryContext) set)->mem_allocated = firstBlockSize;
+ pgstat_report_allocated_bytes(firstBlockSize, INCREASE);
return (MemoryContext) set;
}
@@ -283,6 +285,7 @@ GenerationReset(MemoryContext context)
{
GenerationContext *set = (GenerationContext *) context;
dlist_mutable_iter miter;
+ uint64 deallocation = 0;
Assert(GenerationIsValid(set));
@@ -305,9 +308,14 @@ GenerationReset(MemoryContext context)
if (block == set->keeper)
GenerationBlockMarkEmpty(block);
else
+ {
+ deallocation += block->blksize;
GenerationBlockFree(set, block);
+ }
}
+ pgstat_report_allocated_bytes(deallocation, DECREASE);
+
/* set it so new allocations to make use of the keeper block */
set->block = set->keeper;
@@ -328,6 +336,9 @@ GenerationDelete(MemoryContext context)
{
/* Reset to release all releasable GenerationBlocks */
GenerationReset(context);
+
+ pgstat_report_allocated_bytes(context->mem_allocated, DECREASE);
+
/* And free the context header and keeper block */
free(context);
}
@@ -374,6 +385,7 @@ GenerationAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_allocated_bytes(blksize, INCREASE);
/* block with a single (used) chunk */
block->context = set;
@@ -477,6 +489,7 @@ GenerationAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_allocated_bytes(blksize, INCREASE);
/* initialize the new block */
GenerationBlockInit(set, block, blksize);
@@ -726,6 +739,8 @@ GenerationFree(void *pointer)
dlist_delete(&block->node);
set->header.mem_allocated -= block->blksize;
+ pgstat_report_allocated_bytes(block->blksize, DECREASE);
+
free(block);
}
diff --git a/src/backend/utils/mmgr/slab.c b/src/backend/utils/mmgr/slab.c
index 6df0839b6a..f38256f6f3 100644
--- a/src/backend/utils/mmgr/slab.c
+++ b/src/backend/utils/mmgr/slab.c
@@ -53,6 +53,7 @@
#include "postgres.h"
#include "lib/ilist.h"
+#include "utils/backend_status.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
#include "utils/memutils_memorychunk.h"
@@ -238,6 +239,12 @@ SlabContextCreate(MemoryContext parent,
parent,
name);
+ /*
+ * If SlabContextCreate is updated to add headerSize to
+ * context->mem_allocated, then update here and SlabDelete appropriately
+ */
+ pgstat_report_allocated_bytes(headerSize, INCREASE);
+
return (MemoryContext) slab;
}
@@ -253,6 +260,7 @@ SlabReset(MemoryContext context)
{
SlabContext *slab = (SlabContext *) context;
int i;
+ uint64 deallocation = 0;
Assert(SlabIsValid(slab));
@@ -278,9 +286,11 @@ SlabReset(MemoryContext context)
free(block);
slab->nblocks--;
context->mem_allocated -= slab->blockSize;
+ deallocation += slab->blockSize;
}
}
+ pgstat_report_allocated_bytes(deallocation, DECREASE);
slab->minFreeChunks = 0;
Assert(slab->nblocks == 0);
@@ -294,8 +304,17 @@ SlabReset(MemoryContext context)
void
SlabDelete(MemoryContext context)
{
+ /*
+ * Until header allocation is included in context->mem_allocated, cast to
+ * slab and decrement the headerSize
+ */
+ SlabContext *slab = castNode(SlabContext, context);
+
/* Reset to release all the SlabBlocks */
SlabReset(context);
+
+ pgstat_report_allocated_bytes(slab->headerSize, DECREASE);
+
/* And free the context header */
free(context);
}
@@ -364,6 +383,7 @@ SlabAlloc(MemoryContext context, Size size)
slab->minFreeChunks = slab->chunksPerBlock;
slab->nblocks += 1;
context->mem_allocated += slab->blockSize;
+ pgstat_report_allocated_bytes(slab->blockSize, INCREASE);
}
/* grab the block from the freelist (even the new block is there) */
@@ -537,6 +557,7 @@ SlabFree(void *pointer)
free(block);
slab->nblocks--;
slab->header.mem_allocated -= slab->blockSize;
+ pgstat_report_allocated_bytes(slab->blockSize, DECREASE);
}
else
dlist_push_head(&slab->freelist[block->nfree], &block->node);
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f9301b2627..1bf02758d4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5373,9 +5373,9 @@
proname => 'pg_stat_get_activity', prorows => '100', proisstrict => 'f',
proretset => 't', provolatile => 's', proparallel => 'r',
prorettype => 'record', proargtypes => 'int4',
- proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,int4,int8}',
- proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,leader_pid,query_id}',
+ proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,int4,int8,int8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,leader_pid,query_id,allocated_bytes}',
prosrc => 'pg_stat_get_activity' },
{ oid => '3318',
descr => 'statistics: information about progress of backends running maintenance command',
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index b582b46e9f..e5aa90b101 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -15,6 +15,7 @@
#include "miscadmin.h" /* for BackendType */
#include "storage/backendid.h"
#include "utils/backend_progress.h"
+#include "common/int.h"
/* ----------
@@ -32,6 +33,13 @@ typedef enum BackendState
STATE_DISABLED
} BackendState;
+/* Enum helper for reporting memory allocated bytes */
+enum allocation_direction
+{
+ DECREASE = -1,
+ IGNORE,
+ INCREASE,
+};
/* ----------
* Shared-memory data structures
@@ -169,6 +177,9 @@ typedef struct PgBackendStatus
/* query identifier, optionally computed using post_parse_analyze_hook */
uint64 st_query_id;
+
+ /* Current memory allocated to this backend */
+ uint64 allocated_bytes;
} PgBackendStatus;
@@ -282,6 +293,7 @@ extern PGDLLIMPORT int pgstat_track_activity_query_size;
* ----------
*/
extern PGDLLIMPORT PgBackendStatus *MyBEEntry;
+extern PGDLLIMPORT uint64 *my_allocated_bytes;
/* ----------
@@ -313,7 +325,8 @@ extern const char *pgstat_get_backend_current_activity(int pid, bool checkUser);
extern const char *pgstat_get_crashed_backend_activity(int pid, char *buffer,
int buflen);
extern uint64 pgstat_get_my_query_id(void);
-
+extern void pgstat_set_allocated_bytes_storage(uint64 *allocated_bytes);
+extern void pgstat_reset_allocated_bytes_storage(void);
/* ----------
* Support functions for the SQL-callable functions to
@@ -325,5 +338,49 @@ extern PgBackendStatus *pgstat_fetch_stat_beentry(BackendId beid);
extern LocalPgBackendStatus *pgstat_fetch_stat_local_beentry(int beid);
extern char *pgstat_clip_activity(const char *raw_activity);
+/* ----------
+ * pgstat_report_allocated_bytes() -
+ *
+ * Called to report change in memory allocated for this backend.
+ *
+ * my_allocated_bytes initially points to local memory, making it safe to call
+ * this before pgstats has been initialized. allocation_direction is a
+ * positive/negative multiplier enum defined above.
+ * ----------
+ */
+static inline void
+pgstat_report_allocated_bytes(uint64 allocated_bytes, int allocation_direction)
+{
+ uint64 temp;
+
+ /* Avoid *my_allocated_bytes unsigned integer overflow on DECREASE */
+ if (allocation_direction == DECREASE &&
+ pg_sub_u64_overflow(*my_allocated_bytes, allocated_bytes, &temp))
+ {
+ *my_allocated_bytes = 0;
+ ereport(LOG,
+ errmsg("Backend %d deallocated %ld bytes, exceeding the %ld bytes it is currently reporting allocated. Setting reported to 0.",
+ MyProcPid, allocated_bytes, *my_allocated_bytes));
+ }
+ else
+ *my_allocated_bytes += (allocated_bytes) * allocation_direction;
+
+ return;
+}
+
+/* ---------
+ * pgstat_zero_my_allocated_bytes() -
+ *
+ * Called to zero out local allocated bytes variable after fork to avoid double
+ * counting allocations.
+ * ---------
+ */
+static inline void
+pgstat_zero_my_allocated_bytes(void)
+{
+ *my_allocated_bytes = 0;
+
+ return;
+}
#endif /* BACKEND_STATUS_H */
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 37c1c86473..5263294ad4 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1756,10 +1756,11 @@ pg_stat_activity| SELECT s.datid,
s.state,
s.backend_xid,
s.backend_xmin,
+ s.allocated_bytes,
s.query_id,
s.query,
s.backend_type
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, allocated_bytes)
LEFT JOIN pg_database d ON ((s.datid = d.oid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_all_indexes| SELECT c.oid AS relid,
@@ -1874,7 +1875,7 @@ pg_stat_gssapi| SELECT s.pid,
s.gss_auth AS gss_authenticated,
s.gss_princ AS principal,
s.gss_enc AS encrypted
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, allocated_bytes)
WHERE (s.client_port IS NOT NULL);
pg_stat_progress_analyze| SELECT s.pid,
s.datid,
@@ -2055,7 +2056,7 @@ pg_stat_replication| SELECT s.pid,
w.sync_priority,
w.sync_state,
w.reply_time
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, allocated_bytes)
JOIN pg_stat_get_wal_senders() w(pid, state, sent_lsn, write_lsn, flush_lsn, replay_lsn, write_lag, flush_lag, replay_lag, sync_priority, sync_state, reply_time) ON ((s.pid = w.pid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_replication_slots| SELECT s.slot_name,
@@ -2089,7 +2090,7 @@ pg_stat_ssl| SELECT s.pid,
s.ssl_client_dn AS client_dn,
s.ssl_client_serial AS client_serial,
s.ssl_issuer_dn AS issuer_dn
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, allocated_bytes)
WHERE (s.client_port IS NOT NULL);
pg_stat_subscription| SELECT su.oid AS subid,
su.subname,
diff --git a/src/test/regress/expected/stats.out b/src/test/regress/expected/stats.out
index 1d84407a03..ab7e95c367 100644
--- a/src/test/regress/expected/stats.out
+++ b/src/test/regress/expected/stats.out
@@ -1126,4 +1126,15 @@ SELECT pg_stat_get_subscription_stats(NULL);
(1 row)
+-- ensure that allocated_bytes exist for backends
+SELECT allocated_bytes > 0 AS result FROM pg_stat_activity WHERE backend_type
+IN ('checkpointer', 'background writer', 'walwriter', 'autovacuum launcher');
+ result
+--------
+ t
+ t
+ t
+ t
+(4 rows)
+
-- End of Stats Test
diff --git a/src/test/regress/sql/stats.sql b/src/test/regress/sql/stats.sql
index b4d6753c71..2f0b1cc9d8 100644
--- a/src/test/regress/sql/stats.sql
+++ b/src/test/regress/sql/stats.sql
@@ -535,5 +535,8 @@ SET enable_seqscan TO on;
SELECT pg_stat_get_replication_slot(NULL);
SELECT pg_stat_get_subscription_stats(NULL);
+-- ensure that allocated_bytes exist for backends
+SELECT allocated_bytes > 0 AS result FROM pg_stat_activity WHERE backend_type
+IN ('checkpointer', 'background writer', 'walwriter', 'autovacuum launcher');
-- End of Stats Test
--
2.25.1
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: Add the ability to limit the amount of memory that can be allocated to backends.
2022-08-31 16:50 Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-08-31 17:34 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Justin Pryzby <[email protected]>
2022-09-04 03:40 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-09-09 17:14 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Justin Pryzby <[email protected]>
2022-09-12 16:25 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-09-15 08:07 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Ibrar Ahmed <[email protected]>
2022-09-15 14:58 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-10-24 15:27 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Arne Roland <[email protected]>
2022-10-25 15:49 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-11-03 15:48 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-11-27 03:22 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
@ 2022-12-06 18:32 ` Andres Freund <[email protected]>
2022-12-09 15:05 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
0 siblings, 1 reply; 22+ messages in thread
From: Andres Freund @ 2022-12-06 18:32 UTC (permalink / raw)
To: Reid Thompson <[email protected]>; +Cc: Arne Roland <[email protected]>; [email protected] <[email protected]>; Justin Pryzby <[email protected]>; Ibrar Ahmed <[email protected]>
Hi,
On 2022-11-26 22:22:15 -0500, Reid Thompson wrote:
> rebased/patched to current master && current pg-stat-activity-backend-memory-allocated
This version fails to build with msvc, and builds with warnings on other
platforms.
https://cirrus-ci.com/build/5410696721072128
msvc:
[20:26:51.286] c:\cirrus\src\include\utils/backend_status.h(40): error C2059: syntax error: 'constant'
mingw cross:
[20:26:26.358] from /usr/share/mingw-w64/include/winsock2.h:23,
[20:26:26.358] from ../../src/include/port/win32_port.h:60,
[20:26:26.358] from ../../src/include/port.h:24,
[20:26:26.358] from ../../src/include/c.h:1306,
[20:26:26.358] from ../../src/include/postgres.h:47,
[20:26:26.358] from controldata_utils.c:18:
[20:26:26.358] ../../src/include/utils/backend_status.h:40:2: error: expected identifier before numeric constant
[20:26:26.358] 40 | IGNORE,
[20:26:26.358] | ^~~~~~
[20:26:26.358] In file included from ../../src/include/postgres.h:48,
[20:26:26.358] from controldata_utils.c:18:
[20:26:26.358] ../../src/include/utils/backend_status.h: In function ‘pgstat_report_allocated_bytes’:
[20:26:26.358] ../../src/include/utils/backend_status.h:365:12: error: format ‘%ld’ expects argument of type ‘long int’, but argument 3 has type ‘uint64’ {aka ‘long long unsigned int’} [-Werror=format=]
[20:26:26.358] 365 | errmsg("Backend %d deallocated %ld bytes, exceeding the %ld bytes it is currently reporting allocated. Setting reported to 0.",
[20:26:26.358] | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[20:26:26.358] 366 | MyProcPid, allocated_bytes, *my_allocated_bytes));
[20:26:26.358] | ~~~~~~~~~~~~~~~
[20:26:26.358] | |
[20:26:26.358] | uint64 {aka long long unsigned int}
Due to windows having long be 32bit, you need to use %lld. Our custom to deal
with that is to cast the argument to errmsg as long long unsigned and use
%llu.
Btw, given that the argument is uint64, it doesn't seem correct to use %ld,
that's signed. Not that it's going to matter, but ...
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: Add the ability to limit the amount of memory that can be allocated to backends.
2022-08-31 16:50 Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-08-31 17:34 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Justin Pryzby <[email protected]>
2022-09-04 03:40 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-09-09 17:14 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Justin Pryzby <[email protected]>
2022-09-12 16:25 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-09-15 08:07 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Ibrar Ahmed <[email protected]>
2022-09-15 14:58 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-10-24 15:27 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Arne Roland <[email protected]>
2022-10-25 15:49 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-11-03 15:48 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-11-27 03:22 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-12-06 18:32 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Andres Freund <[email protected]>
@ 2022-12-09 15:05 ` Reid Thompson <[email protected]>
0 siblings, 0 replies; 22+ messages in thread
From: Reid Thompson @ 2022-12-09 15:05 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: [email protected]; Arne Roland <[email protected]>; [email protected] <[email protected]>; Justin Pryzby <[email protected]>; Ibrar Ahmed <[email protected]>
On Tue, 2022-12-06 at 10:32 -0800, Andres Freund wrote:
> Hi,
>
> On 2022-11-26 22:22:15 -0500, Reid Thompson wrote:
> > rebased/patched to current master && current pg-stat-activity-
> > backend-memory-allocated
>
> This version fails to build with msvc, and builds with warnings on
> other
> platforms.
> https://cirrus-ci.com/build/5410696721072128
> msvc:
>
> Andres Freund
updated patches
--
Reid Thompson
Senior Software Engineer
Crunchy Data, Inc.
[email protected]
www.crunchydata.com
Attachments:
[text/x-patch] 0002-Add-the-ability-to-limit-the-amount-of-memory-that-c.patch (14.1K, ../../[email protected]/2-0002-Add-the-ability-to-limit-the-amount-of-memory-that-c.patch)
download | inline diff:
From e48292eaf402bfe397f1c2fdc1b3efd8cd0a9137 Mon Sep 17 00:00:00 2001
From: Reid Thompson <[email protected]>
Date: Sat, 4 Jun 2022 22:23:59 -0400
Subject: [PATCH 2/2] Add the ability to limit the amount of memory that can be
allocated to backends.
This builds on the work that adds backend memory allocated to pg_stat_activity.
Add GUC variable max_total_backend_memory.
Specifies a limit to the amount of memory (in MB) that may be allocated to
backends in total (i.e. this is not a per user or per backend limit). If unset,
or set to 0 it is disabled. It is intended as a resource to help avoid the OOM
killer on LINUX and manage resources in general. A backend request that would
push the total over the limit will be denied with an out of memory error causing
that backend's current query/transaction to fail. Due to the dynamic nature of
memory allocations, this limit is not exact. If within 1.5MB of the limit and
two backends request 1MB each at the same time both may be allocated, and exceed
the limit. Further requests will not be allocated until dropping below the
limit. Keep this in mind when setting this value. This limit does not affect
auxiliary backend processes. Backend memory allocations are displayed in the
pg_stat_activity view.
---
doc/src/sgml/config.sgml | 26 +++++
src/backend/storage/ipc/dsm_impl.c | 12 ++
src/backend/utils/activity/backend_status.c | 108 ++++++++++++++++++
src/backend/utils/misc/guc_tables.c | 11 ++
src/backend/utils/misc/postgresql.conf.sample | 3 +
src/backend/utils/mmgr/aset.c | 17 +++
src/backend/utils/mmgr/generation.c | 9 ++
src/backend/utils/mmgr/slab.c | 8 ++
src/include/utils/backend_status.h | 3 +
9 files changed, 197 insertions(+)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index ff6fcd902a..2899f109ac 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -2079,6 +2079,32 @@ include_dir 'conf.d'
</listitem>
</varlistentry>
+ <varlistentry id="guc-max-total-backend-memory" xreflabel="max_total_backend_memory">
+ <term><varname>max_total_backend_memory</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>max_total_backend_memory</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies a limit to the amount of memory (MB) that may be allocated to
+ backends in total (i.e. this is not a per user or per backend limit).
+ If unset, or set to 0 it is disabled. A backend request that would
+ push the total over the limit will be denied with an out of memory
+ error causing that backend's current query/transaction to fail. Due to
+ the dynamic nature of memory allocations, this limit is not exact. If
+ within 1.5MB of the limit and two backends request 1MB each at the same
+ time both may be allocated, and exceed the limit. Further requests will
+ not be allocated until dropping below the limit. Keep this in mind when
+ setting this value. This limit does not affect auxiliary backend
+ processes <xref linkend="glossary-auxiliary-proc"/> . Backend memory
+ allocations (<varname>allocated_bytes</varname>) are displayed in the
+ <link linkend="monitoring-pg-stat-activity-view"><structname>pg_stat_activity</structname></link>
+ view.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect2>
diff --git a/src/backend/storage/ipc/dsm_impl.c b/src/backend/storage/ipc/dsm_impl.c
index 36ef3e425e..58fb690c69 100644
--- a/src/backend/storage/ipc/dsm_impl.c
+++ b/src/backend/storage/ipc/dsm_impl.c
@@ -254,6 +254,10 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
return true;
}
+ /* Do not exceed maximum allowed memory allocation */
+ if (op == DSM_OP_CREATE && exceeds_max_total_bkend_mem(request_size))
+ return false;
+
/*
* Create new segment or open an existing one for attach.
*
@@ -525,6 +529,10 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size,
int flags = IPCProtection;
size_t segsize;
+ /* Do not exceed maximum allowed memory allocation */
+ if (op == DSM_OP_CREATE && exceeds_max_total_bkend_mem(request_size))
+ return false;
+
/*
* Allocate the memory BEFORE acquiring the resource, so that we don't
* leak the resource if memory allocation fails.
@@ -719,6 +727,10 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size,
return true;
}
+ /* Do not exceed maximum allowed memory allocation */
+ if (op == DSM_OP_CREATE && exceeds_max_total_bkend_mem(request_size))
+ return false;
+
/* Create new segment or open an existing one for attach. */
if (op == DSM_OP_CREATE)
{
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 3785e8af53..07dfd8f490 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -45,6 +45,9 @@
bool pgstat_track_activities = false;
int pgstat_track_activity_query_size = 1024;
+/* Max backend memory allocation allowed (MB). 0 = disabled */
+int max_total_bkend_mem = 0;
+
/* exposed so that backend_progress.c can access it */
PgBackendStatus *MyBEEntry = NULL;
@@ -1236,3 +1239,108 @@ pgstat_reset_allocated_bytes_storage(void)
my_allocated_bytes = &local_my_allocated_bytes;
}
+/* ----------
+ * pgstat_get_all_memory_allocated() -
+ *
+ * Return a uint64 representing the current shared memory allocated to all
+ * backends. This looks directly at the BackendStatusArray, and so will
+ * provide current information regardless of the age of our transaction's
+ * snapshot of the status array.
+ * In the future we will likely utilize additional values - perhaps limit
+ * backend allocation by user/role, etc.
+ * ----------
+ */
+uint64
+pgstat_get_all_backend_memory_allocated(void)
+{
+ PgBackendStatus *beentry;
+ int i;
+ uint64 all_memory_allocated = 0;
+
+ beentry = BackendStatusArray;
+
+ /*
+ * We probably shouldn't get here before shared memory has been set up,
+ * but be safe.
+ */
+ if (beentry == NULL || BackendActivityBuffer == NULL)
+ return 0;
+
+ /*
+ * We include AUX procs in all backend memory calculation
+ */
+ for (i = 1; i <= NumBackendStatSlots; i++)
+ {
+ /*
+ * We use a volatile pointer here to ensure the compiler doesn't try
+ * to get cute.
+ */
+ volatile PgBackendStatus *vbeentry = beentry;
+ bool found;
+ uint64 allocated_bytes = 0;
+
+ for (;;)
+ {
+ int before_changecount;
+ int after_changecount;
+
+ pgstat_begin_read_activity(vbeentry, before_changecount);
+
+ /*
+ * Ignore invalid entries, which may contain invalid data.
+ * See pgstat_beshutdown_hook()
+ */
+ if (vbeentry->st_procpid > 0)
+ allocated_bytes = vbeentry->allocated_bytes;
+
+ pgstat_end_read_activity(vbeentry, after_changecount);
+
+ if ((found = pgstat_read_activity_complete(before_changecount,
+ after_changecount)))
+ break;
+
+ /* Make sure we can break out of loop if stuck... */
+ CHECK_FOR_INTERRUPTS();
+ }
+
+ if (found)
+ all_memory_allocated += allocated_bytes;
+
+ beentry++;
+ }
+
+ return all_memory_allocated;
+}
+
+/*
+ * Determine if allocation request will exceed max backend memory allowed.
+ * Do not apply to auxiliary processes.
+ */
+bool
+exceeds_max_total_bkend_mem(uint64 allocation_request)
+{
+ bool result = false;
+
+ /* Exclude auxiliary processes from the check */
+ if (MyAuxProcType != NotAnAuxProcess)
+ return result;
+
+ /* Convert max_total_bkend_mem to bytes for comparison */
+ if (max_total_bkend_mem &&
+ pgstat_get_all_backend_memory_allocated() +
+ allocation_request > (uint64) max_total_bkend_mem * 1024 * 1024)
+ {
+ /*
+ * Explicitly identify the OOM being a result of this configuration
+ * parameter vs a system failure to allocate OOM.
+ */
+ ereport(WARNING,
+ errmsg("allocation would exceed max_total_memory limit (%llu > %llu)",
+ (unsigned long long) pgstat_get_all_backend_memory_allocated() +
+ allocation_request, (unsigned long long) max_total_bkend_mem * 1024 * 1024));
+
+ result = true;
+ }
+
+ return result;
+}
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 1bf14eec66..bfde338a8d 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -3423,6 +3423,17 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_total_backend_memory", PGC_SU_BACKEND, RESOURCES_MEM,
+ gettext_noop("Restrict total backend memory allocations to this max."),
+ gettext_noop("0 turns this feature off."),
+ GUC_UNIT_MB
+ },
+ &max_total_bkend_mem,
+ 0, 0, INT_MAX,
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, 0, 0, 0, NULL, NULL, NULL
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 043864597f..75ea0c9af1 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -155,6 +155,9 @@
# mmap
# (change requires restart)
#min_dynamic_shared_memory = 0MB # (change requires restart)
+#max_total_backend_memory = 0MB # Restrict total backend memory allocations
+ # to this max (in MB). 0 turns this feature
+ # off.
# - Disk -
diff --git a/src/backend/utils/mmgr/aset.c b/src/backend/utils/mmgr/aset.c
index cc10dfd609..3ce2191555 100644
--- a/src/backend/utils/mmgr/aset.c
+++ b/src/backend/utils/mmgr/aset.c
@@ -440,6 +440,10 @@ AllocSetContextCreateInternal(MemoryContext parent,
else
firstBlockSize = Max(firstBlockSize, initBlockSize);
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(firstBlockSize))
+ return NULL;
+
/*
* Allocate the initial block. Unlike other aset.c blocks, it starts with
* the context header and its block header follows that.
@@ -741,6 +745,11 @@ AllocSetAlloc(MemoryContext context, Size size)
#endif
blksize = chunk_size + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ;
+
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
block = (AllocBlock) malloc(blksize);
if (block == NULL)
return NULL;
@@ -938,6 +947,10 @@ AllocSetAlloc(MemoryContext context, Size size)
while (blksize < required_size)
blksize <<= 1;
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
/* Try to allocate it */
block = (AllocBlock) malloc(blksize);
@@ -1178,6 +1191,10 @@ AllocSetRealloc(void *pointer, Size size)
blksize = chksize + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ;
oldblksize = block->endptr - ((char *) block);
+ /* Do not exceed maximum allowed memory allocation */
+ if (blksize > oldblksize && exceeds_max_total_bkend_mem(blksize - oldblksize))
+ return NULL;
+
block = (AllocBlock) realloc(block, blksize);
if (block == NULL)
{
diff --git a/src/backend/utils/mmgr/generation.c b/src/backend/utils/mmgr/generation.c
index b0460d97c2..3183c9a8dc 100644
--- a/src/backend/utils/mmgr/generation.c
+++ b/src/backend/utils/mmgr/generation.c
@@ -201,6 +201,9 @@ GenerationContextCreate(MemoryContext parent,
else
allocSize = Max(allocSize, initBlockSize);
+ if (exceeds_max_total_bkend_mem(allocSize))
+ return NULL;
+
/*
* Allocate the initial block. Unlike other generation.c blocks, it
* starts with the context header and its block header follows that.
@@ -380,6 +383,9 @@ GenerationAlloc(MemoryContext context, Size size)
{
Size blksize = required_size + Generation_BLOCKHDRSZ;
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
block = (GenerationBlock *) malloc(blksize);
if (block == NULL)
return NULL;
@@ -483,6 +489,9 @@ GenerationAlloc(MemoryContext context, Size size)
if (blksize < required_size)
blksize = pg_nextpower2_size_t(required_size);
+ if (exceeds_max_total_bkend_mem(blksize))
+ return NULL;
+
block = (GenerationBlock *) malloc(blksize);
if (block == NULL)
diff --git a/src/backend/utils/mmgr/slab.c b/src/backend/utils/mmgr/slab.c
index 08bb013f7c..2074f3853a 100644
--- a/src/backend/utils/mmgr/slab.c
+++ b/src/backend/utils/mmgr/slab.c
@@ -197,6 +197,10 @@ SlabContextCreate(MemoryContext parent,
headerSize += chunksPerBlock * sizeof(bool);
#endif
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(headerSize))
+ return NULL;
+
slab = (SlabContext *) malloc(headerSize);
if (slab == NULL)
{
@@ -351,6 +355,10 @@ SlabAlloc(MemoryContext context, Size size)
*/
if (slab->minFreeChunks == 0)
{
+ /* Do not exceed maximum allowed memory allocation */
+ if (exceeds_max_total_bkend_mem(slab->blockSize))
+ return NULL;
+
block = (SlabBlock *) malloc(slab->blockSize);
if (block == NULL)
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index 3ba479cb0d..1787d75ace 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -286,6 +286,7 @@ typedef struct LocalPgBackendStatus
*/
extern PGDLLIMPORT bool pgstat_track_activities;
extern PGDLLIMPORT int pgstat_track_activity_query_size;
+extern PGDLLIMPORT int max_total_bkend_mem;
/* ----------
@@ -325,6 +326,7 @@ extern const char *pgstat_get_backend_current_activity(int pid, bool checkUser);
extern const char *pgstat_get_crashed_backend_activity(int pid, char *buffer,
int buflen);
extern uint64 pgstat_get_my_query_id(void);
+extern uint64 pgstat_get_all_backend_memory_allocated(void);
extern void pgstat_set_allocated_bytes_storage(uint64 *allocated_bytes);
extern void pgstat_reset_allocated_bytes_storage(void);
@@ -337,6 +339,7 @@ extern int pgstat_fetch_stat_numbackends(void);
extern PgBackendStatus *pgstat_fetch_stat_beentry(BackendId beid);
extern LocalPgBackendStatus *pgstat_fetch_stat_local_beentry(int beid);
extern char *pgstat_clip_activity(const char *raw_activity);
+extern bool exceeds_max_total_bkend_mem(uint64 allocation_request);
/* ----------
* pgstat_report_allocated_bytes() -
--
2.25.1
[text/x-patch] 0001-Add-tracking-of-backend-memory-allocated-to-pg_stat_.patch (33.1K, ../../[email protected]/3-0001-Add-tracking-of-backend-memory-allocated-to-pg_stat_.patch)
download | inline diff:
From fdb7e6d5bb653e9c5031fd058bf168bdf80a20eb Mon Sep 17 00:00:00 2001
From: Reid Thompson <[email protected]>
Date: Thu, 11 Aug 2022 12:01:25 -0400
Subject: [PATCH 1/2] Add tracking of backend memory allocated to
pg_stat_activity
This new field displays the current bytes of memory allocated to the
backend process. It is updated as memory for the process is
malloc'd/free'd. Memory allocated to items on the freelist is included in
the displayed value. Dynamic shared memory allocations are included
only in the value displayed for the backend that created them, they are
not included in the value for backends that are attached to them to
avoid double counting. On occasion, orphaned memory segments may be
cleaned up on postmaster startup. This may result in decreasing the sum
without a prior increment. We limit the floor of backend_mem_allocated
to zero. Updated pg_stat_activity documentation for the new column.
---
doc/src/sgml/monitoring.sgml | 15 ++++
src/backend/catalog/system_views.sql | 1 +
src/backend/postmaster/autovacuum.c | 6 ++
src/backend/postmaster/postmaster.c | 13 ++++
src/backend/postmaster/syslogger.c | 3 +
src/backend/storage/ipc/dsm_impl.c | 81 +++++++++++++++++++++
src/backend/utils/activity/backend_status.c | 45 ++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 4 +-
src/backend/utils/mmgr/aset.c | 17 +++++
src/backend/utils/mmgr/generation.c | 15 ++++
src/backend/utils/mmgr/slab.c | 21 ++++++
src/include/catalog/pg_proc.dat | 6 +-
src/include/utils/backend_status.h | 59 ++++++++++++++-
src/test/regress/expected/rules.out | 9 ++-
src/test/regress/expected/stats.out | 11 +++
src/test/regress/sql/stats.sql | 3 +
16 files changed, 300 insertions(+), 9 deletions(-)
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 11a8ebe5ec..13ecbe5877 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -948,6 +948,21 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>allocated_bytes</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Memory currently allocated to this backend in bytes. This is the balance
+ of bytes allocated and freed by this backend. Dynamic shared memory
+ allocations are included only in the value displayed for the backend that
+ created them, they are not included in the value for backends that are
+ attached to them to avoid double counting. Use <function>pg_size_pretty</function>
+ described in <xref linkend="functions-admin-dbsize"/> to make this value
+ more easily readable.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>query</structfield> <type>text</type>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 2d8104b090..9ea8f78c95 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -864,6 +864,7 @@ CREATE VIEW pg_stat_activity AS
S.state,
S.backend_xid,
s.backend_xmin,
+ S.allocated_bytes,
S.query_id,
S.query,
S.backend_type
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 0746d80224..f6b6f71cdb 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -407,6 +407,9 @@ StartAutoVacLauncher(void)
#ifndef EXEC_BACKEND
case 0:
+ /* Zero allocated bytes to avoid double counting parent allocation */
+ pgstat_zero_my_allocated_bytes();
+
/* in postmaster child ... */
InitPostmasterChild();
@@ -1485,6 +1488,9 @@ StartAutoVacWorker(void)
#ifndef EXEC_BACKEND
case 0:
+ /* Zero allocated bytes to avoid double counting parent allocation */
+ pgstat_zero_my_allocated_bytes();
+
/* in postmaster child ... */
InitPostmasterChild();
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index a8a246921f..89a6caec78 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -4102,6 +4102,9 @@ BackendStartup(Port *port)
{
free(bn);
+ /* Zero allocated bytes to avoid double counting parent allocation */
+ pgstat_zero_my_allocated_bytes();
+
/* Detangle from postmaster */
InitPostmasterChild();
@@ -5307,6 +5310,11 @@ StartChildProcess(AuxProcType type)
MemoryContextDelete(PostmasterContext);
PostmasterContext = NULL;
+ /* Zero allocated bytes to avoid double counting parent allocation.
+ * Needs to be after the MemoryContextDelete(PostmasterContext) above.
+ */
+ pgstat_zero_my_allocated_bytes();
+
AuxiliaryProcessMain(type); /* does not return */
}
#endif /* EXEC_BACKEND */
@@ -5700,6 +5708,11 @@ do_start_bgworker(RegisteredBgWorker *rw)
MemoryContextDelete(PostmasterContext);
PostmasterContext = NULL;
+ /* Zero allocated bytes to avoid double counting parent allocation.
+ * Needs to be after the MemoryContextDelete(PostmasterContext) above.
+ */
+ pgstat_zero_my_allocated_bytes();
+
StartBackgroundWorker();
exit(1); /* should not get here */
diff --git a/src/backend/postmaster/syslogger.c b/src/backend/postmaster/syslogger.c
index d6d02e3c63..4cbc59cda5 100644
--- a/src/backend/postmaster/syslogger.c
+++ b/src/backend/postmaster/syslogger.c
@@ -679,6 +679,9 @@ SysLogger_Start(void)
#ifndef EXEC_BACKEND
case 0:
+ /* Zero allocated bytes to avoid double counting parent allocation */
+ pgstat_zero_my_allocated_bytes();
+
/* in postmaster child ... */
InitPostmasterChild();
diff --git a/src/backend/storage/ipc/dsm_impl.c b/src/backend/storage/ipc/dsm_impl.c
index 6ddd46a4e7..36ef3e425e 100644
--- a/src/backend/storage/ipc/dsm_impl.c
+++ b/src/backend/storage/ipc/dsm_impl.c
@@ -66,6 +66,7 @@
#include "postmaster/postmaster.h"
#include "storage/dsm_impl.h"
#include "storage/fd.h"
+#include "utils/backend_status.h"
#include "utils/guc.h"
#include "utils/memutils.h"
@@ -232,6 +233,14 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ if (op == DSM_OP_DESTROY)
+ pgstat_report_allocated_bytes(*mapped_size, PG_ALLOC_DECREASE);
*mapped_address = NULL;
*mapped_size = 0;
if (op == DSM_OP_DESTROY && shm_unlink(name) != 0)
@@ -332,6 +341,36 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ {
+ /*
+ * Posix creation calls dsm_impl_posix_resize implying that resizing
+ * occurs or may be added in the future. As implemented
+ * dsm_impl_posix_resize utilizes fallocate or truncate, passing the
+ * whole new size as input, growing the allocation as needed (only
+ * truncate supports shrinking). We update by replacing the old
+ * allocation with the new.
+ */
+#if defined(HAVE_POSIX_FALLOCATE) && defined(__linux__)
+ /*
+ * posix_fallocate does not shrink allocations, adjust only on
+ * allocation increase.
+ */
+ if (request_size > *mapped_size)
+ {
+ pgstat_report_allocated_bytes(request_size - *mapped_size,
+ PG_ALLOC_INCREASE);
+ }
+#else
+ pgstat_report_allocated_bytes(*mapped_size, PG_ALLOC_DECREASE);
+ pgstat_report_allocated_bytes(request_size, PG_ALLOC_INCREASE);
+#endif
+ }
*mapped_address = address;
*mapped_size = request_size;
close(fd);
@@ -537,6 +576,14 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ if (op == DSM_OP_DESTROY)
+ pgstat_report_allocated_bytes(*mapped_size, PG_ALLOC_DECREASE);
*mapped_address = NULL;
*mapped_size = 0;
if (op == DSM_OP_DESTROY && shmctl(ident, IPC_RMID, NULL) < 0)
@@ -584,6 +631,13 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ pgstat_report_allocated_bytes(request_size, PG_ALLOC_INCREASE);
*mapped_address = address;
*mapped_size = request_size;
@@ -652,6 +706,13 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size,
return false;
}
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ if (op == DSM_OP_DESTROY)
+ pgstat_report_allocated_bytes(*mapped_size, PG_ALLOC_DECREASE);
*impl_private = NULL;
*mapped_address = NULL;
*mapped_size = 0;
@@ -768,6 +829,12 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size,
return false;
}
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ pgstat_report_allocated_bytes(info.RegionSize, PG_ALLOC_INCREASE);
*mapped_address = address;
*mapped_size = info.RegionSize;
*impl_private = hmap;
@@ -812,6 +879,13 @@ dsm_impl_mmap(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Detach and destroy pass through here, only decrease the memory
+ * shown allocated in pg_stat_activity when the creator destroys the
+ * allocation.
+ */
+ pgstat_report_allocated_bytes(*mapped_size, PG_ALLOC_DECREASE);
*mapped_address = NULL;
*mapped_size = 0;
if (op == DSM_OP_DESTROY && unlink(name) != 0)
@@ -933,6 +1007,13 @@ dsm_impl_mmap(dsm_op op, dsm_handle handle, Size request_size,
name)));
return false;
}
+
+ /*
+ * Attach and create pass through here, only update backend memory
+ * allocated in pg_stat_activity for the creator process.
+ */
+ if (op == DSM_OP_CREATE)
+ pgstat_report_allocated_bytes(request_size, PG_ALLOC_INCREASE);
*mapped_address = address;
*mapped_size = request_size;
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 1146a6c33c..3785e8af53 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -49,6 +49,9 @@ int pgstat_track_activity_query_size = 1024;
/* exposed so that backend_progress.c can access it */
PgBackendStatus *MyBEEntry = NULL;
+/* Memory allocated to this backend prior to pgstats initialization */
+uint64 local_my_allocated_bytes = 0;
+uint64 *my_allocated_bytes = &local_my_allocated_bytes;
static PgBackendStatus *BackendStatusArray = NULL;
static char *BackendAppnameBuffer = NULL;
@@ -400,6 +403,15 @@ pgstat_bestart(void)
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = UINT64CONST(0);
+ /* Alter allocation reporting from local_my_allocated_bytes to shared memory */
+ pgstat_set_allocated_bytes_storage(&MyBEEntry->allocated_bytes);
+
+ /* Populate sum of memory allocated prior to pgstats initialization to pgstats
+ * and zero the local variable.
+ */
+ lbeentry.allocated_bytes += local_my_allocated_bytes;
+ local_my_allocated_bytes = 0;
+
/*
* we don't zero st_progress_param here to save cycles; nobody should
* examine it until st_progress_command has been set to something other
@@ -459,6 +471,11 @@ pgstat_beshutdown_hook(int code, Datum arg)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ /*
+ * Stop reporting memory allocation changes to &MyBEEntry->allocated_bytes
+ */
+ pgstat_reset_allocated_bytes_storage();
+
/*
* Clear my status entry, following the protocol of bumping st_changecount
* before and after. We use a volatile pointer here to ensure the
@@ -1191,3 +1208,31 @@ pgstat_clip_activity(const char *raw_activity)
return activity;
}
+
+/*
+ * Configure bytes allocated reporting to report allocated bytes to
+ * *allocated_bytes. *allocated_bytes needs to be valid until
+ * pgstat_set_allocated_bytes_storage() is called.
+ *
+ * Expected to be called during backend startup (in pgstat_bestart), to point
+ * my_allocated_bytes into shared memory.
+ */
+void
+pgstat_set_allocated_bytes_storage(uint64 *new_allocated_bytes)
+{
+ my_allocated_bytes = new_allocated_bytes;
+ *new_allocated_bytes = local_my_allocated_bytes;
+}
+
+/*
+ * Reset allocated bytes storage location.
+ *
+ * Expected to be called during backend shutdown, before the location set up
+ * by pgstat_set_allocated_bytes_storage() becomes invalid.
+ */
+void
+pgstat_reset_allocated_bytes_storage(void)
+{
+ my_allocated_bytes = &local_my_allocated_bytes;
+}
+
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 25a159b5e5..c4350cfa50 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -302,7 +302,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum
pg_stat_get_activity(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_ACTIVITY_COLS 30
+#define PG_STAT_GET_ACTIVITY_COLS 31
int num_backends = pgstat_fetch_stat_numbackends();
int curr_backend;
int pid = PG_ARGISNULL(0) ? -1 : PG_GETARG_INT32(0);
@@ -358,6 +358,8 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
else
nulls[16] = true;
+ values[30] = UInt64GetDatum(beentry->allocated_bytes);
+
/* Values only available to role member or pg_read_all_stats */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
diff --git a/src/backend/utils/mmgr/aset.c b/src/backend/utils/mmgr/aset.c
index b6a8bbcd59..cc10dfd609 100644
--- a/src/backend/utils/mmgr/aset.c
+++ b/src/backend/utils/mmgr/aset.c
@@ -47,6 +47,7 @@
#include "postgres.h"
#include "port/pg_bitutils.h"
+#include "utils/backend_status.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
#include "utils/memutils_memorychunk.h"
@@ -521,6 +522,7 @@ AllocSetContextCreateInternal(MemoryContext parent,
name);
((MemoryContext) set)->mem_allocated = firstBlockSize;
+ pgstat_report_allocated_bytes(firstBlockSize, PG_ALLOC_INCREASE);
return (MemoryContext) set;
}
@@ -543,6 +545,7 @@ AllocSetReset(MemoryContext context)
AllocSet set = (AllocSet) context;
AllocBlock block;
Size keepersize PG_USED_FOR_ASSERTS_ONLY;
+ uint64 deallocation = 0;
Assert(AllocSetIsValid(set));
@@ -585,6 +588,7 @@ AllocSetReset(MemoryContext context)
{
/* Normal case, release the block */
context->mem_allocated -= block->endptr - ((char *) block);
+ deallocation += block->endptr - ((char *) block);
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
@@ -595,6 +599,7 @@ AllocSetReset(MemoryContext context)
}
Assert(context->mem_allocated == keepersize);
+ pgstat_report_allocated_bytes(deallocation, PG_ALLOC_DECREASE);
/* Reset block size allocation sequence, too */
set->nextBlockSize = set->initBlockSize;
@@ -613,6 +618,7 @@ AllocSetDelete(MemoryContext context)
AllocSet set = (AllocSet) context;
AllocBlock block = set->blocks;
Size keepersize PG_USED_FOR_ASSERTS_ONLY;
+ uint64 deallocation = 0;
Assert(AllocSetIsValid(set));
@@ -651,11 +657,13 @@ AllocSetDelete(MemoryContext context)
freelist->first_free = (AllocSetContext *) oldset->header.nextchild;
freelist->num_free--;
+ deallocation += oldset->header.mem_allocated;
/* All that remains is to free the header/initial block */
free(oldset);
}
Assert(freelist->num_free == 0);
+ pgstat_report_allocated_bytes(deallocation, PG_ALLOC_DECREASE);
}
/* Now add the just-deleted context to the freelist. */
@@ -672,7 +680,10 @@ AllocSetDelete(MemoryContext context)
AllocBlock next = block->next;
if (block != set->keeper)
+ {
context->mem_allocated -= block->endptr - ((char *) block);
+ deallocation += block->endptr - ((char *) block);
+ }
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
@@ -685,6 +696,7 @@ AllocSetDelete(MemoryContext context)
}
Assert(context->mem_allocated == keepersize);
+ pgstat_report_allocated_bytes(deallocation + context->mem_allocated, PG_ALLOC_DECREASE);
/* Finally, free the context header, including the keeper block */
free(set);
@@ -734,6 +746,7 @@ AllocSetAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_allocated_bytes(blksize, PG_ALLOC_INCREASE);
block->aset = set;
block->freeptr = block->endptr = ((char *) block) + blksize;
@@ -944,6 +957,7 @@ AllocSetAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_allocated_bytes(blksize, PG_ALLOC_INCREASE);
block->aset = set;
block->freeptr = ((char *) block) + ALLOC_BLOCKHDRSZ;
@@ -1043,6 +1057,7 @@ AllocSetFree(void *pointer)
block->next->prev = block->prev;
set->header.mem_allocated -= block->endptr - ((char *) block);
+ pgstat_report_allocated_bytes(block->endptr - ((char *) block), PG_ALLOC_DECREASE);
#ifdef CLOBBER_FREED_MEMORY
wipe_mem(block, block->freeptr - ((char *) block));
@@ -1173,7 +1188,9 @@ AllocSetRealloc(void *pointer, Size size)
/* updated separately, not to underflow when (oldblksize > blksize) */
set->header.mem_allocated -= oldblksize;
+ pgstat_report_allocated_bytes(oldblksize, PG_ALLOC_DECREASE);
set->header.mem_allocated += blksize;
+ pgstat_report_allocated_bytes(blksize, PG_ALLOC_INCREASE);
block->freeptr = block->endptr = ((char *) block) + blksize;
diff --git a/src/backend/utils/mmgr/generation.c b/src/backend/utils/mmgr/generation.c
index b432a92be3..b0460d97c2 100644
--- a/src/backend/utils/mmgr/generation.c
+++ b/src/backend/utils/mmgr/generation.c
@@ -37,6 +37,7 @@
#include "lib/ilist.h"
#include "port/pg_bitutils.h"
+#include "utils/backend_status.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
#include "utils/memutils_memorychunk.h"
@@ -267,6 +268,7 @@ GenerationContextCreate(MemoryContext parent,
name);
((MemoryContext) set)->mem_allocated = firstBlockSize;
+ pgstat_report_allocated_bytes(firstBlockSize, PG_ALLOC_INCREASE);
return (MemoryContext) set;
}
@@ -283,6 +285,7 @@ GenerationReset(MemoryContext context)
{
GenerationContext *set = (GenerationContext *) context;
dlist_mutable_iter miter;
+ uint64 deallocation = 0;
Assert(GenerationIsValid(set));
@@ -305,9 +308,14 @@ GenerationReset(MemoryContext context)
if (block == set->keeper)
GenerationBlockMarkEmpty(block);
else
+ {
+ deallocation += block->blksize;
GenerationBlockFree(set, block);
+ }
}
+ pgstat_report_allocated_bytes(deallocation, PG_ALLOC_DECREASE);
+
/* set it so new allocations to make use of the keeper block */
set->block = set->keeper;
@@ -328,6 +336,9 @@ GenerationDelete(MemoryContext context)
{
/* Reset to release all releasable GenerationBlocks */
GenerationReset(context);
+
+ pgstat_report_allocated_bytes(context->mem_allocated, PG_ALLOC_DECREASE);
+
/* And free the context header and keeper block */
free(context);
}
@@ -374,6 +385,7 @@ GenerationAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_allocated_bytes(blksize, PG_ALLOC_INCREASE);
/* block with a single (used) chunk */
block->context = set;
@@ -477,6 +489,7 @@ GenerationAlloc(MemoryContext context, Size size)
return NULL;
context->mem_allocated += blksize;
+ pgstat_report_allocated_bytes(blksize, PG_ALLOC_INCREASE);
/* initialize the new block */
GenerationBlockInit(set, block, blksize);
@@ -726,6 +739,8 @@ GenerationFree(void *pointer)
dlist_delete(&block->node);
set->header.mem_allocated -= block->blksize;
+ pgstat_report_allocated_bytes(block->blksize, PG_ALLOC_DECREASE);
+
free(block);
}
diff --git a/src/backend/utils/mmgr/slab.c b/src/backend/utils/mmgr/slab.c
index 6df0839b6a..08bb013f7c 100644
--- a/src/backend/utils/mmgr/slab.c
+++ b/src/backend/utils/mmgr/slab.c
@@ -53,6 +53,7 @@
#include "postgres.h"
#include "lib/ilist.h"
+#include "utils/backend_status.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
#include "utils/memutils_memorychunk.h"
@@ -238,6 +239,12 @@ SlabContextCreate(MemoryContext parent,
parent,
name);
+ /*
+ * If SlabContextCreate is updated to add headerSize to
+ * context->mem_allocated, then update here and SlabDelete appropriately
+ */
+ pgstat_report_allocated_bytes(headerSize, PG_ALLOC_INCREASE);
+
return (MemoryContext) slab;
}
@@ -253,6 +260,7 @@ SlabReset(MemoryContext context)
{
SlabContext *slab = (SlabContext *) context;
int i;
+ uint64 deallocation = 0;
Assert(SlabIsValid(slab));
@@ -278,9 +286,11 @@ SlabReset(MemoryContext context)
free(block);
slab->nblocks--;
context->mem_allocated -= slab->blockSize;
+ deallocation += slab->blockSize;
}
}
+ pgstat_report_allocated_bytes(deallocation, PG_ALLOC_DECREASE);
slab->minFreeChunks = 0;
Assert(slab->nblocks == 0);
@@ -294,8 +304,17 @@ SlabReset(MemoryContext context)
void
SlabDelete(MemoryContext context)
{
+ /*
+ * Until header allocation is included in context->mem_allocated, cast to
+ * slab and decrement the headerSize
+ */
+ SlabContext *slab = castNode(SlabContext, context);
+
/* Reset to release all the SlabBlocks */
SlabReset(context);
+
+ pgstat_report_allocated_bytes(slab->headerSize, PG_ALLOC_DECREASE);
+
/* And free the context header */
free(context);
}
@@ -364,6 +383,7 @@ SlabAlloc(MemoryContext context, Size size)
slab->minFreeChunks = slab->chunksPerBlock;
slab->nblocks += 1;
context->mem_allocated += slab->blockSize;
+ pgstat_report_allocated_bytes(slab->blockSize, PG_ALLOC_INCREASE);
}
/* grab the block from the freelist (even the new block is there) */
@@ -537,6 +557,7 @@ SlabFree(void *pointer)
free(block);
slab->nblocks--;
slab->header.mem_allocated -= slab->blockSize;
+ pgstat_report_allocated_bytes(slab->blockSize, PG_ALLOC_DECREASE);
}
else
dlist_push_head(&slab->freelist[block->nfree], &block->node);
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f9301b2627..1bf02758d4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5373,9 +5373,9 @@
proname => 'pg_stat_get_activity', prorows => '100', proisstrict => 'f',
proretset => 't', provolatile => 's', proparallel => 'r',
prorettype => 'record', proargtypes => 'int4',
- proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,int4,int8}',
- proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,leader_pid,query_id}',
+ proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,int4,int8,int8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,leader_pid,query_id,allocated_bytes}',
prosrc => 'pg_stat_get_activity' },
{ oid => '3318',
descr => 'statistics: information about progress of backends running maintenance command',
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index b582b46e9f..3ba479cb0d 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -15,6 +15,7 @@
#include "miscadmin.h" /* for BackendType */
#include "storage/backendid.h"
#include "utils/backend_progress.h"
+#include "common/int.h"
/* ----------
@@ -32,6 +33,13 @@ typedef enum BackendState
STATE_DISABLED
} BackendState;
+/* Enum helper for reporting memory allocated bytes */
+enum allocation_direction
+{
+ PG_ALLOC_DECREASE = -1,
+ PG_ALLOC_IGNORE,
+ PG_ALLOC_INCREASE,
+};
/* ----------
* Shared-memory data structures
@@ -169,6 +177,9 @@ typedef struct PgBackendStatus
/* query identifier, optionally computed using post_parse_analyze_hook */
uint64 st_query_id;
+
+ /* Current memory allocated to this backend */
+ uint64 allocated_bytes;
} PgBackendStatus;
@@ -282,6 +293,7 @@ extern PGDLLIMPORT int pgstat_track_activity_query_size;
* ----------
*/
extern PGDLLIMPORT PgBackendStatus *MyBEEntry;
+extern PGDLLIMPORT uint64 *my_allocated_bytes;
/* ----------
@@ -313,7 +325,8 @@ extern const char *pgstat_get_backend_current_activity(int pid, bool checkUser);
extern const char *pgstat_get_crashed_backend_activity(int pid, char *buffer,
int buflen);
extern uint64 pgstat_get_my_query_id(void);
-
+extern void pgstat_set_allocated_bytes_storage(uint64 *allocated_bytes);
+extern void pgstat_reset_allocated_bytes_storage(void);
/* ----------
* Support functions for the SQL-callable functions to
@@ -325,5 +338,49 @@ extern PgBackendStatus *pgstat_fetch_stat_beentry(BackendId beid);
extern LocalPgBackendStatus *pgstat_fetch_stat_local_beentry(int beid);
extern char *pgstat_clip_activity(const char *raw_activity);
+/* ----------
+ * pgstat_report_allocated_bytes() -
+ *
+ * Called to report change in memory allocated for this backend.
+ *
+ * my_allocated_bytes initially points to local memory, making it safe to call
+ * this before pgstats has been initialized. allocation_direction is a
+ * positive/negative multiplier enum defined above.
+ * ----------
+ */
+static inline void
+pgstat_report_allocated_bytes(int64 allocated_bytes, int allocation_direction)
+{
+ uint64 temp;
+
+ /* Avoid *my_allocated_bytes unsigned integer overflow on PG_ALLOC_DECREASE */
+ if (allocation_direction == PG_ALLOC_DECREASE &&
+ pg_sub_u64_overflow(*my_allocated_bytes, allocated_bytes, &temp))
+ {
+ *my_allocated_bytes = 0;
+ ereport(LOG,
+ errmsg("Backend %d deallocated %lld bytes, exceeding the %llu bytes it is currently reporting allocated. Setting reported to 0.",
+ MyProcPid, (long long)allocated_bytes, (unsigned long long)*my_allocated_bytes));
+ }
+ else
+ *my_allocated_bytes += (allocated_bytes) * allocation_direction;
+
+ return;
+}
+
+/* ---------
+ * pgstat_zero_my_allocated_bytes() -
+ *
+ * Called to zero out local allocated bytes variable after fork to avoid double
+ * counting allocations.
+ * ---------
+ */
+static inline void
+pgstat_zero_my_allocated_bytes(void)
+{
+ *my_allocated_bytes = 0;
+
+ return;
+}
#endif /* BACKEND_STATUS_H */
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index fb9f936d43..5f854aab18 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1756,10 +1756,11 @@ pg_stat_activity| SELECT s.datid,
s.state,
s.backend_xid,
s.backend_xmin,
+ s.allocated_bytes,
s.query_id,
s.query,
s.backend_type
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, allocated_bytes)
LEFT JOIN pg_database d ON ((s.datid = d.oid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_all_indexes| SELECT c.oid AS relid,
@@ -1874,7 +1875,7 @@ pg_stat_gssapi| SELECT s.pid,
s.gss_auth AS gss_authenticated,
s.gss_princ AS principal,
s.gss_enc AS encrypted
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, allocated_bytes)
WHERE (s.client_port IS NOT NULL);
pg_stat_progress_analyze| SELECT s.pid,
s.datid,
@@ -2055,7 +2056,7 @@ pg_stat_replication| SELECT s.pid,
w.sync_priority,
w.sync_state,
w.reply_time
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, allocated_bytes)
JOIN pg_stat_get_wal_senders() w(pid, state, sent_lsn, write_lsn, flush_lsn, replay_lsn, write_lag, flush_lag, replay_lag, sync_priority, sync_state, reply_time) ON ((s.pid = w.pid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_replication_slots| SELECT s.slot_name,
@@ -2089,7 +2090,7 @@ pg_stat_ssl| SELECT s.pid,
s.ssl_client_dn AS client_dn,
s.ssl_client_serial AS client_serial,
s.ssl_issuer_dn AS issuer_dn
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id, allocated_bytes)
WHERE (s.client_port IS NOT NULL);
pg_stat_subscription| SELECT su.oid AS subid,
su.subname,
diff --git a/src/test/regress/expected/stats.out b/src/test/regress/expected/stats.out
index 1d84407a03..ab7e95c367 100644
--- a/src/test/regress/expected/stats.out
+++ b/src/test/regress/expected/stats.out
@@ -1126,4 +1126,15 @@ SELECT pg_stat_get_subscription_stats(NULL);
(1 row)
+-- ensure that allocated_bytes exist for backends
+SELECT allocated_bytes > 0 AS result FROM pg_stat_activity WHERE backend_type
+IN ('checkpointer', 'background writer', 'walwriter', 'autovacuum launcher');
+ result
+--------
+ t
+ t
+ t
+ t
+(4 rows)
+
-- End of Stats Test
diff --git a/src/test/regress/sql/stats.sql b/src/test/regress/sql/stats.sql
index b4d6753c71..2f0b1cc9d8 100644
--- a/src/test/regress/sql/stats.sql
+++ b/src/test/regress/sql/stats.sql
@@ -535,5 +535,8 @@ SET enable_seqscan TO on;
SELECT pg_stat_get_replication_slot(NULL);
SELECT pg_stat_get_subscription_stats(NULL);
+-- ensure that allocated_bytes exist for backends
+SELECT allocated_bytes > 0 AS result FROM pg_stat_activity WHERE backend_type
+IN ('checkpointer', 'background writer', 'walwriter', 'autovacuum launcher');
-- End of Stats Test
--
2.25.1
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: Add the ability to limit the amount of memory that can be allocated to backends.
2022-08-31 16:50 Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
@ 2022-09-01 02:48 ` Kyotaro Horiguchi <[email protected]>
2022-09-07 00:17 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
3 siblings, 1 reply; 22+ messages in thread
From: Kyotaro Horiguchi @ 2022-09-01 02:48 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]
At Wed, 31 Aug 2022 12:50:19 -0400, Reid Thompson <[email protected]> wrote in
> Hi Hackers,
>
> Add the ability to limit the amount of memory that can be allocated to
> backends.
The patch seems to limit both of memory-context allocations and DSM
allocations happen on a specific process by the same budget. In the
fist place I don't think it's sensible to cap the amount of DSM
allocations by per-process budget.
DSM is used by pgstats subsystem. There can be cases where pgstat
complains for denial of DSM allocation after the budget has been
exhausted by memory-context allocations, or every command complains
for denial of memory-context allocation after once the per-process
budget is exhausted by DSM allocations. That doesn't seem reasonable.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: Add the ability to limit the amount of memory that can be allocated to backends.
2022-08-31 16:50 Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-09-01 02:48 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Kyotaro Horiguchi <[email protected]>
@ 2022-09-07 00:17 ` Reid Thompson <[email protected]>
0 siblings, 0 replies; 22+ messages in thread
From: Reid Thompson @ 2022-09-07 00:17 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]
On Thu, 2022-09-01 at 11:48 +0900, Kyotaro Horiguchi wrote:
> >
> > The patch seems to limit both of memory-context allocations and DSM
> > allocations happen on a specific process by the same budget. In the
> > fist place I don't think it's sensible to cap the amount of DSM
> > allocations by per-process budget.
> >
> > DSM is used by pgstats subsystem. There can be cases where pgstat
> > complains for denial of DSM allocation after the budget has been
> > exhausted by memory-context allocations, or every command complains
> > for denial of memory-context allocation after once the per-process
> > budget is exhausted by DSM allocations. That doesn't seem
> > reasonable.
> > regards.
It's intended as a mechanism for administrators to limit total
postgresql memory consumption to avoid the OOM killer causing a crash
and restart, or to ensure that resources are available for other
processes on shared hosts, etc. It limits all types of allocations in
order to accomplish this. Our documentation will note this, so that
administrators that have the need to set it are aware that it can
affect all non-auxiliary processes and what the effect is.
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: Add the ability to limit the amount of memory that can be allocated to backends.
2022-08-31 16:50 Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
@ 2022-09-02 07:30 ` Drouvot, Bertrand <[email protected]>
2022-09-07 00:25 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
3 siblings, 1 reply; 22+ messages in thread
From: Drouvot, Bertrand @ 2022-09-02 07:30 UTC (permalink / raw)
To: [email protected]; pgsql-hackers <[email protected]>
Hi,
On 8/31/22 6:50 PM, Reid Thompson wrote:
> Hi Hackers,
>
> Add the ability to limit the amount of memory that can be allocated to
> backends.
Thanks for the patch.
+ 1 on the idea.
> Specifies a limit to the amount of memory (MB) that may be allocated to
> backends in total (i.e. this is not a per user or per backend limit).
> If unset, or set to 0 it is disabled. It is intended as a resource to
> help avoid the OOM killer. A backend request that would push the total
> over the limit will be denied with an out of memory error causing that
> backends current query/transaction to fail.
I'm not sure we are choosing the right victims here (aka the ones that
are doing the request that will push the total over the limit).
Imagine an extreme case where a single backend consumes say 99% of the
limit, shouldn't it be the one to be "punished"? (and somehow forced to
give the memory back).
The problem that i see with the current approach is that a "bad" backend
could impact all the others and continue to do so.
what about punishing say the highest consumer , what do you think? (just
speaking about the general idea here, not about the implementation)
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: Add the ability to limit the amount of memory that can be allocated to backends.
2022-08-31 16:50 Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-09-02 07:30 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Drouvot, Bertrand <[email protected]>
@ 2022-09-07 00:25 ` Reid Thompson <[email protected]>
0 siblings, 0 replies; 22+ messages in thread
From: Reid Thompson @ 2022-09-07 00:25 UTC (permalink / raw)
To: pgsql-hackers <[email protected]>; +Cc: [email protected]
On Fri, 2022-09-02 at 09:30 +0200, Drouvot, Bertrand wrote:
> Hi,
>
> I'm not sure we are choosing the right victims here (aka the ones
> that are doing the request that will push the total over the limit).
>
> Imagine an extreme case where a single backend consumes say 99% of
> the limit, shouldn't it be the one to be "punished"? (and somehow forced
> to give the memory back).
>
> The problem that i see with the current approach is that a "bad"
> backend could impact all the others and continue to do so.
>
> what about punishing say the highest consumer , what do you think?
> (just speaking about the general idea here, not about the implementation)
Initially, we believe that punishing the detector is reasonable if we
can help administrators avoid the OOM killer/resource starvation. But
we can and should expand on this idea.
Another thought is, rather than just failing the query/transaction we
have the affected backend do a clean exit, freeing all it's resources.
--
Reid Thompson
Senior Software Engineer
Crunchy Data, Inc.
[email protected]
www.crunchydata.com
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: Add the ability to limit the amount of memory that can be allocated to backends.
2022-08-31 16:50 Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
@ 2022-09-02 08:52 ` David Rowley <[email protected]>
2022-09-09 16:48 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Stephen Frost <[email protected]>
3 siblings, 1 reply; 22+ messages in thread
From: David Rowley @ 2022-09-02 08:52 UTC (permalink / raw)
To: [email protected]; Thomas Munro <[email protected]>; +Cc: pgsql-hackers <[email protected]>
On Thu, 1 Sept 2022 at 04:52, Reid Thompson
<[email protected]> wrote:
> Add the ability to limit the amount of memory that can be allocated to
> backends.
Are you aware that relcache entries are stored in backend local memory
and that once we've added a relcache entry for a relation that we have
no current code which attempts to reduce the memory consumption used
by cache entries when there's memory pressure?
It seems to me that if we had this feature as you propose that a
backend could hit the limit and stay there just from the memory
requirements of the relation cache after some number of tables have
been accessed from the given backend. It's not hard to imagine a
situation where the palloc() would start to fail during parse, which
might make it quite infuriating for anyone trying to do something
like:
SET max_total_backend_memory TO 0;
or
ALTER SYSTEM SET max_total_backend_memory TO 0;
I think a better solution to this problem would be to have "memory
grants", where we configure some amount of "pool" memory that backends
are allowed to use for queries. The planner would have to add the
expected number of work_mem that the given query is expected to use
and before that query starts, the executor would have to "checkout"
that amount of memory from the pool and return it when finished. If
there is not enough memory in the pool then the query would have to
wait until enough memory is available. This creates a deadlocking
hazard that the deadlock detector would need to be made aware of.
I know Thomas Munro has mentioned this "memory grant" or "memory pool"
feature to me previously and I think he even has some work in progress
code for it. It's a very tricky problem, however, as aside from the
deadlocking issue, it requires working out how much memory a given
plan will use concurrently. That's not as simple as counting the nodes
that use work_mem and summing those up.
There is some discussion about the feature in [1]. I was unable to
find what Thomas mentioned on the list about this. I've included him
here in case he has any extra information to share.
David
[1] https://www.postgresql.org/message-id/flat/20220713222342.GE18011%40telsasoft.com#b4f526aa8f2c893567...
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: Add the ability to limit the amount of memory that can be allocated to backends.
2022-08-31 16:50 Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-09-02 08:52 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. David Rowley <[email protected]>
@ 2022-09-09 16:48 ` Stephen Frost <[email protected]>
0 siblings, 0 replies; 22+ messages in thread
From: Stephen Frost @ 2022-09-09 16:48 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: [email protected]; Thomas Munro <[email protected]>; pgsql-hackers <[email protected]>
Greetings,
* David Rowley ([email protected]) wrote:
> On Thu, 1 Sept 2022 at 04:52, Reid Thompson
> <[email protected]> wrote:
> > Add the ability to limit the amount of memory that can be allocated to
> > backends.
>
> Are you aware that relcache entries are stored in backend local memory
> and that once we've added a relcache entry for a relation that we have
> no current code which attempts to reduce the memory consumption used
> by cache entries when there's memory pressure?
Short answer to this is yes, and that's an issue, but it isn't this
patch's problem to deal with- that's an issue that the relcache system
needs to be changed to address.
> It seems to me that if we had this feature as you propose that a
> backend could hit the limit and stay there just from the memory
> requirements of the relation cache after some number of tables have
> been accessed from the given backend. It's not hard to imagine a
> situation where the palloc() would start to fail during parse, which
> might make it quite infuriating for anyone trying to do something
> like:
Agreed that this could happen but I don't imagine it to be super likely-
and even if it does, this is probably a better position to be in as the
backend could then be disconnected from and would then go away and its
memory free'd, unlike the current OOM-killer situation where we crash
and go through recovery. We should note this in the documentation
though, sure, so that administrators understand how this can occur and
can take action to address it.
> I think a better solution to this problem would be to have "memory
> grants", where we configure some amount of "pool" memory that backends
> are allowed to use for queries. The planner would have to add the
> expected number of work_mem that the given query is expected to use
> and before that query starts, the executor would have to "checkout"
> that amount of memory from the pool and return it when finished. If
> there is not enough memory in the pool then the query would have to
> wait until enough memory is available. This creates a deadlocking
> hazard that the deadlock detector would need to be made aware of.
Sure, that also sounds great and a query acceptance system would be
wonderful. If someone is working on that with an expectation of it
landing before v16, great. Otherwise, I don't see it as relevant to
the question about if we should include this feature or not, and I'm not
even sure that we'd refuse this feature even if we already had an
acceptance system as a stop-gap should we guess wrong and not realize it
until it's too late.
Thanks,
Stephen
Attachments:
[application/pgp-signature] signature.asc (819B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: Add the ability to limit the amount of memory that can be allocated to backends.
@ 2024-01-23 11:47 Anton A. Melnikov <[email protected]>
2024-01-28 19:11 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Tomas Vondra <[email protected]>
0 siblings, 1 reply; 22+ messages in thread
From: Anton A. Melnikov @ 2024-01-23 11:47 UTC (permalink / raw)
To: Tomas Vondra <[email protected]>; Andres Freund <[email protected]>; Andrei Lepikhov <[email protected]>; +Cc: Stephen Frost <[email protected]>; [email protected]; Arne Roland <[email protected]>; [email protected] <[email protected]>; vignesh C <[email protected]>; Justin Pryzby <[email protected]>; Ibrar Ahmed <[email protected]>; stephen.frost <[email protected]>
Hi!
Thanks for your interest and reply!
On 26.12.2023 20:28, Tomas Vondra wrote:
> Can you share some info about the hardware? For example the CPU model,
> number of cores, and so on. 12GB RAM is not quite huge, so presumably it
> was a small machine.
>
It is HP ProLanit 2x socket server. 2x6 cores Intel(R) Xeon(R) CPU X5675 @ 3.07GHz,
2x12GB RAM, RAID from SSD drives.
Linux 5.10.0-21-amd64 #1 SMP Debian 5.10.162-1 (2023-01-21) x86_64 GNU/Linux
One cpu was disabled and some tweaks was made as Andres advised to avoid
NUMA and other side effects.
Full set of the configuration commands for server was like that:
numactl --cpunodebind=0 --membind=0 --physcpubind=1,3,5,7,9,11 bash
sudo cpupower frequency-set -g performance
sudo cpupower idle-set -D0
echo 3059000 | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_min_freq
(Turbo Boost and hyperthreading was disabled in BIOS.)
> I think what would be interesting are runs with
>
> pgbench -M prepared -S -c $N -j $N
>
> i.e. read-only tests (to not hit I/O), and $N being sufficiently large
> to maybe also show some concurrency/locking bottlenecks, etc.
>
> I may do some benchmarks if I happen to find a bit of time, but maybe
> you could collect such numbers too?
Firstly, i repeated the same -c and -j values in read-only mode as you advised.
As one can see in the read-only.png, the absolute TPS value has increased
significantly, by about 13 times.
Patched version with limit 200Mb was slightly slower than with limit 0 by ~2%.
The standard error in all series was ~0.5%.
Since the deviation has increased in comparison with rw test
the difference between unpatched version and patched ones with
limits 0, 8Gb and 16Gb is not sensible.
There is a raw data in the raw_data-read-only.txt.
> I think 6350 is a pretty terrible number, especially for scale 8, which
> is maybe 150MB of data. I think that's a pretty clear sign the system
> was hitting some other bottleneck, which can easily mask regressions in
> the memory allocation code. AFAICS the pgbench runs were regular r/w
> benchmarks, so I'd bet it was hitting I/O, and possibly even subject to
> some random effects at that level.
>
To avoid possible I/O bottleneck i followed these steps:
- gave all 24G mem to cpu 0 rather than 12G as in [1];
- created a ramdisk of 12G size;
- disable swap like that:
numactl --cpunodebind=0 --physcpubind=1,3,5,7,9,11 bash
sudo swapoff -a
sudo mkdir /mnt/ramdisk
sudo mount -t tmpfs -o rw,size=12G tmpfs /mnt/ramdisk
The inst dir, data dir and log file were all on ramdisk.
Pgbench in rw mode gave the following results:
- the difference between unpatched version and patched ones with
limits 0 and 16Gb almost the same: ~7470+-0.2% TPS.
(orange, green and blue distributions on the RW-ramdisk.png respectively)
- patched version with limit 8GB is slightly slower than three above;
(yellow distribution)
- patched version with limit 200MB slower than the first three
by a measurable value ~0,4% (~7440 TPS);
(black distribution)
The standard error in all series was ~0.2%. There is a raw data in the
raw_data-rw-ramdisk.txt
For the sake of completeness i'm going to repeat read-only measurements
with ramdisk. Аnd perform some tests with increased -c and -j values
as you advised to find the possible point where concurrency/blocking
bottlenecks start to play a role. And do this, of cause, for the last
version of the patch. Thanks for rebased it!
In general, i don't observe any considerable degradation in performance
from this patch of several or even 10%, which were mentioned in [2].
With the best regards,
--
Anton A. Melnikov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
[1] https://www.postgresql.org/message-id/1d3a7d8f-cb7c-4468-a578-d8a1194ea2de%40postgrespro.ru
[2] https://www.postgresql.org/message-id/3178e9a1b7acbcf023fafed68ca48d76afc07907.camel%40crunchydata.c...
REL_16_BETA1 (e0b82fc8e83) unpatched
85761.111563
85021.605432
85454.083033
85917.957666
85876.042244
85642.072514
86296.248572
85619.937094
85297.361215
85862.202724
84540.978854 (x)
85396.633119
85690.647063
84988.591565
85823.448629
84773.324608
86216.489115
85501.651461
86009.350080
85358.496042
85486.594373
85779.622224
85814.955234
84737.148907
85732.179382
85586.101863
85978.317659
85288.636666
85710.628041
85152.516444
85907.719844
84770.623411
85886.087293
85793.628493
85009.003868
85291.167576
86049.283058
85240.478343
86482.472191
85178.651590
85081.329863
85819.632868
85828.367169
86140.325742
85910.775960
85317.224079
REL_16_BETA1 (e0b82fc8e83) patched with https://www.postgresql.org/message-id/4edafedc0f8acb12a2979088ac1317bd7dd42145.camel%40crunchydata.c...
max_total_backend_memory undefined (default = 0)
86111.507420
86191.677434
85884.145207
86548.583375
85925.729848
86073.041345
85556.167244
86515.518378
85882.827060
86089.279375
84442.238391 (x)
86181.390728
86605.506000
86131.111260
85850.438380
86285.492544
85878.295476
85485.605781
86280.781506
85481.309046
86791.510072
85878.009945
86855.543915
85724.634195
84685.561101 (x)
85810.684502
84852.093784 (x)
86393.727723
86950.634063 (x)
85296.535016
85952.156912
85833.631996
85533.831539
85972.110542
85918.424161
85144.936232 (x)
86078.101155
86042.642900
86309.387185
85563.696537
85984.949710
86189.425869
85346.739647
86672.086794
86576.575964
85683.585455
REL_16_BETA1 (e0b82fc8e83) patched with https://www.postgresql.org/message-id/4edafedc0f8acb12a2979088ac1317bd7dd42145.camel%40crunchydata.c...
max_total_backend_memory = 16GB
85786.954867
86099.397176
86161.602185
86263.763959
85538.865680
85432.794926
85728.016002
85979.774947
85075.008854
86327.744016
85537.786933
86245.287068
86487.177382
85934.602769
85621.782529
85874.416847
86418.906684
85653.498016
86167.811566
85640.948196
86767.193839 (x)
86159.553447
86199.056598
85947.101168
85542.140478
86082.251029
85523.251587
85856.236524
85924.012803
85972.899323
85644.499894
85462.754650
85724.552728
85787.436042
85965.040630
85687.389455
84922.795072
85358.044611
86205.357569
85064.849443
84984.181808
85256.160481
85680.558543
85334.634567
85280.112157
84868.810444 (x)
REL_16_BETA1 (e0b82fc8e83) patched with https://www.postgresql.org/message-id/4edafedc0f8acb12a2979088ac1317bd7dd42145.camel%40crunchydata.c...
max_total_backend_memory = 8GB
85100.232214 (x)
86145.956362
86454.848486
85973.080571
86175.046121
86002.597626
85908.845992
85843.346909
86276.874013
86941.448180 (x)
85335.027277
86315.033717
85044.675027 (x)
86212.423345
85594.552032
85787.800684
85980.895557
85758.813670
85410.670102
85640.278088
86270.514436
86126.509444
86472.934287
86352.242914
86266.541944
85643.059914
86490.924339
84973.214587 (x)
86219.673171
86898.294513 (x)
85280.591982
86790.409937
85804.289889
86145.406187
86092.140989
84692.229690 (x)
85253.872633
85057.863152 (x)
85630.225186
84910.186408 (x)
86260.305705
86153.212996
85936.390876
86208.713158
84413.248101 (x)
86529.579668
REL_16_BETA1 (e0b82fc8e83) patched with https://www.postgresql.org/message-id/4edafedc0f8acb12a2979088ac1317bd7dd42145.camel%40crunchydata.c...
max_total_backend_memory = 200MB
71.710505 (x) out of memory errors in log
84243.587084
84576.485240
84546.191505
84769.692835
84811.989459
83941.333178
84390.922436
84080.510960
84406.063419
84815.054344
84926.435865
83382.114485 (x)
84233.526059
83737.372571
84710.087530
84266.060690
83979.118354
84966.343014
85145.211891
84821.723252
84575.757291
83986.231873
85087.270906
85140.140349
84125.963441
84090.664968
83898.033737
83646.132496
83868.054474
84194.324422
84295.701512
84154.903928
83877.842592
84018.297675
84840.741742
84893.298428
83717.601912
84367.085598
84675.434517
84871.418337
84412.975964
83599.945813
84713.575852
83989.731131
84818.386157
REL_16_BETA1 (e0b82fc8e83) unpatched
7467.419891
7472.014013
7493.242036
7408.774587
7496.802158
7457.542737
7487.910772
7475.640175
7485.466664
7499.874236
7474.825139
7479.573388
7456.172459
7454.242942
7500.062756
7440.119529
7429.548826
7464.584901
7466.640344
7472.130154
7457.652816
7498.109421
7505.506842
7475.923936
7465.309986
7470.681113
7411.998612
7457.822630
7509.467781
7449.197064
7504.898429
7464.936431
7447.685630
7497.218840
7465.513425
7481.328682
7463.571360
7429.434920
7462.178302
7505.404955
7490.309328
7488.477102
7462.639174
7471.688637
7461.336425
7499.165045
REL_16_BETA1 (e0b82fc8e83) patched with https://www.postgresql.org/message-id/4edafedc0f8acb12a2979088ac1317bd7dd42145.camel%40crunchydata.c...
max_total_backend_memory undefined (default = 0)
7464.611403
7478.403075
7450.917444
7483.562575
7500.521682
7479.266432
7459.974700
7507.553000
7495.172003
7452.101295
7513.070841
7463.382158
7456.085082
7472.833051
7503.577746
7433.234381
7461.966344
7483.855314
7495.659665
7493.387966
7397.391891
7471.962202
7476.375392
7460.974030
7514.881548
7505.659485
7455.310069
7453.988434
7448.730999
7445.222442
7501.325753
7420.613952
7409.553181
7482.888561
7493.232293
7507.268369
7482.801647
7429.492431
7470.676577
7482.153079
7457.478934
7469.759722
7416.620216
7471.906157
7454.781634
7466.844149
REL_16_BETA1 (e0b82fc8e83) patched with https://www.postgresql.org/message-id/4edafedc0f8acb12a2979088ac1317bd7dd42145.camel%40crunchydata.c...
max_total_backend_memory = 16GB
7481.056709
7468.899961
7489.134861
7498.626080
7504.534880
7503.127220
7471.474104
7488.119337
7458.440996
7464.739382
7488.627320
7455.193405
7459.978467
7482.468559
7491.174587
7432.616983
7492.806772
7469.908569
7449.892009
7508.213210
7458.613016
7476.039933
7492.311968
7496.616756
7435.722964
7476.914236
7478.436463
7474.484447
7431.591836
7475.391966
7495.125213
7481.887843
7436.923440
7421.318947
7461.459195
7491.085440
7491.841513
7467.331830
7506.848057
7509.429479
7483.237177
7488.256111
7459.021238
7387.323083 (x)
7428.320688
7471.441284
REL_16_BETA1 (e0b82fc8e83) patched with https://www.postgresql.org/message-id/4edafedc0f8acb12a2979088ac1317bd7dd42145.camel%40crunchydata.c...
max_total_backend_memory = 8GB
38.884155 (x) out of memory errors in log
7418.194045
7484.248051
7429.342711
7445.906427
7479.215456
7482.549510
7479.521513
7489.821055
7425.255640
7473.898287
7475.271906
7410.911195
7482.227422
7471.891959
7441.920472
7461.338663
7414.412202
7472.875753
7430.044709
7465.442867
7473.856803
7415.170436
7457.467000
7474.149360
7462.451469
7454.570057
7486.853451
7476.660190
7441.361623
7464.706562
7477.256048
7465.801516
7476.568591
7475.006562
7493.299365
7456.332872
7454.174261
7432.872466
7477.096962
7431.126181
7452.820622
7469.352342
7430.955516
7481.356856
7458.982820
REL_16_BETA1 (e0b82fc8e83) patched with https://www.postgresql.org/message-id/4edafedc0f8acb12a2979088ac1317bd7dd42145.camel%40crunchydata.c...
max_total_backend_memory = 200MB
36.223742 (x)
7399.245419
7451.173946
7396.641811
7454.101001
7479.472891
7427.912475
7441.225376
7450.589681
7412.533841
7437.228197
7446.286700
7441.405104
7477.718911
7398.548184
7392.161290
7458.758503
7467.946442
7424.785796
7459.894852
7440.628290
7454.374715
7445.956025
7425.295741
7448.542992
7437.003008
7440.505807
7462.882483
7453.967328
7449.913158
7431.653525
7465.553274
7420.272079
7458.774888
7476.748517
7469.100314
7433.124333
7424.384757
7462.361532
7475.693894
7431.876259
7414.515549
7451.841013
7407.344808
7455.825825
7384.738105
Attachments:
[image/png] read-only.png (213.7K, ../../[email protected]/2-read-only.png)
download | view image
[text/plain] raw_data-read-only.txt (3.8K, ../../[email protected]/3-raw_data-read-only.txt)
download | inline:
REL_16_BETA1 (e0b82fc8e83) unpatched
85761.111563
85021.605432
85454.083033
85917.957666
85876.042244
85642.072514
86296.248572
85619.937094
85297.361215
85862.202724
84540.978854 (x)
85396.633119
85690.647063
84988.591565
85823.448629
84773.324608
86216.489115
85501.651461
86009.350080
85358.496042
85486.594373
85779.622224
85814.955234
84737.148907
85732.179382
85586.101863
85978.317659
85288.636666
85710.628041
85152.516444
85907.719844
84770.623411
85886.087293
85793.628493
85009.003868
85291.167576
86049.283058
85240.478343
86482.472191
85178.651590
85081.329863
85819.632868
85828.367169
86140.325742
85910.775960
85317.224079
REL_16_BETA1 (e0b82fc8e83) patched with https://www.postgresql.org/message-id/4edafedc0f8acb12a2979088ac1317bd7dd42145.camel%40crunchydata.com
max_total_backend_memory undefined (default = 0)
86111.507420
86191.677434
85884.145207
86548.583375
85925.729848
86073.041345
85556.167244
86515.518378
85882.827060
86089.279375
84442.238391 (x)
86181.390728
86605.506000
86131.111260
85850.438380
86285.492544
85878.295476
85485.605781
86280.781506
85481.309046
86791.510072
85878.009945
86855.543915
85724.634195
84685.561101 (x)
85810.684502
84852.093784 (x)
86393.727723
86950.634063 (x)
85296.535016
85952.156912
85833.631996
85533.831539
85972.110542
85918.424161
85144.936232 (x)
86078.101155
86042.642900
86309.387185
85563.696537
85984.949710
86189.425869
85346.739647
86672.086794
86576.575964
85683.585455
REL_16_BETA1 (e0b82fc8e83) patched with https://www.postgresql.org/message-id/4edafedc0f8acb12a2979088ac1317bd7dd42145.camel%40crunchydata.com
max_total_backend_memory = 16GB
85786.954867
86099.397176
86161.602185
86263.763959
85538.865680
85432.794926
85728.016002
85979.774947
85075.008854
86327.744016
85537.786933
86245.287068
86487.177382
85934.602769
85621.782529
85874.416847
86418.906684
85653.498016
86167.811566
85640.948196
86767.193839 (x)
86159.553447
86199.056598
85947.101168
85542.140478
86082.251029
85523.251587
85856.236524
85924.012803
85972.899323
85644.499894
85462.754650
85724.552728
85787.436042
85965.040630
85687.389455
84922.795072
85358.044611
86205.357569
85064.849443
84984.181808
85256.160481
85680.558543
85334.634567
85280.112157
84868.810444 (x)
REL_16_BETA1 (e0b82fc8e83) patched with https://www.postgresql.org/message-id/4edafedc0f8acb12a2979088ac1317bd7dd42145.camel%40crunchydata.com
max_total_backend_memory = 8GB
85100.232214 (x)
86145.956362
86454.848486
85973.080571
86175.046121
86002.597626
85908.845992
85843.346909
86276.874013
86941.448180 (x)
85335.027277
86315.033717
85044.675027 (x)
86212.423345
85594.552032
85787.800684
85980.895557
85758.813670
85410.670102
85640.278088
86270.514436
86126.509444
86472.934287
86352.242914
86266.541944
85643.059914
86490.924339
84973.214587 (x)
86219.673171
86898.294513 (x)
85280.591982
86790.409937
85804.289889
86145.406187
86092.140989
84692.229690 (x)
85253.872633
85057.863152 (x)
85630.225186
84910.186408 (x)
86260.305705
86153.212996
85936.390876
86208.713158
84413.248101 (x)
86529.579668
REL_16_BETA1 (e0b82fc8e83) patched with https://www.postgresql.org/message-id/4edafedc0f8acb12a2979088ac1317bd7dd42145.camel%40crunchydata.com
max_total_backend_memory = 200MB
71.710505 (x) out of memory errors in log
84243.587084
84576.485240
84546.191505
84769.692835
84811.989459
83941.333178
84390.922436
84080.510960
84406.063419
84815.054344
84926.435865
83382.114485 (x)
84233.526059
83737.372571
84710.087530
84266.060690
83979.118354
84966.343014
85145.211891
84821.723252
84575.757291
83986.231873
85087.270906
85140.140349
84125.963441
84090.664968
83898.033737
83646.132496
83868.054474
84194.324422
84295.701512
84154.903928
83877.842592
84018.297675
84840.741742
84893.298428
83717.601912
84367.085598
84675.434517
84871.418337
84412.975964
83599.945813
84713.575852
83989.731131
84818.386157
[image/png] RW-ramdisk.png (214.6K, ../../[email protected]/4-RW-ramdisk.png)
download | view image
[text/plain] raw_data-rw-ramdisk.txt (3.5K, ../../[email protected]/5-raw_data-rw-ramdisk.txt)
download | inline:
REL_16_BETA1 (e0b82fc8e83) unpatched
7467.419891
7472.014013
7493.242036
7408.774587
7496.802158
7457.542737
7487.910772
7475.640175
7485.466664
7499.874236
7474.825139
7479.573388
7456.172459
7454.242942
7500.062756
7440.119529
7429.548826
7464.584901
7466.640344
7472.130154
7457.652816
7498.109421
7505.506842
7475.923936
7465.309986
7470.681113
7411.998612
7457.822630
7509.467781
7449.197064
7504.898429
7464.936431
7447.685630
7497.218840
7465.513425
7481.328682
7463.571360
7429.434920
7462.178302
7505.404955
7490.309328
7488.477102
7462.639174
7471.688637
7461.336425
7499.165045
REL_16_BETA1 (e0b82fc8e83) patched with https://www.postgresql.org/message-id/4edafedc0f8acb12a2979088ac1317bd7dd42145.camel%40crunchydata.com
max_total_backend_memory undefined (default = 0)
7464.611403
7478.403075
7450.917444
7483.562575
7500.521682
7479.266432
7459.974700
7507.553000
7495.172003
7452.101295
7513.070841
7463.382158
7456.085082
7472.833051
7503.577746
7433.234381
7461.966344
7483.855314
7495.659665
7493.387966
7397.391891
7471.962202
7476.375392
7460.974030
7514.881548
7505.659485
7455.310069
7453.988434
7448.730999
7445.222442
7501.325753
7420.613952
7409.553181
7482.888561
7493.232293
7507.268369
7482.801647
7429.492431
7470.676577
7482.153079
7457.478934
7469.759722
7416.620216
7471.906157
7454.781634
7466.844149
REL_16_BETA1 (e0b82fc8e83) patched with https://www.postgresql.org/message-id/4edafedc0f8acb12a2979088ac1317bd7dd42145.camel%40crunchydata.com
max_total_backend_memory = 16GB
7481.056709
7468.899961
7489.134861
7498.626080
7504.534880
7503.127220
7471.474104
7488.119337
7458.440996
7464.739382
7488.627320
7455.193405
7459.978467
7482.468559
7491.174587
7432.616983
7492.806772
7469.908569
7449.892009
7508.213210
7458.613016
7476.039933
7492.311968
7496.616756
7435.722964
7476.914236
7478.436463
7474.484447
7431.591836
7475.391966
7495.125213
7481.887843
7436.923440
7421.318947
7461.459195
7491.085440
7491.841513
7467.331830
7506.848057
7509.429479
7483.237177
7488.256111
7459.021238
7387.323083 (x)
7428.320688
7471.441284
REL_16_BETA1 (e0b82fc8e83) patched with https://www.postgresql.org/message-id/4edafedc0f8acb12a2979088ac1317bd7dd42145.camel%40crunchydata.com
max_total_backend_memory = 8GB
38.884155 (x) out of memory errors in log
7418.194045
7484.248051
7429.342711
7445.906427
7479.215456
7482.549510
7479.521513
7489.821055
7425.255640
7473.898287
7475.271906
7410.911195
7482.227422
7471.891959
7441.920472
7461.338663
7414.412202
7472.875753
7430.044709
7465.442867
7473.856803
7415.170436
7457.467000
7474.149360
7462.451469
7454.570057
7486.853451
7476.660190
7441.361623
7464.706562
7477.256048
7465.801516
7476.568591
7475.006562
7493.299365
7456.332872
7454.174261
7432.872466
7477.096962
7431.126181
7452.820622
7469.352342
7430.955516
7481.356856
7458.982820
REL_16_BETA1 (e0b82fc8e83) patched with https://www.postgresql.org/message-id/4edafedc0f8acb12a2979088ac1317bd7dd42145.camel%40crunchydata.com
max_total_backend_memory = 200MB
36.223742 (x)
7399.245419
7451.173946
7396.641811
7454.101001
7479.472891
7427.912475
7441.225376
7450.589681
7412.533841
7437.228197
7446.286700
7441.405104
7477.718911
7398.548184
7392.161290
7458.758503
7467.946442
7424.785796
7459.894852
7440.628290
7454.374715
7445.956025
7425.295741
7448.542992
7437.003008
7440.505807
7462.882483
7453.967328
7449.913158
7431.653525
7465.553274
7420.272079
7458.774888
7476.748517
7469.100314
7433.124333
7424.384757
7462.361532
7475.693894
7431.876259
7414.515549
7451.841013
7407.344808
7455.825825
7384.738105
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: Add the ability to limit the amount of memory that can be allocated to backends.
2024-01-23 11:47 Re: Add the ability to limit the amount of memory that can be allocated to backends. Anton A. Melnikov <[email protected]>
@ 2024-01-28 19:11 ` Tomas Vondra <[email protected]>
2025-12-27 14:52 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Tomas Vondra <[email protected]>
0 siblings, 1 reply; 22+ messages in thread
From: Tomas Vondra @ 2024-01-28 19:11 UTC (permalink / raw)
To: Anton A. Melnikov <[email protected]>; Andres Freund <[email protected]>; Andrei Lepikhov <[email protected]>; +Cc: Stephen Frost <[email protected]>; [email protected]; Arne Roland <[email protected]>; [email protected] <[email protected]>; vignesh C <[email protected]>; Justin Pryzby <[email protected]>; Ibrar Ahmed <[email protected]>; stephen.frost <[email protected]>
Hi,
I took a closer look at this patch over the last couple days, and I did
a bunch of benchmarks to see how expensive the accounting is. The good
news is that I haven't observed any overhead - I did two simple tests,
that I think would be particularly sensitive to this:
1) regular "pgbench -S" with scale 10, so tiny and not hitting I/O
2) "select count(*) from generate_series(1,N)" where N is either 10k or
1M, which should be enough to cause a fair number of allocations
And I tested this on three branches - master (no patches applied),
patched (but limit=0, so just accounting) and patched-limit (with limit
set to 4.5GB, so high enough not to be hit).
Attached are script and raw results for both benchmarks from two
machines, i5 (small, 4C) and xeon (bigger, 16C/32C), and a PDF showing
the results as candlestick charts (with 1 and 2 sigma intervals). AFAICS
there's no measurable difference between master and patched builds.
Which is good, clearly the local allowance makes the overhead negligible
(at least in these benchmarks).
Now, for the patch itself - I already said some of the stuff about a
month ago [1], but I'll repeat some of it for the sake of completeness
before I get to comments about the code etc.
Firstly, I agree with the goal of having a way to account for memory
used by the backends, and also ability to enforce some sort of limit.
It's difficult to track the memory at the OS level (interpreting RSS
values is not trivial), and work_mem is not sufficient to enforce a
backend-level limit, not even talking about a global limit.
But as I said earlier, it seems quite strange to start by introducing
some sort of global limit, combining memory for all backends. I do
understand that the intent is to have such global limit in order to
prevent issues with the OOM killer and/or not to interfere with other
stuff running on the same machine. And while I'm not saying we should
not have such limit, every time I wished to have a memory limit it was a
backend-level one. Ideally a workmem-like limit that would "adjust" the
work_mem values used by the optimizer (but that's not what this patch
aims to do), or at least a backstop in case something goes wrong (say, a
memory leak, OLTP application issuing complex queries, etc.).
The accounting and infrastructure introduced by the patch seems to be
suitable for both types of limits (global and per-backend), but then it
goes for the global limit first. It may seem simpler to implement, but
in practice there's a bunch of problems mentioned earlier, but ignored.
For example, I really don't believe it's OK to just report the error in
the first backend that happens to hit the limit. Bertrand mentioned that
[2], asking about a situation when a runaway process allocates 99% of
memory. The response to that was
> Initially, we believe that punishing the detector is reasonable if we
> can help administrators avoid the OOM killer/resource starvation. But
> we can and should expand on this idea.
which I find rather unsatisfactory. Belief is not an argument, and it
relies on the assumption that this helps the administrator to avoid the
OOM killer etc. ISTM it can easily mean the administrator can't even
connect to the database, run a query (to inspect the new system views),
etc. because any of that would hit the memory limit. That seems more
like a built-in DoS facility ...
If this was up to me, I'd probably start with the per-backend limit. But
that's my personal opinion.
Now, some comments about the code etc. Stephen was promising a new patch
version in October [6], but that didn't happen yet, so I looked at the
patches I rebased in December.
0001
----
1) I don't see why we should separate memory by context type - without
knowing which exact backends are using the memory, this seems pretty
irrelevant/useless. Either it's private or shared memory, I don't think
the accounting should break this into more counters. Also, while I'm not
aware of anyone proposing a new memory context type, I doubt we'd want
to add more and more counters if that happens.
suggestion: only two counters, for local and shared memory
If we absolutely want to have per-type counters, we should not mix the
private memory and shared memory ones.
2) I have my doubts about the tracking of shared memory. It seems weird
to count them only into the backend that allocated them, and transfer
them to "global" on process exit. Surely we should know which shared
memory is meant to be long-lived from the start, no?
For example, it's not uncommon that an extension allocates a chunk of
shared memory to store some sort of global state (e.g. BDR/pglogical
does that, I'm sure other extensions do that). But the process that
allocates the shared memory keeps running. AFAICS this would be tracked
as backend DSM memory, which I'm not sure this is what we want ...
And I'm not alone in thinking this should work differently - [3], [4].
3) We limit the floor of allocation counters to zero.
This seems really weird. Why would it be necessary? I mean, how could we
get a negative amount of memory? Seems like it might easily mask issues
with incorrect accounting, or something.
4) pg_stat_memory_allocation
Maybe pg_stat_memory_usage would be a better name ...
5) The comments/docs repeatedly talk about "dynamic nature" and that it
makes the values not exact:
> Due to the dynamic nature of memory allocations the allocated bytes
> values may not be exact but should be sufficient for the intended
> purposes.
I don't understand what "dynamic nature" refers to, and why would it
make the values not exact. Presumably this refers to the "allowance" but
how is that "dynamic nature"?
This definitely needs to explain this better, with some basic estimate
how how accurate the values are expected to be.
6) The SGML docs keep recommending to use pg_size_pretty(). I find that
unnecessary, no other docs reporting values in bytes do that.
7) I see no reason to include shared_memory_size_mb in the view, and
same for shared_memory_size_in_huge_pages. It has little to do with
"allocated" memory, IMO. And a value in "MB" goes directly against the
suggestion to use pg_size_pretty().
suggestion: drop this from the view
8) In a lot of places we do
context->mem_allocated += blksize;
pgstat_report_allocated_bytes_increase(blksize, PG_ALLOC_ASET);
Maybe we should integrate the two types of accounting, wrap them into a
single function call, or something? This makes it very simple to forget
updating one of those places. AFAIC the patch tries to minimize the
number of updates of the new shared counters, but with the allowance
that should not be an issue I think.
9) This seems wrong. Why would it be OK to ever overflow? Isn't that a
sign of incorrect accounting?
/* Avoid allocated_bytes unsigned integer overflow on decrease */
if (pg_sub_u64_overflow(*my_allocated_bytes, proc_allocated_bytes,
&temp))
{
/* On overflow, set allocated bytes and allocator type bytes to
zero */
*my_allocated_bytes = 0;
*my_aset_allocated_bytes = 0;
*my_dsm_allocated_bytes = 0;
*my_generation_allocated_bytes = 0;
*my_slab_allocated_bytes = 0;
}
9) How could any of these values be negative? It's all capped to 0 and
also stored in uint64. Seems pretty useless.
+SELECT
+ datid > 0, pg_size_bytes(shared_memory_size) >= 0,
shared_memory_size_in_huge_pages >= -1, global_dsm_allocated_bytes >= 0
+FROM
+ pg_stat_global_memory_allocation;
+ ?column? | ?column? | ?column? | ?column?
+----------+----------+----------+----------
+ t | t | t | t
+(1 row)
+
+-- ensure that pg_stat_memory_allocation view exists
+SELECT
+ pid > 0, allocated_bytes >= 0, aset_allocated_bytes >= 0,
dsm_allocated_bytes >= 0, generation_allocated_bytes >= 0,
slab_allocated_bytes >= 0
0003
----
1) The commit message says:
> Further requests will not be allocated until dropping below the limit.
> Keep this in mind when setting this value.
The SGML docs have a similar "keep this in mind" suggestion in relation
to the 1MB local allowance. I find this pretty useless, as it doesn't
really say what to do with the information / what it means. I mean,
should I set the value higher, or what am I supposed to do? This needs
to be understandable for average user reading the SGML docs, who is
unlikely to know the implementation details.
2) > This limit does not affect auxiliary backend processes.
This seems pretty unfortunate, because in the cases where I actually saw
OOM killer to intervene, it was often because of auxiliary processes
allocating a lot of memory (say, autovacuum with maintenance_work_mem
set very high, etc.).
In a way, not excluding these auxiliary processes from the limit seems
to go against the idea of preventing the OOM killer.
3) I don't think the patch ever explains what the "local allowance" is,
how the refill works, why it's done this way, etc. I do think I
understand that now, but I had to go through the thread, That's not
really how it should be. There should be an explanation of how this
works somewhere (comment in mcxt.c? separate README?).
4) > doc/src/sgml/monitoring.sgml
Not sure why this removes the part about DSM being included only in the
backend that created it.
5) > total_bkend_mem_bytes_available
The "bkend" name is strange (to save 2 characters), and the fact how it
combines local and shared memory seems confusing too.
6)
+ /*
+ * Local counter to manage shared memory allocations. At backend
startup, set to
+ * initial_allocation_allowance via pgstat_init_allocated_bytes().
Decrease as
+ * memory is malloc'd. When exhausted, atomically refill if available from
+ * ProcGlobal->max_total_bkend_mem via exceeds_max_total_bkend_mem().
+ */
+uint64 allocation_allowance = 0;
But this allowance is for shared memory too, and shared memory is not
allocated using malloc.
7) There's a lot of new global variables. Maybe it'd be better to group
them into a struct, or something?
8) Why does pgstat_set_allocated_bytes_storage need the new return?
9) Unnecessary changes in src/backend/utils/hash/dynahash.c (whitespace,
new comment that seems not very useful)
10) Shouldn't we have a new malloc wrapper that does the check? That way
we wouldn't need to have the call in every memory context place calling
malloc.
11) I'm not sure about the ereport() calls after hitting the limit.
Adres thinks it might lead to recursion [4], but Stephen [5] seems to
think this does not really make the situation worse. I'm not sure about
that, though - I agree the ENOMEM can already happen, but but maybe
having a limit (which clearly needs to be more stricter than the limit
used by kernel for OOM) would be more likely to hit?
12) In general, I agree with Andres [4] that we'd be better of to focus
on the accounting part, see how it works in practice, and then add some
ability to limit memory after a while.
regards
[1]
https://www.postgresql.org/message-id/4fb99fb7-8a6a-2828-dd77-e2f1d75c7dd0%40enterprisedb.com
[2]
https://www.postgresql.org/message-id/3b9b90c6-f4ae-a7df-6519-847ea9d5fe1e%40amazon.com
[3]
https://www.postgresql.org/message-id/20230110023118.qqbjbtyecofh3uvd%40awork3.anarazel.de
[4]
https://www.postgresql.org/message-id/20230113180411.rdqbrivz5ano2uat%40awork3.anarazel.de
[5]
https://www.postgresql.org/message-id/ZTArWsctGn5fEVPR%40tamriel.snowman.net
[6]
https://www.postgresql.org/message-id/ZTGycSYuFrsixv6q%40tamriel.snowman.net
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
[application/pdf] results-count.pdf (388.3K, ../../[email protected]/2-results-count.pdf)
download
[application/pdf] results-pgbench.pdf (228.4K, ../../[email protected]/3-results-pgbench.pdf)
download
[application/x-compressed-tar] scripts.tgz (768B, ../../[email protected]/4-scripts.tgz)
download
[application/x-compressed-tar] xeon-count.tgz (49.7K, ../../[email protected]/5-xeon-count.tgz)
download
[application/x-compressed-tar] xeon-pgbench.tgz (29.9K, ../../[email protected]/6-xeon-pgbench.tgz)
download
[application/x-compressed-tar] i5-count.tgz (44.5K, ../../[email protected]/7-i5-count.tgz)
download
[application/x-compressed-tar] i5-pgbench.tgz (28.5K, ../../[email protected]/8-i5-pgbench.tgz)
download
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: Add the ability to limit the amount of memory that can be allocated to backends.
2024-01-23 11:47 Re: Add the ability to limit the amount of memory that can be allocated to backends. Anton A. Melnikov <[email protected]>
2024-01-28 19:11 ` Re: Add the ability to limit the amount of memory that can be allocated to backends. Tomas Vondra <[email protected]>
@ 2025-12-27 14:52 ` Tomas Vondra <[email protected]>
0 siblings, 0 replies; 22+ messages in thread
From: Tomas Vondra @ 2025-12-27 14:52 UTC (permalink / raw)
To: James Hunter <[email protected]>; Jim Nasby <[email protected]>; +Cc: Jeremy Schneider <[email protected]>; Anton A. Melnikov <[email protected]>; Andres Freund <[email protected]>; Andrei Lepikhov <[email protected]>; Stephen Frost <[email protected]>; [email protected]; Arne Roland <[email protected]>; [email protected] <[email protected]>; vignesh C <[email protected]>; Justin Pryzby <[email protected]>; Ibrar Ahmed <[email protected]>; stephen.frost <[email protected]>
Hi,
Let me bump this dormant thread after about a year. I don't have any
patch to share or anything like that, but I happened to read two quite
interesting papers relevant to the topic of setting a per-query memory
limit, and distributing the memory between operators in a query plan:
1) Exploiting pipeline interruptions for efficient memory allocation
Authors: Josep Aguilar-Saborit, Mohammad Jalali, Dave Sharpe, Victor
Muntés MuleroAuthors Info & Claims
Published: 2008
PDF: https://dl.acm.org/doi/epdf/10.1145/1458082.1458169
2) Saving Private Hash Join
Authors: Laurens Kuiper, Paul Groß, Peter Boncz, Hannes Mühleisen
Published: 2024-2025
PDF: https://www.vldb.org/pvldb/vol18/p2748-kuiper.pdf
I read the (2) paper first, expecting to learn some new stuff about hash
joins, but to my surprise it's focusing on how to distribute memory
between multiple hash joins in a single query. The hash joins serve
mostly as an example of an actual/common operator in a query.
The (1) paper establishes the theoretical framework and algorithms, and
(2) presents a more precise/complete model for hash joins.
I think both papers are worth reading, and the framework seems quite
helpful. Even if there some parts may need tweaks, as Postgres does
certain things differently for whatever reason.
I'm not going to explain all the details (the papers are not that long
anyway), but the proposed approach combines a couple basic parts:
1) leverage "pipeline interruption" operations
Some operators materialize the intermediate results. This splits the
plan into parts that don't need memory at the same time. It's enough to
enforce the limit for these parts, not for the whole plan. Which means
the optimization problems are smaller, and the operators can get more
memory than when assuming the whole query runs at the same time.
Of course, the reality is more complicated. Some operators are only
partial pipeline interruptions (e.g. hashjoin breaks for the inner
subplan, not the outer).
And Postgres currently delays the Hash build until the first outer
tuple, unlike DuckDB used in the (2) paper.
2) cost model for memory distribution
The memory is distributed between operators based on a simple cost
model, instead of using a simple scheme where operators get 1/N of
memory. The (2) paper presents a hash join cost model combining
"materialization cost" and "throughput".
But it's a pretty generic approach, and should not be too difficult to
do for other operators. Most operators don't even need to allocate that
much memory, so the cost model can ignore those, I think. Only nodes
that "break the pipeline" would matter.
The important part is that this is a second optimization phase. The
optimizer picks a query plan (assuming some default memory sizes), and
then the cost model determines how to distribute memory within that
particular plan.
The paper does that repeatedly during execution, but I suspect these
adjustments are easier in DuckDB. In Postgres we'd probably do that only
once right after planning? Or maybe not, not sure.
The (1) paper does things a bit differently, so that it works on DB2 (so
less dynamic), but it also focuses on plans with hash joins etc. The
general framework seems to be mostly the same.
Anyway, that's all I have. I don't expect to work on this anytime soon,
but I found those papers interesting.
regards
--
Tomas Vondra
^ permalink raw reply [nested|flat] 22+ messages in thread
end of thread, other threads:[~2025-12-27 14:52 UTC | newest]
Thread overview: 22+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2022-08-31 16:50 Add the ability to limit the amount of memory that can be allocated to backends. Reid Thompson <[email protected]>
2022-08-31 17:34 ` Justin Pryzby <[email protected]>
2022-09-04 03:40 ` Reid Thompson <[email protected]>
2022-09-09 17:14 ` Justin Pryzby <[email protected]>
2022-09-12 16:25 ` Reid Thompson <[email protected]>
2022-09-15 08:07 ` Ibrar Ahmed <[email protected]>
2022-09-15 14:58 ` Reid Thompson <[email protected]>
2022-10-24 15:27 ` Arne Roland <[email protected]>
2022-10-25 15:49 ` Reid Thompson <[email protected]>
2022-11-03 15:48 ` Reid Thompson <[email protected]>
2022-11-27 03:22 ` Reid Thompson <[email protected]>
2022-12-06 18:32 ` Andres Freund <[email protected]>
2022-12-09 15:05 ` Reid Thompson <[email protected]>
2022-09-01 02:48 ` Kyotaro Horiguchi <[email protected]>
2022-09-07 00:17 ` Reid Thompson <[email protected]>
2022-09-02 07:30 ` Drouvot, Bertrand <[email protected]>
2022-09-07 00:25 ` Reid Thompson <[email protected]>
2022-09-02 08:52 ` David Rowley <[email protected]>
2022-09-09 16:48 ` Stephen Frost <[email protected]>
2024-01-23 11:47 Re: Add the ability to limit the amount of memory that can be allocated to backends. Anton A. Melnikov <[email protected]>
2024-01-28 19:11 ` Tomas Vondra <[email protected]>
2025-12-27 14:52 ` Tomas Vondra <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox