public inbox for [email protected]
help / color / mirror / Atom feedFrom: Ashutosh Bapat <[email protected]>
To: Heikki Linnakangas <[email protected]>
Cc: Matthias van de Meent <[email protected]>
Cc: Robert Haas <[email protected]>
Cc: Andres Freund <[email protected]>
Cc: pgsql-hackers <[email protected]>
Cc: [email protected]
Subject: Re: Better shared data structure management and resizable shared data structures
Date: Mon, 6 Apr 2026 19:23:19 +0530
Message-ID: <CAExHW5u5LOgB3Y4Ee3VWVEurYN2GwnkbVEcfrGCQLgcGgf_zKw@mail.gmail.com> (raw)
In-Reply-To: <[email protected]>
References: <CAExHW5vM1bneLYfg0wGeAa=52UiJ3z4vKd3AJ72X8Fw6k3KKrg@mail.gmail.com>
<CAExHW5uTNWOSxJDWQAUnS0tZawob2_J3dRAtc67NHNZ98X4_xA@mail.gmail.com>
<CAExHW5t439y61YD9bc7d5wZWHp6J=M43Qu3eEZOBPguZML7o2A@mail.gmail.com>
<CAExHW5v5FVZbsO9sLzztMZ11C3hgGStE=HkkV2bQkCyncess4w@mail.gmail.com>
<[email protected]>
<CAExHW5tCC0T1ky=Jnq-AvMxa67Adaw7aQ4iQAO=BSdHcbSNBVg@mail.gmail.com>
<[email protected]>
<CAExHW5tS7GncN90oJWOSzW_3F1EHL9xwe59L7Req3nUVgmObUw@mail.gmail.com>
<[email protected]>
<CAEze2WhMOHVgH2Xeyzx=VEk-Ta_YnQUqT+TdBiv5Lx8ESn2WZA@mail.gmail.com>
<CAExHW5s6h=c_q2m72Nvyj1ghMEhPkOBkeN5Htn7YR=1BrNN-Sw@mail.gmail.com>
<[email protected]>
<CAEze2WjQZff3znd6CtG-OBzYZMMqy5TyQSoAo=QTFT38tDndeQ@mail.gmail.com>
<[email protected]>
<CAEze2WjgCROMMXY0+j8FFdm3iFcr7By-+6Mwiz=PgGSEydiW3A@mail.gmail.com>
<[email protected]>
<[email protected]>
On Mon, Apr 6, 2026 at 4:58 AM Heikki Linnakangas <[email protected]> wrote:
>
> On 05/04/2026 23:06, Heikki Linnakangas wrote:
> > Here's patch version 12 [*]. I believe I've addressed all the feedback,
> > and I feel this is in pretty good shape now. There hasn't been any big
> > design changes lately.
> >
> > One notable change is that I replaced the separate {request|init|attach}
> > _fn_arg fields in ShmemCallbacks with a single 'opaque_arg' field, and
> > added a brief comment to it. You both commented on whether we need that
> > at all, and maybe you're right that we don't, but at least it's now just
> > one field rather than three. As before, callers can simply ignore it if
> > they don't need it.
>
> After another round of comment cleanups and such, committed. Thanks!
Thanks. Attached are rebased patches.
0001 is the same patch as submitted before to support resizable shared
memory structures.
0002 changes resizable_shmem_usage() in the resizable_shmem test
module to use /proc/self/smaps as suggested by Andres offline. Using
smaps the function estimates the actual memory mapped against the main
shared memory segment. Expecting this to fix failure reported on
https://cirrus-ci.com/task/5501660157444096.
0003 adds some more diagnostic about shared memory segments in the
server log to debug the failure in case it appears even with 0002
0004 addresses Matthias's comments in the discussion below
On Mon, Apr 6, 2026 at 12:35 AM Matthias van de Meent
<[email protected]> wrote:
>
> On Sun, 5 Apr 2026 at 13:20, Ashutosh Bapat
> <[email protected]> wrote:
> >
> > On Sun, Apr 5, 2026 at 2:36 PM Matthias van de Meent
> > <[email protected]> wrote:
> > >
> > > On Sun, 5 Apr 2026, 07:59 Ashutosh Bapat, <[email protected]> wrote:
> > > >
> > > > On Sun, Apr 5, 2026 at 11:18 AM Ashutosh Bapat
> > > > <[email protected]> wrote:
> > > > >
> > >
> > > I'm not opposed to HAVE_RESIZABLE_SHMEM, but is it universal enough on
> > > its platforms to make it part of the exposed ABI for Shmem? I think
> > > that we should expose the same functions and structs, and just have
> > > the shmem internals throw an error if the configuration used by the
> > > user implies the user wants to update shmem sizing when the system
> > > doesn't support it. That would avoid extensions having to recompile
> > > between have/have not systems that have an otherwise compatible ABI;
> > > especially when those extensions don't actually need the resizeable
> > > part of the shmem system.
> > >
> >
> > I don't think I understand this fully. An extension may want to
> > support a structure in both modes - fixed as well as resizable
> > depending upon whether the latter is supported. If the structure has
> > maximum_size always the extension code needs to set it to 0 when the
> > resizable shared structure is not supported and set to actual
> > maximum_size when the resizable structure is supported. Without a
> > macro or some flag they can not do that. The flag/macro then becomes
> > part ABI for shmem. Am I correct?
>
> That's not quite what I meant.
>
> With your patch, the size and field offsets in `struct
> ShmemStructOpts` changes depending only on HAVE_RESIZABLE_SHMEM, as
> does function's availability. This means that an extension that's
> built without HAVE_RESIZABLE_SHMEM (an otherwise identical system)
> can't correctly be loaded into a server that does have
> HAVE_RESIZABLE_SHMEM defined - or at least it'll misbehave when it
> tries to use the new shmem system without trying out resizeable areas.
>
> If instead the fields used for definining resizable shmem areas (and
> the relevant functions) are always defined, but with runtime checks to
> make sure that in !HAVE_RESIZEABLE_SHMEM nobody tries to use the
> resizing functionality, then that'd reduce the unchecked hidden
> incompatibility; assuming that no extension manually does memory
> management syscall operations on those shmem areas.
>
> > Since extension binaries need to be
> > built on different platforms anyway, that would automatically take
> > care of building with or without HAVE_RESIZABLE_SHMEM. I feel it makes
> > testing simpler since run time behaviour is fixed. Maybe I am missing
> > something. Maybe a code diff or some example platform might make it
> > more clear for me.
>
> I'm not entirely sure it would be automatic. Is it guaranteed that
> HAVE_RESIZABLE_SHMEM won't change over the lifetime of any
> distribution's platform? Because it's definitely not apparent to me
> that rebuilding the new server version against an upgraded platform
> (now possibly with HAVE_RESIZABLE_SHMEM) should also mean rebuilding
> the extensions that have been built against a previous minor version
> (without HAVE_RESIZABLE_SHMEM).
>
Please review changes in 0004 to see if they address your concerns.
The structures and functions are same irrespective of
HAVE_RESIZABLE_SHMEM. If HAVE_RESIZABLE_SHMEM is not defined, but
maximum_size > 0, then the request to add this structure will fail.
Thus on a server running with binary built with HAVE_RESIZABLE_SHMEM
undefined, maximum_size = 0 always. Many of the code blocks don't need
#ifdef HAVE_RESIZABLE_SHMEM blocks then. I think code this way is much
more readable. However binary will contain more instructions than
necessary - maybe the compiler can optimize those out.
> > > > For now, it
> > > > seems only for the sanity checks, but it could be seen as a useful
> > > > safety feature. A difference in maximum_size and minimum_size would
> > > > indicate that the structure is resizable.
> > >
> > > I think that's the right approach.
> >
> >
> > I also think that introducing minimum_size is useful. Let's hear from
> > Heikki before implementing it, in case he has a different opinion. I
> > am not sure about min_allocated_space though - what use do you see for
> > it. reserved_space is useful in pg_shmem_allocations() C function
> > itself and gives impact to the fully grown structure. What would
> > min_allocated_space give us? If at all it would be min_allocated_size
> > not space since reserved space will never change. But even that I am
> > not sure about.
>
> I'd say it's mostly interesting for people looking at or debugging
> shmem allocations. Which isn't a huge group of developers or DBAs, but
> if we're exposing data like this, and are going to allow resizing,
> then someone could see some benefits from this.
>
> E.g., it may be useful to have the information to see how low the
> currently running server can scale down its memory usage, so that the
> admin can see whether a reboot is required if they want to allow it to
> scale it down further (assuming there's a lower limit for allocations
> - some shmem structs may have a lower scaling limit defined at
> startup, while others may be able to scale linearly from 0 to 100)
I guess, most of the time the lower limit will be a hard lower limit
which could not be changed, so I guess the usecases are pretty narrow.
0005 adds minimum_size member alongside maximum_size as per the
discussion starting [1]. I like the end result since maximum_size is
managed consistently across fixed-size and resizable structures and
also the condition to check whether a structure is resizable or fixed
is more natural/logical.
0006 adds support to add mprotect appropriately on the used and unused
part of a resizable structure, again per discussion starting [1]. This
is still a bit rough, but review comments are welcome.
I have kept these two patches separate from the main patch so that I
can remove them if others feel they are not worth including in the
feature.
[1] https://www.postgresql.org/message-id/[email protected]...
--
Best Wishes,
Ashutosh Bapat
Attachments:
[text/x-patch] v20260406-0001-resizable-shared-memory-structures.patch (66.6K, 2-v20260406-0001-resizable-shared-memory-structures.patch)
download | inline diff:
From 06b8669cffcf09294706c536ade397a726ed0633 Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat <[email protected]>
Date: Tue, 17 Feb 2026 16:51:20 +0530
Subject: [PATCH v20260406 1/6] resizable shared memory structures
Resizable shared memory structures can be allocated by specifying a new
member ShmemStructOpts::maximum_size. At the startup or when the
structure is created, we reserve address space worth maximum_size in the
shared memory segment. It is expected that the subsystem which creates
the structure would initialize only the initial size worth of memory
when creating it. In an mmap'ed memory, this should allocate memory
worth the initial size. It should not allocate maximum_size worth of
memory initially. As the structure is resized using ShmemResizeStruct()
memory is freed or allocated in chunks of memory pages when shrinking
and expanding the structure respectively.
Resizable shared memory feature depends upon existence of function
madvise() and constants MADV_REMOVE and MADV_WRITE_POPULATE.
On the platforms which do not have these, we disable this feature at
compile time. The commit introduces a compile time flag
HAVE_RESIZABLE_SHMEM which is defined if MADV_REMOVE and
MADV_WRITE_POPULATE exist. We don't check existence of madvise
separately, since existence of the constants implies existence of the
function.
HAVE_RESIZABLE_SHMEM is not defined in EXEC_BACKEND builds since that's
largely used for Windows where the APIs to free and allocate memory from
and to a given address space are not known to the author right now.
Given that PostgreSQL is used widely on Linux, providing this feature on
Linux covers benefits most of its users. Once we figure out the required
Windows APIs, we will support this feature on Windows as well.
The feature is also not available when Sys-V shared memory is used even
on Linux since we do not know whether required Sys-V APIs exist; mostly
they don't. Since that combination is only available for development and
testing, not supporting the feature there isn't going to impact
PostgreSQL users.
Using HAVE_RESIZABLE_SHMEM we disable compiling the code related to
resizable shared memory structures on the platforms which do not support
the feature. But we also have run time checks to disable this feature
when Sys-V shared memory is used. In order to know whether a given
instance of running server supports resizable structures, we have
introduced GUC have_resizable_shmem.
Author: Ashutosh Bapat <[email protected]>
Reviewed-by: Matthias van de Meent <[email protected]>
---
configure.ac | 4 +
doc/src/sgml/config.sgml | 15 +
doc/src/sgml/system-views.sgml | 30 +-
doc/src/sgml/xfunc.sgml | 54 +++
meson.build | 16 +
src/backend/port/sysv_shmem.c | 69 ++++
src/backend/port/win32_shmem.c | 23 ++
src/backend/storage/ipc/shmem.c | 269 +++++++++++++--
src/backend/utils/misc/guc_parameters.dat | 7 +
src/backend/utils/misc/guc_tables.c | 7 +
src/include/catalog/pg_proc.dat | 4 +-
src/include/pg_config.h.in | 8 +
src/include/pg_config_manual.h | 9 +
src/include/storage/pg_shmem.h | 5 +
src/include/storage/shmem.h | 16 +
src/test/modules/Makefile | 1 +
src/test/modules/meson.build | 1 +
src/test/modules/resizable_shmem/Makefile | 25 ++
src/test/modules/resizable_shmem/meson.build | 36 ++
.../resizable_shmem/resizable_shmem--1.0.sql | 37 ++
.../modules/resizable_shmem/resizable_shmem.c | 326 ++++++++++++++++++
.../resizable_shmem/resizable_shmem.control | 4 +
.../resizable_shmem/t/001_resizable_shmem.pl | 239 +++++++++++++
.../test_shmem/t/001_late_shmem_alloc.pl | 23 ++
.../modules/test_shmem/test_shmem--1.0.sql | 4 +
src/test/modules/test_shmem/test_shmem.c | 20 ++
src/test/regress/expected/rules.out | 6 +-
src/tools/pgindent/typedefs.list | 1 +
28 files changed, 1223 insertions(+), 36 deletions(-)
create mode 100644 src/test/modules/resizable_shmem/Makefile
create mode 100644 src/test/modules/resizable_shmem/meson.build
create mode 100644 src/test/modules/resizable_shmem/resizable_shmem--1.0.sql
create mode 100644 src/test/modules/resizable_shmem/resizable_shmem.c
create mode 100644 src/test/modules/resizable_shmem/resizable_shmem.control
create mode 100644 src/test/modules/resizable_shmem/t/001_resizable_shmem.pl
diff --git a/configure.ac b/configure.ac
index ff5dd64468e..7acd844ccb2 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1895,6 +1895,10 @@ AC_CHECK_DECLS([memset_s], [], [], [#define __STDC_WANT_LIB_EXT1__ 1
# This is probably only present on macOS, but may as well check always
AC_CHECK_DECLS(F_FULLFSYNC, [], [], [#include <fcntl.h>])
+# Linux-specific madvise constants needed for resizable shared memory. See similar checks in meson.build for explanation of why these checks are here.
+AC_CHECK_DECLS([MADV_POPULATE_WRITE], [], [], [#include <sys/mman.h>])
+AC_CHECK_DECLS([MADV_REMOVE], [], [], [#include <sys/mman.h>])
+
AC_REPLACE_FUNCS(m4_normalize([
explicit_bzero
getopt
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index b44231a362d..7a01f2cf967 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -12114,6 +12114,21 @@ dynamic_library_path = '/usr/local/lib/postgresql:$libdir'
</listitem>
</varlistentry>
+ <varlistentry id="guc-have-resizable-shmem" xreflabel="have_resizable_shmem">
+ <term><varname>have_resizable_shmem</varname> (<type>boolean</type>)
+ <indexterm>
+ <primary><varname>have_resizable_shmem</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Reports whether <productname>PostgreSQL</productname> has been built
+ with <literal>HAVE_RESIZABLE_SHMEM</literal> enabled and supports
+ <link linkend="xfunc-shared-addin-resizable">Resizable shared memory structures</link>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-huge-pages-status" xreflabel="huge_pages_status">
<term><varname>huge_pages_status</varname> (<type>enum</type>)
<indexterm>
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 2ebec6928d5..9717f8434bb 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -4243,8 +4243,34 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
Size of the allocation in bytes including padding. For anonymous
allocations, no information about padding is available, so the
<literal>size</literal> and <literal>allocated_size</literal> columns
- will always be equal. Padding is not meaningful for free memory, so
- the columns will be equal in that case also.
+ will always be equal. Padding is not meaningful for free memory, so the
+ columns will be equal in that case also. For resizable allocations which
+ may span multiple memory pages, the padding includes the padding due to
+ page alignment.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>maximum_size</structfield> <type>int8</type>
+ </para>
+ <para>
+ Maximum size in bytes that the resizable allocation can grow to. Zero for
+ fixed-size allocations, for anonymous allocations, and for free memory.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>reserved_space</structfield> <type>int8</type>
+ </para>
+ <para>
+ Address space reserved for the allocation in bytes. For resizable
+ structures, this is the total address space reserved to accommodate
+ growth up to <structfield>maximum_size</structfield>, and is greater
+ than or equal to <structfield>allocated_size</structfield>. For
+ fixed-size allocations, anonymous allocations, and free memory this
+ is same as <structfield>allocated_size</structfield>.
</para></entry>
</row>
</tbody>
diff --git a/doc/src/sgml/xfunc.sgml b/doc/src/sgml/xfunc.sgml
index 789cac9fcab..3d25139c334 100644
--- a/doc/src/sgml/xfunc.sgml
+++ b/doc/src/sgml/xfunc.sgml
@@ -3744,6 +3744,60 @@ my_shmem_init(void *arg)
</para>
</sect3>
+ <sect3 id="xfunc-shared-addin-resizable">
+ <title>Resizable shared memory structures</title>
+
+ <para>
+ A resizable memory structure can be requested using
+ <function>ShmemRequestStruct</function> by passing
+ <parameter>.maximum_size</parameter> along with
+ <parameter>.size</parameter>. <parameter>.maximum_size</parameter> is
+ maximum size upto which the structure can grow where as
+ <parameter>.size</parameter> is the initial size of the structure. While
+ contiguous address space worth <parameter>maximum_size</parameter> is
+ allocated to the structure, only memory worth <parameter>size</parameter>
+ bytes is allocated initially. The <function>init_fn</function> should only
+ initialize the <parameter>size</parameter> amount of memory. The actual
+ memory allocated to this structure at any point in time is given by <link
+ linkend="view-pg-shmem-allocations"><structname>pg_shmem_allocations</structname>.<structfield>allocated_size</structfield></link>
+ and the address space reserved for this structure is given by <link
+ linkend="view-pg-shmem-allocations"><structname>pg_shmem_allocations</structname>.<structfield>reserved_space</structfield></link>.
+ </para>
+
+ <para>
+ The structure can be resized using <function>ShmemResizeStruct</function> by
+ passing it the structure's <structname>ShmemStructDesc</structname> and the
+ new size which can be anywhere between 0 to
+ <parameter>maximum_size</parameter>. If the new size is smaller than the
+ current size of the structure, the memory between the new size and current
+ size is freed while keeping the contents of the memory upto new size intact.
+ If the new size is greater than the current size, memory is allocated upto
+ new size while keeping the current contents of the structure intact. The
+ starting address of the structure does not change because of resizing
+ operation. The caller may need to take care of the additional
+ synchronization between the resizing process and the processes using the
+ shared structure. Also accessing the memory beyond the current size of the
+ structure will not cause any segmentation fault or a bus error. Memory will
+ be allocated during such a write access. 0s will be returned on such a read
+ access if memory is not allocated yet. The additional synchronization may
+ use mprotect() with PROT_NONE in every backend that may access this memory
+ to ensure that such an access results in a fault.
+ </para>
+
+ <para>
+ This functionality is available only on the platforms which provide the APIs
+ necessary to reserve contiguous address space and to allocate or free memory
+ in that address space on demand. Macro <symbol>HAVE_RESIZABLE_SHMEM</symbol>
+ is defined on such platforms. It can be used to guard code related to
+ resizing a shared memory structure. The functionality is available on with
+ mmap'ed memory, so subsystems which use resizable structures may have to
+ addtionally disable resizable memory usage when <symbol>shared_memory_type</symbol> is not
+ <symbol>SHMEM_TYPE_MMAP</symbol>. A GUC <xref linkend="guc-have-resizable-shmem"/> is set to
+ <literal>on</literal> when this functionality is available in a running
+ server, <literal>off</literal> otherwise.
+ </para>
+ </sect3>
+
<sect3 id="xfunc-shared-addin-dynamic">
<title>Allocating Dynamic Shared Memory After Startup</title>
diff --git a/meson.build b/meson.build
index 43d5ffc30b1..790845762e1 100644
--- a/meson.build
+++ b/meson.build
@@ -2904,6 +2904,22 @@ decl_checks = [
['timingsafe_bcmp', 'string.h'],
]
+# Linux-specific madvise constants needed for resizable shared memory.
+# Usually we use AC_CHECK_DECLS to check for function declarations, but in this
+# case we are using it to detect existence of constants. These constants are
+# used to define HAVE_RESIZABLE_SHMEM which is used in storage/pg_shmem.h as
+# well as storage/shmem.h. The first abstracts the APIs to allocate shared
+# memory segments from the operating system whereas the second abstracts APIs to
+# allocate shared memory to various subsystems. Since they are related but
+# orthogonal to each other, including any one of them in the other file doesn't
+# make sense. pg_config_manual.h is the only place where HAVE_RESIZABLE_SHMEM
+# can be defined and made available to both without including sys/mman.h. But
+# for that we need constants that indicate the existence of following defines.
+decl_checks += [
+ ['MADV_POPULATE_WRITE', 'sys/mman.h'],
+ ['MADV_REMOVE', 'sys/mman.h'],
+]
+
# Need to check for function declarations for these functions, because
# checking for library symbols wouldn't handle deployment target
# restrictions on macOS
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 2e3886cf9fe..8d859dfbbfb 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -589,6 +589,27 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
return true;
}
+/*
+ * Get the page size being used by the shared memory.
+ *
+ * The function should be called only after the shared memory has been setup.
+ */
+Size
+GetOSPageSize(void)
+{
+ Size os_page_size;
+
+ Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
+
+ os_page_size = sysconf(_SC_PAGESIZE);
+
+ /* If huge pages are actually in use, use huge page size */
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+
+ return os_page_size;
+}
+
/*
* Creates an anonymous mmap()ed shared memory segment.
*
@@ -991,3 +1012,51 @@ PGSharedMemoryDetach(void)
AnonymousShmem = NULL;
}
}
+
+#ifdef HAVE_RESIZABLE_SHMEM
+/*
+ * Make sure that the memory of given size from the given address is released.
+ *
+ * The address and size are expected to be page aligned.
+ *
+ * Only supported on platforms that support anonymous shared memory.
+ */
+void
+PGSharedMemoryEnsureFreed(void *addr, Size size)
+{
+ if (!AnonymousShmem)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("only anonymous shared memory can be freed")));
+
+ Assert(addr == (void *) TYPEALIGN(GetOSPageSize(), addr));
+ Assert(size == TYPEALIGN(GetOSPageSize(), size));
+
+ if (madvise(addr, size, MADV_REMOVE) == -1)
+ ereport(ERROR,
+ (errmsg("could not free shared memory: %m")));
+}
+
+/*
+ * Make sure that the memory of given size from the given address is allocated.
+ *
+ * The address and size are expected to be page aligned.
+ *
+ * Only supported on platforms that support anonymous shared memory.
+ */
+void
+PGSharedMemoryEnsureAllocated(void *addr, Size size)
+{
+ if (!AnonymousShmem)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("only anonymous shared memory can be allocated at runtime")));
+
+ Assert(addr == (void *) TYPEALIGN(GetOSPageSize(), addr));
+ Assert(size == TYPEALIGN(GetOSPageSize(), size));
+
+ if (madvise(addr, size, MADV_POPULATE_WRITE) == -1)
+ ereport(ERROR,
+ (errmsg("could not allocate shared memory: %m")));
+}
+#endif /* HAVE_RESIZABLE_SHMEM */
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 794e4fcb2ad..dc2ee018845 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -648,3 +648,26 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
}
return true;
}
+
+/*
+ * Get the page size used by the shared memory.
+ *
+ * The function should be called only after the shared memory has been setup.
+ */
+Size
+GetOSPageSize(void)
+{
+ SYSTEM_INFO sysinfo;
+ Size os_page_size;
+
+ Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
+
+ GetSystemInfo(&sysinfo);
+ os_page_size = sysinfo.dwPageSize;
+
+ /* If huge pages are actually in use, use huge page size */
+ if (huge_pages_status == HUGE_PAGES_ON)
+ GetHugePageSize(&os_page_size, NULL);
+
+ return os_page_size;
+}
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 1ebffe5a32a..03de5d88d51 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -19,11 +19,11 @@
* methods). The routines in this file are used for allocating and
* binding to shared memory data structures.
*
- * This module provides facilities to allocate fixed-size structures in shared
- * memory, for things like variables shared between all backend processes.
- * Each such structure has a string name to identify it, specified when it is
- * requested. shmem_hash.c provides a shared hash table implementation on top
- * of that.
+ * This module provides facilities to allocate fixed-size as well as resizable
+ * structures in shared memory, for things like variables shared between all
+ * backend processes. Each such structure has a string name to identify it,
+ * specified when it is requested. shmem_hash.c provides a shared hash table
+ * implementation on top of fixed-size structures.
*
* Shared memory areas should usually not be allocated after postmaster
* startup, although we do allow small allocations later for the benefit of
@@ -102,6 +102,21 @@
* (*options->ptr), and calls the attach_fn callback, if any, for additional
* per-backend setup.
*
+ * Resizable shared memory structures
+ * ----------------------------------
+ *
+ * In order to allocate resizable shared memory structures, set
+ * ShmemRequestStructOpts::maximum_size to the maximum size that the structure
+ * can grow to. The address space for the maximum size will be reserved at
+ * startup, but memory is allocated or freed as the structure grows or shrinks
+ * respectively. ShmemRequestStructOpts::size should be set to the initial size
+ * of the structure, which is the amount of memory allocated at the startup.
+ * After startup, the structure can be resized by calling ShmemResizeStruct() by
+ * passing it the ShmemStructDesc for the structure and the new size.
+ *
+ * While resizable structures can be created after the startup, the memory
+ * available for them is quite limited.
+ *
* Legacy ShmemInitStruct()/ShmemInitHash() functions
* --------------------------------------------------
*
@@ -167,6 +182,18 @@ typedef struct
ShmemRequestKind kind;
} ShmemRequest;
+/*
+ * A convenient macro to get the space required for a shmem request consistently.
+ * A resizable structure, requested by non-zero maximum_size, requires space for
+ * its maximum size.
+ */
+#ifdef HAVE_RESIZABLE_SHMEM
+#define SHMEM_REQUEST_SPACE_SIZE(request) \
+ ((request)->options->maximum_size > 0 ? (request)->options->maximum_size : (request)->options->size)
+#else
+#define SHMEM_REQUEST_SPACE_SIZE(request) ((request)->options->size)
+#endif
+
static List *pending_shmem_requests;
/*
@@ -269,6 +296,10 @@ typedef struct
void *location; /* location in shared mem */
Size size; /* # bytes requested for the structure */
Size allocated_size; /* # bytes actually allocated */
+#ifdef HAVE_RESIZABLE_SHMEM
+ Size maximum_size; /* the maximum size the structure can grow to */
+ Size reserved_space; /* the total address space reserved */
+#endif
} ShmemIndexEnt;
/* To get reliable results for NUMA inquiry we need to "touch pages" once */
@@ -277,6 +308,9 @@ static bool firstNumaTouch = true;
static void CallShmemCallbacksAfterStartup(const ShmemCallbacks *callbacks);
static void InitShmemIndexEntry(ShmemRequest *request);
static bool AttachShmemIndexEntry(ShmemRequest *request, bool missing_ok);
+#ifdef HAVE_RESIZABLE_SHMEM
+static Size EstimateAllocatedSize(ShmemIndexEnt *entry);
+#endif
Datum pg_numa_available(PG_FUNCTION_ARGS);
@@ -347,6 +381,11 @@ ShmemRequestInternal(ShmemStructOpts *options, ShmemRequestKind kind)
if (options->size <= 0 && options->size != SHMEM_ATTACH_UNKNOWN_SIZE)
elog(ERROR, "invalid size %zd for shared memory request for \"%s\"",
options->size, options->name);
+#ifdef HAVE_RESIZABLE_SHMEM
+ if (options->maximum_size < 0 && options->maximum_size != SHMEM_ATTACH_UNKNOWN_SIZE)
+ elog(ERROR, "invalid maximum_size %zd for shared memory request for \"%s\"",
+ options->maximum_size, options->name);
+#endif
}
else
{
@@ -355,12 +394,28 @@ ShmemRequestInternal(ShmemStructOpts *options, ShmemRequestKind kind)
if (options->size <= 0)
elog(ERROR, "invalid size %zd for shared memory request for \"%s\"",
options->size, options->name);
+#ifdef HAVE_RESIZABLE_SHMEM
+ if (options->maximum_size == SHMEM_ATTACH_UNKNOWN_SIZE)
+ elog(ERROR, "SHMEM_ATTACH_UNKNOWN_SIZE cannot be used during startup");
+ if (options->maximum_size < 0)
+ elog(ERROR, "invalid maximum_size %zd for shared memory request for \"%s\"",
+ options->maximum_size, options->name);
+#endif
}
if (options->alignment != 0 && pg_nextpower2_size_t(options->alignment) != options->alignment)
elog(ERROR, "invalid alignment %zu for shared memory request for \"%s\"",
options->alignment, options->name);
+#ifdef HAVE_RESIZABLE_SHMEM
+ if (options->maximum_size > 0 && options->size > options->maximum_size)
+ elog(ERROR, "resizable shared memory structure \"%s\" should have maximum size (%zd) greater than size (%zd)",
+ options->name, options->maximum_size, options->size);
+
+ if (options->maximum_size > 0 && shared_memory_type != SHMEM_TYPE_MMAP)
+ elog(ERROR, "resizable shared memory requires shared_memory_type = mmap");
+#endif
+
/* Check that we're in the right state */
if (shmem_request_state != SRS_REQUESTING)
elog(ERROR, "ShmemRequestStruct can only be called from a shmem_request callback");
@@ -382,8 +437,13 @@ ShmemRequestInternal(ShmemStructOpts *options, ShmemRequestKind kind)
}
/*
- * ShmemGetRequestedSize() --- estimate the total size of all registered shared
- * memory structures.
+ * ShmemGetRequestedSize() --- estimate the total size of all registered shared
+ * memory structures.
+ *
+ * When maximum_size is specified when requesting resizable shared memory
+ * structures. We use that, instead of the (initial) size, for the estimation,
+ * to ensure that enough space is reserved for growing the resizable structures
+ * to its maximum size.
*
* This is called at postmaster startup, before the shared memory segment has
* been created.
@@ -408,7 +468,7 @@ ShmemGetRequestedSize(void)
alignment = PG_CACHE_LINE_SIZE;
size = TYPEALIGN(alignment, size);
- size = add_size(size, request->options->size);
+ size = add_size(size, SHMEM_REQUEST_SPACE_SIZE(request));
}
return size;
@@ -515,6 +575,7 @@ InitShmemIndexEntry(ShmemRequest *request)
ShmemIndexEnt *index_entry;
bool found;
size_t allocated_size;
+ size_t requested_size;
void *structPtr;
/* look it up in the shmem index */
@@ -532,10 +593,18 @@ InitShmemIndexEntry(ShmemRequest *request)
}
/*
- * We inserted the entry to the shared memory index. Allocate requested
- * amount of shared memory for it, and initialize the index entry.
+ * We inserted the entry to the shared memory index. Allocate requested
+ * amount of address space in the shared memory segment for it, and do
+ * basic initializion. The memory gets allocated during initialization as
+ * the corresponding memory pages are written to. Allocate enough space
+ * for a resizable structure to grow to its maximum size. It is expected
+ * that the initialization callback will use only as much memory as the
+ * initial size of the resizable structure. (Well, if it doesn't, more
+ * memory will be allocated initially than expected, no further harm is
+ * done.)
*/
- structPtr = ShmemAllocRaw(request->options->size,
+ requested_size = SHMEM_REQUEST_SPACE_SIZE(request);
+ structPtr = ShmemAllocRaw(requested_size,
request->options->alignment,
&allocated_size);
if (structPtr == NULL)
@@ -544,13 +613,22 @@ InitShmemIndexEntry(ShmemRequest *request)
hash_search(ShmemIndex, name, HASH_REMOVE, NULL);
ereport(ERROR,
(errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("not enough shared memory for data structure"
+ errmsg("not enough shared memory space for data structure"
" \"%s\" (%zu bytes requested)",
- name, request->options->size)));
+ name, requested_size)));
}
index_entry->size = request->options->size;
index_entry->allocated_size = allocated_size;
index_entry->location = structPtr;
+#ifdef HAVE_RESIZABLE_SHMEM
+ index_entry->reserved_space = allocated_size;
+ index_entry->maximum_size = request->options->maximum_size;
+ if (request->options->maximum_size > 0)
+ {
+ /* Adjust allocated size of a resizable structure. */
+ index_entry->allocated_size = EstimateAllocatedSize(index_entry);
+ }
+#endif
/* Initialize depending on the kind of shmem area it is */
switch (request->kind)
@@ -595,7 +673,7 @@ AttachShmemIndexEntry(ShmemRequest *request, bool missing_ok)
return false;
}
- /* Check that the size in the index matches the request */
+ /* Check that the sizes in the index match the request. */
if (index_entry->size != request->options->size &&
request->options->size != SHMEM_ATTACH_UNKNOWN_SIZE)
{
@@ -605,6 +683,18 @@ AttachShmemIndexEntry(ShmemRequest *request, bool missing_ok)
name, index_entry->size, request->options->size)));
}
+#ifdef HAVE_RESIZABLE_SHMEM
+ if (index_entry->maximum_size != request->options->maximum_size &&
+ request->options->maximum_size != SHMEM_ATTACH_UNKNOWN_SIZE)
+ {
+ ereport(ERROR,
+ (errmsg("shared memory struct \"%s\" was created with" \
+ " different maximum_size: existing %zu, requested %zu",
+ name, index_entry->maximum_size,
+ request->options->maximum_size)));
+ }
+#endif
+
/*
* Re-establish the caller's pointer variable, or do other actions to
* attach depending on the kind of shmem area it is.
@@ -626,6 +716,115 @@ AttachShmemIndexEntry(ShmemRequest *request, bool missing_ok)
return true;
}
+#ifdef HAVE_RESIZABLE_SHMEM
+/*
+ * Estimate the actual memory allocated for a resizable structure.
+ *
+ * ... based on the assumption that the memory is allocated in pages.
+ *
+ * The memory pages covered by the current size of a resizable structure are
+ * fully allocated when the currently allocated part of the structure is written
+ * to. The memory page where the maximal structure ends also hosts the next
+ * structure, unless the maximal structure ends on a page boundary. Hence that
+ * page is allocated when the next structure is written to. The memory pages
+ * between the page where the current structure ends and the page where the next
+ * structure starts remain unallocated. Thus the memory allocated for a
+ * resizable structure can be estimated as the total address space reserved for
+ * the structure minus the unallocated memory pages between the current end and
+ * the next structure.
+ */
+static Size
+EstimateAllocatedSize(ShmemIndexEnt *entry)
+{
+ Size page_size = GetOSPageSize();
+ char *align_end = (char *) TYPEALIGN(page_size, (char *) entry->location + entry->size);
+ char *floor_max_end = (char *) TYPEALIGN_DOWN(page_size, (char *) entry->location + entry->maximum_size);
+
+ Assert(entry->maximum_size >= entry->size);
+ Assert(entry->reserved_space >= entry->maximum_size);
+
+ if (align_end < floor_max_end)
+ return entry->reserved_space - (floor_max_end - align_end);
+
+ return entry->reserved_space;
+}
+
+/*
+ * ShmemResizeStruct() --- resize a resizable shared memory structure.
+ *
+ * If the structure is being shrunk, the memory pages that are no longer needed
+ * are freed. If the structure is being expanded, the memory pages that are
+ * needed for the new size are allocated. See EstimateAllocatedSize() for
+ * explanation of which pages are allocated for a resizable structure.
+ */
+void
+ShmemResizeStruct(const char *name, Size new_size)
+{
+ ShmemIndexEnt *result;
+ bool found;
+ Size page_size = GetOSPageSize();
+ char *new_end;
+
+ Assert(new_size > 0);
+
+ /*
+ * Resizable shared memory structures are only supported with mmap'ed
+ * memory.
+ */
+ Assert(shared_memory_type == SHMEM_TYPE_MMAP);
+
+ /* look it up in the shmem index */
+ LWLockAcquire(ShmemIndexLock, LW_EXCLUSIVE);
+ result = (ShmemIndexEnt *) hash_search(ShmemIndex, name, HASH_FIND, &found);
+ if (!found)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("shmem struct \"%s\" is not initialized", name)));
+
+ Assert(result);
+
+ if (result->maximum_size <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("shared memory struct \"%s\" is not resizable", name)));
+
+ if (result->maximum_size < new_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough address space is reserved for resizing structure \"%s\"" \
+ "(required %zu bytes, reserved %zu bytes)",
+ name, new_size, result->maximum_size)));
+
+ /*
+ * When shrinking the memory from the page aligned new end to the start of
+ * the page containing end of the reserved space is not required. Whereas
+ * when expanding the memory from the start of the page containing the
+ * start of the structure to the page aligned new end is required.
+ */
+ new_end = (char *) TYPEALIGN(page_size, (char *) result->location + new_size);
+ if (new_size < result->size)
+ {
+ char *max_end = (char *) TYPEALIGN_DOWN(page_size, (char *) result->location + result->maximum_size);
+
+ if (max_end > new_end)
+ PGSharedMemoryEnsureFreed(new_end, max_end - new_end);
+ }
+ else if (new_size > result->size)
+ {
+ char *struct_start = (char *) TYPEALIGN_DOWN(page_size, (char *) result->location);
+
+ if (new_end > struct_start)
+ PGSharedMemoryEnsureAllocated(struct_start, new_end - struct_start);
+ }
+
+ /* Update shmem index entry. */
+ result->size = new_size;
+ result->allocated_size = EstimateAllocatedSize(result);
+
+ LWLockRelease(ShmemIndexLock);
+}
+#endif /* HAVE_RESIZABLE_SHMEM */
+
/*
* InitShmemAllocator() --- set up basic pointers to shared memory.
*
@@ -732,6 +931,10 @@ InitShmemAllocator(PGShmemHeader *seghdr)
Assert(!found);
result->size = ShmemAllocator->index_size;
result->allocated_size = ShmemAllocator->index_size;
+#ifdef HAVE_RESIZABLE_SHMEM
+ result->maximum_size = 0;
+ result->reserved_space = result->allocated_size;
+#endif
result->location = ShmemAllocator->index;
}
}
@@ -1075,7 +1278,7 @@ mul_size(Size s1, Size s2)
Datum
pg_get_shmem_allocations(PG_FUNCTION_ARGS)
{
-#define PG_GET_SHMEM_SIZES_COLS 4
+#define PG_GET_SHMEM_SIZES_COLS 6
ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
HASH_SEQ_STATUS hstat;
ShmemIndexEnt *ent;
@@ -1097,7 +1300,23 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS)
values[1] = Int64GetDatum((char *) ent->location - (char *) ShmemSegHdr);
values[2] = Int64GetDatum(ent->size);
values[3] = Int64GetDatum(ent->allocated_size);
+#ifdef HAVE_RESIZABLE_SHMEM
+ values[4] = Int64GetDatum(ent->maximum_size);
+ values[5] = Int64GetDatum(ent->reserved_space);
+
+ /*
+ * Keep track of the total reserved space for named shmem areas, to be
+ * able to calculate the amount of shared memory allocated for
+ * anonymous areas and the amount of free shared memory at the end of
+ * the segment.
+ */
+ named_allocated += ent->reserved_space;
+#else
+ values[4] = Int64GetDatum(0);
+ values[5] = Int64GetDatum(ent->allocated_size);
+
named_allocated += ent->allocated_size;
+#endif
tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc,
values, nulls);
@@ -1108,6 +1327,8 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS)
nulls[1] = true;
values[2] = Int64GetDatum(ShmemAllocator->free_offset - named_allocated);
values[3] = values[2];
+ values[4] = Int64GetDatum(0);
+ values[5] = values[2];
tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
/* output as-of-yet unused shared memory */
@@ -1116,6 +1337,8 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS)
nulls[1] = false;
values[2] = Int64GetDatum(ShmemSegHdr->totalsize - ShmemAllocator->free_offset);
values[3] = values[2];
+ values[4] = Int64GetDatum(0);
+ values[5] = values[2];
tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
LWLockRelease(ShmemIndexLock);
@@ -1303,23 +1526,9 @@ pg_get_shmem_allocations_numa(PG_FUNCTION_ARGS)
Size
pg_get_shmem_pagesize(void)
{
- Size os_page_size;
-#ifdef WIN32
- SYSTEM_INFO sysinfo;
-
- GetSystemInfo(&sysinfo);
- os_page_size = sysinfo.dwPageSize;
-#else
- os_page_size = sysconf(_SC_PAGESIZE);
-#endif
-
Assert(IsUnderPostmaster);
- Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
-
- if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
- return os_page_size;
+ return GetOSPageSize();
}
Datum
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 7a8a5d0764c..18cb2516d9a 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -1211,6 +1211,13 @@
max => '1000.0',
},
+{ name => 'have_resizable_shmem', type => 'bool', context => 'PGC_INTERNAL', group => 'PRESET_OPTIONS',
+ short_desc => 'Shows whether the running server supports resizable shared memory.',
+ flags => 'GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE',
+ variable => 'have_resizable_shmem_enabled',
+ boot_val => 'HAVE_RESIZABLE_SHMEM_ENABLED',
+},
+
{ name => 'hba_file', type => 'string', context => 'PGC_POSTMASTER', group => 'FILE_LOCATIONS',
short_desc => 'Sets the server\'s "hba" configuration file.',
flags => 'GUC_SUPERUSER_ONLY',
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index d9ca13baff9..6bb08dd10f1 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -653,6 +653,13 @@ static bool assert_enabled = DEFAULT_ASSERT_ENABLED;
#endif
static bool exec_backend_enabled = EXEC_BACKEND_ENABLED;
+#ifdef HAVE_RESIZABLE_SHMEM
+#define HAVE_RESIZABLE_SHMEM_ENABLED true
+#else
+#define HAVE_RESIZABLE_SHMEM_ENABLED false
+#endif
+static bool have_resizable_shmem_enabled = HAVE_RESIZABLE_SHMEM_ENABLED;
+
static char *recovery_target_timeline_string;
static char *recovery_target_string;
static char *recovery_target_xid_string;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3ea17fc5629..32945d73f36 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8702,8 +8702,8 @@
{ oid => '5052', descr => 'allocations from the main shared memory segment',
proname => 'pg_get_shmem_allocations', prorows => '50', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,int8,int8}', proargmodes => '{o,o,o,o}',
- proargnames => '{name,off,size,allocated_size}',
+ proallargtypes => '{text,int8,int8,int8,int8,int8}', proargmodes => '{o,o,o,o,o,o}',
+ proargnames => '{name,off,size,allocated_size,maximum_size,reserved_space}',
prosrc => 'pg_get_shmem_allocations',
proacl => '{POSTGRES=X,pg_read_all_stats=X}' },
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 9f6d512347e..8f2a59ec3a8 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -85,6 +85,14 @@
don't. */
#undef HAVE_DECL_F_FULLFSYNC
+/* Define to 1 if you have the declaration of `MADV_POPULATE_WRITE', and to 0
+ if you don't. */
+#undef HAVE_DECL_MADV_POPULATE_WRITE
+
+/* Define to 1 if you have the declaration of `MADV_REMOVE', and to 0 if you
+ don't. */
+#undef HAVE_DECL_MADV_REMOVE
+
/* Define to 1 if you have the declaration of `memset_s', and to 0 if you
don't. */
#undef HAVE_DECL_MEMSET_S
diff --git a/src/include/pg_config_manual.h b/src/include/pg_config_manual.h
index 521b49b8888..b09d6c91324 100644
--- a/src/include/pg_config_manual.h
+++ b/src/include/pg_config_manual.h
@@ -131,6 +131,15 @@
#define EXEC_BACKEND
#endif
+/*
+ * HAVE_RESIZABLE_SHMEM indicates whether resizable shared memory structures are
+ * supported. The implementation requires Linux-specific madvise constants
+ * (MADV_REMOVE and MADV_POPULATE_WRITE).
+ */
+#if HAVE_DECL_MADV_REMOVE && HAVE_DECL_MADV_POPULATE_WRITE && !defined(EXEC_BACKEND)
+#define HAVE_RESIZABLE_SHMEM
+#endif
+
/*
* USE_POSIX_FADVISE controls whether Postgres will attempt to use the
* posix_fadvise() kernel call. Usually the automatic configure tests are
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 10c7b065861..3d5aceba59c 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -89,6 +89,11 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
+#ifdef HAVE_RESIZABLE_SHMEM
+extern void PGSharedMemoryEnsureFreed(void *addr, Size size);
+extern void PGSharedMemoryEnsureAllocated(void *addr, Size size);
+#endif
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern Size GetOSPageSize(void);
#endif /* PG_SHMEM_H */
diff --git a/src/include/storage/shmem.h b/src/include/storage/shmem.h
index af7fe893bc4..f356027e500 100644
--- a/src/include/storage/shmem.h
+++ b/src/include/storage/shmem.h
@@ -57,6 +57,19 @@ typedef struct ShmemStructOpts
*/
size_t alignment;
+#ifdef HAVE_RESIZABLE_SHMEM
+
+ /*
+ * Maximum size this structure can grow upto in future. The memory is not
+ * allocated right away but the corresponding address space is reserved so
+ * that memory can be mapped to it when the structure grows. Typically
+ * should be used for large resizable structures which need several pages
+ * worth of contiguous memory. Should be set to 0 for fixed-size
+ * structures.
+ */
+ ssize_t maximum_size;
+#endif
+
/*
* When the shmem area is initialized or attached to, pointer to it is
* stored in *ptr. It usually points to a global variable, used to access
@@ -168,6 +181,9 @@ typedef struct ShmemCallbacks
extern void RegisterShmemCallbacks(const ShmemCallbacks *callbacks);
extern bool ShmemAddrIsValid(const void *addr);
+#ifdef HAVE_RESIZABLE_SHMEM
+extern void ShmemResizeStruct(const char *name, Size new_size);
+#endif
/*
* These macros provide syntactic sugar for calling the underlying functions
diff --git a/src/test/modules/Makefile b/src/test/modules/Makefile
index f1b04c99969..2a1e746bf0c 100644
--- a/src/test/modules/Makefile
+++ b/src/test/modules/Makefile
@@ -14,6 +14,7 @@ SUBDIRS = \
libpq_pipeline \
oauth_validator \
plsample \
+ resizable_shmem \
spgist_name_ops \
test_aio \
test_binaryheap \
diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build
index fc99552d9ab..cd94e1fea15 100644
--- a/src/test/modules/meson.build
+++ b/src/test/modules/meson.build
@@ -13,6 +13,7 @@ subdir('libpq_pipeline')
subdir('nbtree')
subdir('oauth_validator')
subdir('plsample')
+subdir('resizable_shmem')
subdir('spgist_name_ops')
subdir('ssl_passphrase_callback')
subdir('test_aio')
diff --git a/src/test/modules/resizable_shmem/Makefile b/src/test/modules/resizable_shmem/Makefile
new file mode 100644
index 00000000000..86bf17bef4a
--- /dev/null
+++ b/src/test/modules/resizable_shmem/Makefile
@@ -0,0 +1,25 @@
+# src/test/modules/resizable_shmem/Makefile
+
+PGFILEDESC = "resizable_shmem - test module for resizable shared memory"
+
+MODULES = resizable_shmem
+
+EXTENSION = resizable_shmem
+DATA = resizable_shmem--1.0.sql
+
+TAP_TESTS = 1
+
+# This test requires library to be loaded at the server start, so disable
+# installcheck
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/resizable_shmem
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/resizable_shmem/meson.build b/src/test/modules/resizable_shmem/meson.build
new file mode 100644
index 00000000000..493bbbc95c3
--- /dev/null
+++ b/src/test/modules/resizable_shmem/meson.build
@@ -0,0 +1,36 @@
+# src/test/modules/resizable_shmem/meson.build
+
+resizable_shmem_sources = files(
+ 'resizable_shmem.c',
+)
+
+if host_system == 'windows'
+ resizable_shmem_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'resizable_shmem',
+ '--FILEDESC', 'resizable_shmem - test module for resizable shared memory',])
+endif
+
+resizable_shmem = shared_module('resizable_shmem',
+ resizable_shmem_sources,
+ kwargs: pg_test_mod_args,
+)
+test_install_libs += resizable_shmem
+
+test_install_data += files(
+ 'resizable_shmem.control',
+ 'resizable_shmem--1.0.sql',
+)
+
+tests += {
+ 'name': 'resizable_shmem',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'tap': {
+ 'tests': [
+ 't/001_resizable_shmem.pl',
+ ],
+ # This test requires library to be loaded at the server start, so disable
+ # installcheck
+ 'runningcheck': false,
+ },
+}
diff --git a/src/test/modules/resizable_shmem/resizable_shmem--1.0.sql b/src/test/modules/resizable_shmem/resizable_shmem--1.0.sql
new file mode 100644
index 00000000000..c1bcb6117b6
--- /dev/null
+++ b/src/test/modules/resizable_shmem/resizable_shmem--1.0.sql
@@ -0,0 +1,37 @@
+/* src/test/modules/resizable_shmem/resizable_shmem--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION resizable_shmem" to load this file. \quit
+
+-- Function to resize the test structure in the shared memory
+CREATE FUNCTION resizable_shmem_resize(new_entries integer)
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C STRICT;
+
+-- Function to write data to all entries in the test structure in shared memory
+-- Writing all the entries makes sure that the memory is actually allocated and
+-- mapped to the process, so that we can later measure the memory usage.
+CREATE FUNCTION resizable_shmem_write(entry_value integer)
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C STRICT;
+
+-- Function to verify that specified number of initial entries have expected value.
+-- Reading all the entries makes sure that the memory is actually mapped to the
+-- process, so that we can later measure the memory usage.
+CREATE FUNCTION resizable_shmem_read(entry_count integer, entry_value integer)
+RETURNS boolean
+AS 'MODULE_PATHNAME'
+LANGUAGE C STRICT;
+
+-- Function to report memory usage statistics of the calling backend
+CREATE FUNCTION resizable_shmem_usage(OUT rss_anon bigint, OUT rss_file bigint, OUT rss_shmem bigint, OUT vm_size bigint)
+AS 'MODULE_PATHNAME'
+LANGUAGE C STRICT;
+
+-- Function to get the shared memory page size
+CREATE FUNCTION resizable_shmem_pagesize()
+RETURNS integer
+AS 'MODULE_PATHNAME'
+LANGUAGE C STRICT;
diff --git a/src/test/modules/resizable_shmem/resizable_shmem.c b/src/test/modules/resizable_shmem/resizable_shmem.c
new file mode 100644
index 00000000000..5ae2d2e2d1d
--- /dev/null
+++ b/src/test/modules/resizable_shmem/resizable_shmem.c
@@ -0,0 +1,326 @@
+/* -------------------------------------------------------------------------
+ *
+ * resizable_shmem.c
+ * Test module for PostgreSQL's resizable shared memory functionality
+ *
+ * This module demonstrates and tests the resizable shared memory API
+ * provided by shmem.c/shmem.h.
+ *
+ * -------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "commands/extension.h"
+#include "fmgr.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "storage/shmem.h"
+#include "storage/spin.h"
+#include "utils/builtins.h"
+#include "utils/guc.h"
+#include "utils/memutils.h"
+#include "utils/timestamp.h"
+#include "access/htup_details.h"
+
+#include <stdio.h>
+
+PG_MODULE_MAGIC;
+
+/* Default values for the GUCs controlling structure size */
+#define TEST_INITIAL_ENTRIES_DEFAULT (25 * 1024 * 1024) /* ~100MB */
+#define TEST_MAX_ENTRIES_DEFAULT (100 * 1024 * 1024) /* ~400MB */
+
+#define TEST_ENTRY_SIZE sizeof(int32) /* Size of each entry */
+
+/*
+ * Resizable test data structure stored in shared memory.
+ *
+ * The test performs resizing, reads or writes, only one at a time and never
+ * concurrently. Hence, there is no need for locks in the test structure.
+ */
+typedef struct TestResizableShmemStruct
+{
+ /* Metadata */
+ int32 num_entries; /* Number of entries that can fit */
+
+ /* Data area - variable size */
+ int32 data[FLEXIBLE_ARRAY_MEMBER];
+} TestResizableShmemStruct;
+
+static TestResizableShmemStruct *resizable_shmem = NULL;
+
+/* GUC variables controlling the size of the test structure */
+static int test_initial_entries;
+static int test_max_entries;
+
+/* Whether to use SHMEM_ATTACH_UNKNOWN_SIZE when attaching to the shared memory */
+static bool use_unknown_size = false;
+
+static void resizable_shmem_request(void *arg);
+static void resizable_shmem_shmem_init(void *arg);
+
+static ShmemCallbacks shmem_callbacks = {
+ .request_fn = resizable_shmem_request,
+ .init_fn = resizable_shmem_shmem_init,
+};
+
+/* SQL-callable functions */
+PG_FUNCTION_INFO_V1(resizable_shmem_resize);
+PG_FUNCTION_INFO_V1(resizable_shmem_write);
+PG_FUNCTION_INFO_V1(resizable_shmem_read);
+PG_FUNCTION_INFO_V1(resizable_shmem_usage);
+PG_FUNCTION_INFO_V1(resizable_shmem_pagesize);
+
+/*
+ * Module load callback
+ */
+void
+_PG_init(void)
+{
+ int guc_context;
+
+ /*
+ * Use PGC_POSTMASTER when loaded at startup so the values are fixed once
+ * the shared memory segment is created. When loaded after startup
+ * PGC_POSTMASTER is not allowed, so we use PGC_SIGHUP instead. Although
+ * we do not intend to change these values at config reload, PGC_SIGHUP is
+ * the least permissive context that allows defining the GUC after startup
+ * and still prevents it from being changed via SET.
+ */
+ if (process_shared_preload_libraries_in_progress)
+ guc_context = PGC_POSTMASTER;
+ else
+ {
+ guc_context = PGC_SIGHUP;
+ shmem_callbacks.flags = SHMEM_CALLBACKS_ALLOW_AFTER_STARTUP;
+ }
+
+ DefineCustomIntVariable("resizable_shmem.initial_entries",
+ "Initial number of entries in the test structure.",
+ NULL,
+ &test_initial_entries,
+ TEST_INITIAL_ENTRIES_DEFAULT,
+ 1,
+ INT_MAX,
+ guc_context,
+ 0,
+ NULL, NULL, NULL);
+
+ DefineCustomIntVariable("resizable_shmem.max_entries",
+ "Maximum number of entries in the test structure.",
+ NULL,
+ &test_max_entries,
+ TEST_MAX_ENTRIES_DEFAULT,
+ 1,
+ INT_MAX,
+ guc_context,
+ 0,
+ NULL, NULL, NULL);
+
+ /*
+ * When loaded after startup by a backend that is not creating the
+ * extension, the shared memory might have been resized to a size other
+ * than the initial size. Use SHMEM_ATTACH_UNKNOWN_SIZE to attach without
+ * knowing the exact size.
+ */
+ if (!process_shared_preload_libraries_in_progress && !creating_extension)
+ use_unknown_size = true;
+
+ RegisterShmemCallbacks(&shmem_callbacks);
+}
+
+/*
+ * Request shared memory resources
+ */
+static void
+resizable_shmem_request(void *arg)
+{
+ Size initial_size = add_size(offsetof(TestResizableShmemStruct, data),
+ mul_size(test_initial_entries, TEST_ENTRY_SIZE));
+#ifdef HAVE_RESIZABLE_SHMEM
+ Size max_size = add_size(offsetof(TestResizableShmemStruct, data),
+ mul_size(test_max_entries, TEST_ENTRY_SIZE));
+
+ /* A preprocessor macro to conditionally include the maximum_size field. */
+#define MAXIMUM_SIZE_ARG .maximum_size = max_size,
+#else
+#define MAXIMUM_SIZE_ARG
+#endif
+
+ /* Register our resizable shared memory structure */
+ ShmemRequestStruct(.name = "resizable_shmem",
+ .size = use_unknown_size ? SHMEM_ATTACH_UNKNOWN_SIZE : initial_size,
+ MAXIMUM_SIZE_ARG
+ .ptr = (void **) &resizable_shmem,
+ );
+}
+
+/*
+ * Initialize shared memory structure
+ */
+static void
+resizable_shmem_shmem_init(void *arg)
+{
+ /*
+ * Shared memory structure should have been already allocated. Initialize
+ * it.
+ */
+ Assert(resizable_shmem != NULL);
+
+ resizable_shmem->num_entries = test_initial_entries;
+ memset(resizable_shmem->data, 0, mul_size(test_initial_entries, TEST_ENTRY_SIZE));
+}
+
+/*
+ * Resize the shared memory structure to accommodate the specified number of
+ * entries.
+ */
+Datum
+resizable_shmem_resize(PG_FUNCTION_ARGS)
+{
+#ifdef HAVE_RESIZABLE_SHMEM
+ int32 new_entries = PG_GETARG_INT32(0);
+ Size new_size;
+
+ if (!resizable_shmem)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("resizable_shmem is not initialized")));
+
+ new_size = add_size(offsetof(TestResizableShmemStruct, data),
+ mul_size(new_entries, TEST_ENTRY_SIZE));
+ ShmemResizeStruct("resizable_shmem", new_size);
+ resizable_shmem->num_entries = new_entries;
+
+ PG_RETURN_VOID();
+#else
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("resizable shared memory is not supported on this platform")));
+#endif
+}
+
+/*
+ * Write the given integer value to all entries in the data array.
+ */
+Datum
+resizable_shmem_write(PG_FUNCTION_ARGS)
+{
+ int32 entry_value = PG_GETARG_INT32(0);
+ int32 i;
+
+ if (!resizable_shmem)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("resizable_shmem is not initialized")));
+
+ /* Write the value to all current entries */
+ for (i = 0; i < resizable_shmem->num_entries; i++)
+ resizable_shmem->data[i] = entry_value;
+
+ PG_RETURN_VOID();
+}
+
+/*
+ * Check whether the first 'entry_count' entries all have the expected 'entry_value'.
+ * Returns true if all match, false otherwise.
+ */
+Datum
+resizable_shmem_read(PG_FUNCTION_ARGS)
+{
+ int32 entry_count = PG_GETARG_INT32(0);
+ int32 entry_value = PG_GETARG_INT32(1);
+ int32 i;
+
+ if (resizable_shmem == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("resizable_shmem is not initialized")));
+
+ if (entry_count < 0 || entry_count > resizable_shmem->num_entries)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("entry_count %d is out of range (0..%d)", entry_count, resizable_shmem->num_entries)));
+
+ for (i = 0; i < entry_count; i++)
+ {
+ if (resizable_shmem->data[i] != entry_value)
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
+
+/*
+ * Report multiple memory usage statistics of the calling backend process
+ * as reported by the kernel.
+ * Returns RssAnon, RssFile, RssShmem, VmSize from /proc/self/status as a record.
+ *
+ * The function assumes that these values will be available in
+ * /proc/self/status, any system which also support madvise with MADV_REMOVE and
+ * MADV_POPULATE_WRITE.
+ */
+Datum
+resizable_shmem_usage(PG_FUNCTION_ARGS)
+{
+ FILE *f;
+ char line[256];
+ int64 rss_anon_kb = -1;
+ int64 rss_file_kb = -1;
+ int64 rss_shmem_kb = -1;
+ int64 vm_size_kb = -1;
+ int found = 0;
+ TupleDesc tupdesc;
+ Datum values[4];
+ bool nulls[4];
+ HeapTuple tuple;
+
+ /* Open /proc/self/status to read memory information */
+ f = fopen("/proc/self/status", "r");
+ if (f == NULL)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not open /proc/self/status: %m")));
+
+ /* Look for the memory usage lines */
+ while (fgets(line, sizeof(line), f) != NULL && found < 4)
+ {
+ if (rss_anon_kb == -1 && sscanf(line, "RssAnon: %ld kB", &rss_anon_kb) == 1)
+ found++;
+ else if (rss_file_kb == -1 && sscanf(line, "RssFile: %ld kB", &rss_file_kb) == 1)
+ found++;
+ else if (rss_shmem_kb == -1 && sscanf(line, "RssShmem: %ld kB", &rss_shmem_kb) == 1)
+ found++;
+ else if (vm_size_kb == -1 && sscanf(line, "VmSize: %ld kB", &vm_size_kb) == 1)
+ found++;
+ }
+
+ fclose(f);
+
+ /* Build tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("function returning record called in context "
+ "that cannot accept a record")));
+
+ /* Build the result tuple */
+ values[0] = Int64GetDatum(rss_anon_kb >= 0 ? rss_anon_kb * 1024 : 0);
+ values[1] = Int64GetDatum(rss_file_kb >= 0 ? rss_file_kb * 1024 : 0);
+ values[2] = Int64GetDatum(rss_shmem_kb >= 0 ? rss_shmem_kb * 1024 : 0);
+ values[3] = Int64GetDatum(vm_size_kb >= 0 ? vm_size_kb * 1024 : 0);
+
+ nulls[0] = nulls[1] = nulls[2] = nulls[3] = false;
+
+ tuple = heap_form_tuple(tupdesc, values, nulls);
+ PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+}
+
+/*
+ * resizable_shmem_pagesize() - Get the shared memory page size
+ */
+Datum
+resizable_shmem_pagesize(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_INT32(pg_get_shmem_pagesize());
+}
diff --git a/src/test/modules/resizable_shmem/resizable_shmem.control b/src/test/modules/resizable_shmem/resizable_shmem.control
new file mode 100644
index 00000000000..8031303fe0e
--- /dev/null
+++ b/src/test/modules/resizable_shmem/resizable_shmem.control
@@ -0,0 +1,4 @@
+# resizable_shmem extension test module
+comment = 'test module for testing resizable shared memory structure functionality'
+default_version = '1.0'
+module_pathname = '$libdir/resizable_shmem'
diff --git a/src/test/modules/resizable_shmem/t/001_resizable_shmem.pl b/src/test/modules/resizable_shmem/t/001_resizable_shmem.pl
new file mode 100644
index 00000000000..6d45b1eccdc
--- /dev/null
+++ b/src/test/modules/resizable_shmem/t/001_resizable_shmem.pl
@@ -0,0 +1,239 @@
+# Copyright (c) 2025-2026, PostgreSQL Global Development Group
+
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+# Test resizable shared memory functionality, both when loaded at startup via
+# shared_preload_libraries and when loaded after startup (late allocation).
+
+# Verify that RssShmem does not exceed the total allocated shared memory.
+# Allocated shared memory should be mostly the memory allocated to the
+# resizable_shmem structure. Any large increase in expected RssShmem should
+# reflect the unexpected increase in memory allocated to the resizable_shmem
+# structure.
+sub check_shmem_usage
+{
+ my ($session, $label, $node) = @_;
+
+ my $rss_shmem = $session->query_safe('SELECT rss_shmem FROM resizable_shmem_usage();',
+ verbose => 0);
+ my $total_alloc = $node->safe_psql('postgres',
+ "SELECT sum(allocated_size) FROM pg_shmem_allocations;");
+
+ note "$label: RssShmem=$rss_shmem, sum(allocated_size)=$total_alloc";
+ ok($rss_shmem <= $total_alloc, "$label: RssShmem does not exceed total allocated size");
+}
+
+# Test a resize operation: resize, verify old data, write new data, verify
+# new data, and check shmem usage. Returns updated ($num_entries, $value).
+sub test_resize
+{
+ my ($node, $prefix, $old_num_entries, $old_value, $new_num_entries, $new_value, $label) = @_;
+
+ $label = "$prefix: $label";
+
+ my $session1 = $node->background_psql('postgres');
+ my $session2 = $node->background_psql('postgres');
+
+ $session1->query_safe("SELECT resizable_shmem_resize($new_num_entries);",
+ verbose => 0);
+
+ # Old data should still be intact in the (possibly smaller) area
+ my $readable_entries = ($new_num_entries < $old_num_entries) ? $new_num_entries : $old_num_entries;
+ is($session1->query_safe("SELECT resizable_shmem_read($readable_entries, $old_value);",
+ verbose => 0),
+ 't', "old data readable after $label");
+
+ $session2->query_safe("SELECT resizable_shmem_write($new_value);",
+ verbose => 0);
+ is($session1->query_safe("SELECT resizable_shmem_read($new_num_entries, $new_value);",
+ verbose => 0),
+ 't', "new data readable after $label");
+
+ check_shmem_usage($session1, "$label (session 1)", $node);
+ check_shmem_usage($session2, "$label (session 2)", $node);
+
+ $session1->quit;
+ $session2->quit;
+
+ return ($new_num_entries, $new_value);
+}
+
+# Run the full suite of resizable shared memory tests on the given node.
+sub run_resizable_tests
+{
+ my ($node, $initial_entries, $max_entries, $prefix) = @_;
+
+ my $have_resizable_shmem = $node->safe_psql('postgres', 'SHOW have_resizable_shmem;') eq 'on';
+
+ my $num_entries = $initial_entries;
+
+ # Basic read/write should work on all platforms
+ my $value = 100;
+ $node->safe_psql('postgres', "SELECT resizable_shmem_write($value);");
+ is($node->safe_psql('postgres', "SELECT resizable_shmem_read($num_entries, $value);"),
+ 't', "$prefix: data read after write successful");
+
+ if ($have_resizable_shmem)
+ {
+ # Initial structure state
+ my $session1 = $node->background_psql('postgres');
+ my $session2 = $node->background_psql('postgres');
+
+ $value = 100;
+ # Write and read the initial set of entries.
+ $session1->query_safe("SELECT resizable_shmem_write($value);", verbose => 0);
+ is($session2->query_safe("SELECT resizable_shmem_read($num_entries, $value);",
+ verbose => 0),
+ 't', "$prefix: data read after write successful");
+ check_shmem_usage($session1, "$prefix: initial write (session 1)", $node);
+ check_shmem_usage($session2, "$prefix: initial write (session 2)", $node);
+ $session1->quit;
+ $session2->quit;
+
+ # Verify no other structure is resizable
+ is($node->safe_psql('postgres', "SELECT count(*) FROM pg_shmem_allocations WHERE name <> 'resizable_shmem' AND maximum_size <> 0;"),
+ '0', "$prefix: no other resizable structures");
+
+ # Resize to maximum
+ ($num_entries, $value) = test_resize($node, $prefix, $num_entries, $value,
+ $max_entries, 500, 'resize to maximum');
+
+ # Shrink to 75% of max
+ my $shrink_entries = int($max_entries * 3 / 4);
+ ($num_entries, $value) = test_resize($node, $prefix, $num_entries, $value,
+ $shrink_entries, 999, 'shrinking');
+
+ # Resize to the same size (no-op)
+ ($num_entries, $value) = test_resize($node, $prefix, $num_entries, $value,
+ $num_entries, 1999, 'no-op resize');
+
+ # Test resize failure (attempt to resize beyond max - should fail)
+ my ($ret, $stdout, $stderr) =
+ $node->psql('postgres', "SELECT resizable_shmem_resize(" . ($max_entries * 2) . ");");
+ ok($ret != 0 || $stderr =~ /ERROR/, "$prefix: Resize beyond maximum fails");
+ }
+ else
+ {
+ # On unsupported platforms, resizing should fail with a clear error
+ my ($ret, $stdout, $stderr) =
+ $node->psql('postgres', "SELECT resizable_shmem_resize($num_entries);");
+ ok($ret != 0, "$prefix: resize fails on unsupported platform");
+ like($stderr, qr/not supported/, "$prefix: resize error mentions not supported");
+ }
+}
+
+### Set up a test node.
+#
+#Configure minimal shared memory so that the resizable_shmem structure dominates
+#and any unexpected increase is easy to detect.
+#
+# Also disable huge pages so that RssShmem and allocated_size are comparable.
+# The latter is already aligned to the default page size.
+###
+my $node = PostgreSQL::Test::Cluster->new('resizable_shmem');
+$node->init;
+
+$node->append_conf('postgresql.conf', 'huge_pages = off');
+$node->append_conf('postgresql.conf', 'shared_buffers = 128kB');
+$node->append_conf('postgresql.conf', 'max_connections = 5');
+$node->append_conf('postgresql.conf', 'max_worker_processes = 0');
+$node->append_conf('postgresql.conf', 'max_wal_senders = 0');
+$node->append_conf('postgresql.conf', 'max_prepared_transactions = 0');
+$node->append_conf('postgresql.conf', 'max_locks_per_transaction = 10');
+$node->append_conf('postgresql.conf', 'max_pred_locks_per_transaction = 10');
+$node->append_conf('postgresql.conf', 'wal_buffers = 32kB');
+
+###
+# Test 1: Startup allocation via shared_preload_libraries
+###
+my $startup_initial = 25 * 1024 * 1024;
+my $startup_max = 100 * 1024 * 1024;
+
+$node->append_conf('postgresql.conf', 'shared_preload_libraries = resizable_shmem');
+$node->append_conf('postgresql.conf', "resizable_shmem.initial_entries = $startup_initial");
+$node->append_conf('postgresql.conf', "resizable_shmem.max_entries = $startup_max");
+$node->start;
+$node->safe_psql('postgres', 'CREATE EXTENSION resizable_shmem;');
+run_resizable_tests($node, $startup_initial, $startup_max, 'startup');
+
+my $have_resizable_shmem = $node->safe_psql('postgres', 'SHOW have_resizable_shmem;') eq 'on';
+
+###
+# Test 2: Late allocation (loaded after startup, not in shared_preload_libraries).
+# Use much smaller sizes since only ~100KB of shared memory is available for
+# structures allocated after startup.
+###
+my $late_initial = 5 * 1024;
+my $late_max = 12 * 1024;
+
+$node->safe_psql('postgres', qq{
+ ALTER SYSTEM RESET shared_preload_libraries;
+ ALTER SYSTEM SET resizable_shmem.initial_entries = $late_initial;
+ ALTER SYSTEM SET resizable_shmem.max_entries = $late_max;
+});
+$node->safe_psql('postgres', 'DROP EXTENSION resizable_shmem;');
+$node->restart;
+
+$node->safe_psql('postgres', 'CREATE EXTENSION resizable_shmem;');
+run_resizable_tests($node, $late_initial, $late_max, 'late');
+
+###
+# Test sysv shared memory does not support resizable shmem. Only relevant on
+# platforms that support resizable shmem (HAVE_RESIZABLE_SHMEM), since the
+# module only sets maximum_size in that case.
+###
+if ($have_resizable_shmem)
+{
+ ###
+ # Test 3: Verify that CREATE EXTENSION fails with sysv shared memory
+ # when loaded after startup (not in shared_preload_libraries).
+ ###
+ $node->safe_psql('postgres', 'DROP EXTENSION resizable_shmem;');
+
+ # Remove settings that would cause the library to auto-load at startup:
+ # shared_preload_libraries and module-prefixed GUCs. ALTER SYSTEM RESET
+ # only affects postgresql.auto.conf, so we must use adjust_conf to remove
+ # from postgresql.conf.
+ $node->adjust_conf('postgresql.conf', 'shared_preload_libraries', undef);
+ $node->adjust_conf('postgresql.conf', 'resizable_shmem.initial_entries', undef);
+ $node->adjust_conf('postgresql.conf', 'resizable_shmem.max_entries', undef);
+ $node->adjust_conf('postgresql.auto.conf', 'shared_preload_libraries', undef);
+ $node->adjust_conf('postgresql.auto.conf', 'resizable_shmem.initial_entries', undef);
+ $node->adjust_conf('postgresql.auto.conf', 'resizable_shmem.max_entries', undef);
+ $node->safe_psql('postgres', qq{
+ ALTER SYSTEM SET shared_memory_type = 'sysv';
+ });
+
+ $node->restart;
+
+ my ($ret, $stdout, $stderr) =
+ $node->psql('postgres', 'CREATE EXTENSION resizable_shmem;');
+ ok($ret != 0, 'CREATE EXTENSION fails with resizable shmem on sysv');
+ like($stderr, qr/resizable shared memory requires shared_memory_type = mmap/,
+ 'CREATE EXTENSION error mentions shared_memory_type = mmap requirement');
+
+ ###
+ # Test 4: Verify that resizable structures are also rejected with sysv
+ # shared memory when loaded at startup via shared_preload_libraries.
+ ###
+ $node->safe_psql('postgres', qq{
+ ALTER SYSTEM SET shared_preload_libraries = 'resizable_shmem';
+ ALTER SYSTEM SET resizable_shmem.initial_entries = $startup_initial;
+ ALTER SYSTEM SET resizable_shmem.max_entries = $startup_max;
+ });
+ $node->stop;
+
+ ok(!$node->start(fail_ok => 1),
+ 'server fails to start with resizable shmem on sysv');
+
+ my $log = slurp_file($node->logfile);
+ like($log, qr/resizable shared memory requires shared_memory_type = mmap/,
+ 'log mentions shared_memory_type = mmap requirement');
+}
+
+done_testing();
diff --git a/src/test/modules/test_shmem/t/001_late_shmem_alloc.pl b/src/test/modules/test_shmem/t/001_late_shmem_alloc.pl
index c154f57682a..c89b140871f 100644
--- a/src/test/modules/test_shmem/t/001_late_shmem_alloc.pl
+++ b/src/test/modules/test_shmem/t/001_late_shmem_alloc.pl
@@ -45,5 +45,28 @@ else
ok($attach_count1 == 0 && $attach_count2 == 0, "attach callback is not called when loaded via shared_preload_libraries");
}
+###
+# Test that a fixed-size shared memory structure cannot be resized.
+# Only relevant on platforms that support resizable shmem.
+###
+my $have_resizable_shmem =
+ $node->safe_psql('postgres', 'SHOW have_resizable_shmem;') eq 'on';
+
+if ($have_resizable_shmem)
+{
+ # Try expanding the fixed-size structure
+ my ($ret, $stdout, $stderr) =
+ $node->psql("postgres", "SELECT test_shmem_resize_fixed(1000);");
+ isnt($ret, 0, "expanding a fixed-size structure fails");
+ like($stderr, qr/is not resizable/, "expand error message mentions not resizable");
+
+ # Try shrinking the fixed-size structure
+ ($ret, $stdout, $stderr) =
+ $node->psql("postgres", "SELECT test_shmem_resize_fixed(1);");
+ isnt($ret, 0, "shrinking a fixed-size structure fails");
+ like($stderr, qr/is not resizable/, "shrink error message mentions not resizable");
+}
+
$node->stop;
+
done_testing();
diff --git a/src/test/modules/test_shmem/test_shmem--1.0.sql b/src/test/modules/test_shmem/test_shmem--1.0.sql
index 2d01fd9256c..e169d0d7733 100644
--- a/src/test/modules/test_shmem/test_shmem--1.0.sql
+++ b/src/test/modules/test_shmem/test_shmem--1.0.sql
@@ -7,3 +7,7 @@
CREATE FUNCTION get_test_shmem_attach_count()
RETURNS pg_catalog.int4 STRICT
AS 'MODULE_PATHNAME' LANGUAGE C;
+
+CREATE FUNCTION test_shmem_resize_fixed(pg_catalog.int4)
+RETURNS pg_catalog.void STRICT
+AS 'MODULE_PATHNAME' LANGUAGE C;
diff --git a/src/test/modules/test_shmem/test_shmem.c b/src/test/modules/test_shmem/test_shmem.c
index 9bd4012b435..fc2fd67887f 100644
--- a/src/test/modules/test_shmem/test_shmem.c
+++ b/src/test/modules/test_shmem/test_shmem.c
@@ -99,3 +99,23 @@ get_test_shmem_attach_count(PG_FUNCTION_ARGS)
elog(ERROR, "shmem area not yet initialized");
PG_RETURN_INT32(TestShmem->attach_count);
}
+
+/*
+ * Attempt to resize the fixed-size shared memory structure. This should
+ * fail because the structure was not allocated with a maximum_size.
+ */
+PG_FUNCTION_INFO_V1(test_shmem_resize_fixed);
+Datum
+test_shmem_resize_fixed(PG_FUNCTION_ARGS)
+{
+#ifdef HAVE_RESIZABLE_SHMEM
+ int32 new_size = PG_GETARG_INT32(0);
+
+ ShmemResizeStruct("test_shmem area", new_size);
+ PG_RETURN_VOID();
+#else
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("resizable shared memory is not supported on this platform")));
+#endif
+}
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 81a73c426d2..2bbbf48c96a 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1770,8 +1770,10 @@ pg_shadow| SELECT pg_authid.rolname AS usename,
pg_shmem_allocations| SELECT name,
off,
size,
- allocated_size
- FROM pg_get_shmem_allocations() pg_get_shmem_allocations(name, off, size, allocated_size);
+ allocated_size,
+ maximum_size,
+ reserved_space
+ FROM pg_get_shmem_allocations() pg_get_shmem_allocations(name, off, size, allocated_size, maximum_size, reserved_space);
pg_shmem_allocations_numa| SELECT name,
numa_node,
size
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index e9430e07b36..c079bab4cf0 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3147,6 +3147,7 @@ TestDSMRegistryHashEntry
TestDSMRegistryStruct
TestDecodingData
TestDecodingTxnData
+TestResizableShmemStruct
TestShmemData
TestSpec
TestValueType
base-commit: ed71d7356e3b394f579db87782a41e3d5dfb99ad
--
2.34.1
[text/x-patch] v20260406-0005-Add-minimum_size-specification-for-resizab.patch (19.3K, 3-v20260406-0005-Add-minimum_size-specification-for-resizab.patch)
download | inline diff:
From 6923334bec327bb9cc089d7962bebc73df0afef6 Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat <[email protected]>
Date: Mon, 6 Apr 2026 16:46:33 +0530
Subject: [PATCH v20260406 5/6] Add minimum_size specification for resizable
shared memory structures
Optional minimum size specification for resizable shared memory structures
allows us to enforce that a resizable structure cannot be shrunk below a certain
size. For resizable structures, the minimum size should be less than or equal
to the initial size specified in ShmemRequestStructOpts::size. If not specified,
the minimum size defaults to 0 for resizable structures. For fixed-size
structures, the minimum size and maximum size are set to the initial size
specified in ShmemRequestStructOpts::size.
This makes maximum size and minimum size reported in pg_shmem_allocations view
consistent for both fixed-size and resizable structures.
Author: Ashutosh Bapat <[email protected]>
---
doc/src/sgml/system-views.sgml | 16 ++-
doc/src/sgml/xfunc.sgml | 8 +-
src/backend/storage/ipc/shmem.c | 110 ++++++++++++++----
src/include/catalog/pg_proc.dat | 4 +-
src/include/storage/shmem.h | 6 +
.../modules/resizable_shmem/resizable_shmem.c | 3 +
.../resizable_shmem/t/001_resizable_shmem.pl | 2 +-
.../test_shmem/t/001_late_shmem_alloc.pl | 8 ++
src/test/regress/expected/rules.out | 3 +-
9 files changed, 129 insertions(+), 31 deletions(-)
diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml
index 9717f8434bb..9bbbfdb37c5 100644
--- a/doc/src/sgml/system-views.sgml
+++ b/doc/src/sgml/system-views.sgml
@@ -4250,13 +4250,25 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>minimum_size</structfield> <type>int8</type>
+ </para>
+ <para>
+ Minimum size in bytes that the resizable allocation can shrink to. Equals
+ <structfield>size</structfield>For fixed-size allocations, anonymous
+ allocations, and free memory.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>maximum_size</structfield> <type>int8</type>
</para>
<para>
- Maximum size in bytes that the resizable allocation can grow to. Zero for
- fixed-size allocations, for anonymous allocations, and for free memory.
+ Maximum size in bytes that the resizable allocation can grow to. Equals
+ <structfield>size</structfield> For fixed-size allocations, anonymous
+ allocations, and free memory.
</para></entry>
</row>
diff --git a/doc/src/sgml/xfunc.sgml b/doc/src/sgml/xfunc.sgml
index 3d25139c334..a6c7b8b1b22 100644
--- a/doc/src/sgml/xfunc.sgml
+++ b/doc/src/sgml/xfunc.sgml
@@ -3753,7 +3753,9 @@ my_shmem_init(void *arg)
<parameter>.maximum_size</parameter> along with
<parameter>.size</parameter>. <parameter>.maximum_size</parameter> is
maximum size upto which the structure can grow where as
- <parameter>.size</parameter> is the initial size of the structure. While
+ <parameter>.size</parameter> is the initial size of the structure.
+ Optionally, <parameter>.minimum_size</parameter> can be set to the minimum
+ size that the structure can shrink to. While
contiguous address space worth <parameter>maximum_size</parameter> is
allocated to the structure, only memory worth <parameter>size</parameter>
bytes is allocated initially. The <function>init_fn</function> should only
@@ -3767,8 +3769,8 @@ my_shmem_init(void *arg)
<para>
The structure can be resized using <function>ShmemResizeStruct</function> by
passing it the structure's <structname>ShmemStructDesc</structname> and the
- new size which can be anywhere between 0 to
- <parameter>maximum_size</parameter>. If the new size is smaller than the
+ new size which can be anywhere between <parameter>minimum_size</parameter>
+ and <parameter>maximum_size</parameter>. If the new size is smaller than the
current size of the structure, the memory between the new size and current
size is freed while keeping the contents of the memory upto new size intact.
If the new size is greater than the current size, memory is allocated upto
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 4a3e8a8769e..115c543d36a 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -111,8 +111,11 @@
* startup, but memory is allocated or freed as the structure grows or shrinks
* respectively. ShmemRequestStructOpts::size should be set to the initial size
* of the structure, which is the amount of memory allocated at the startup.
- * After startup, the structure can be resized by calling ShmemResizeStruct() by
- * passing it the ShmemStructDesc for the structure and the new size.
+ * Optionally, ShmemRequestStructOpts::minimum_size can be set to the minimum
+ * size that the structure can shrink to. After startup, the structure can be
+ * resized by calling ShmemResizeStruct() by passing it the ShmemStructDesc for
+ * the structure and the new size. ShmemResizeStruct() enforces that the new
+ * size is within [minimum_size, maximum_size].
*
* While resizable structures can be created after the startup, the memory
* available for them is quite limited.
@@ -294,6 +297,8 @@ typedef struct
void *location; /* location in shared mem */
Size size; /* # bytes requested for the structure */
Size allocated_size; /* # bytes actually allocated */
+ Size minimum_size; /* the minimum size the structure can shrink
+ * to */
Size maximum_size; /* the maximum size the structure can grow to */
Size reserved_space; /* the total address space reserved */
} ShmemIndexEnt;
@@ -383,6 +388,9 @@ ShmemRequestInternal(ShmemStructOpts *options, ShmemRequestKind kind)
if (options->size <= 0 && options->size != SHMEM_ATTACH_UNKNOWN_SIZE)
elog(ERROR, "invalid size %zd for shared memory request for \"%s\"",
options->size, options->name);
+ if (options->minimum_size < 0 && options->minimum_size != SHMEM_ATTACH_UNKNOWN_SIZE)
+ elog(ERROR, "invalid minimum_size %zd for shared memory request for \"%s\"",
+ options->minimum_size, options->name);
if (options->maximum_size < 0 && options->maximum_size != SHMEM_ATTACH_UNKNOWN_SIZE)
elog(ERROR, "invalid maximum_size %zd for shared memory request for \"%s\"",
options->maximum_size, options->name);
@@ -394,6 +402,11 @@ ShmemRequestInternal(ShmemStructOpts *options, ShmemRequestKind kind)
if (options->size <= 0)
elog(ERROR, "invalid size %zd for shared memory request for \"%s\"",
options->size, options->name);
+ if (options->minimum_size == SHMEM_ATTACH_UNKNOWN_SIZE)
+ elog(ERROR, "SHMEM_ATTACH_UNKNOWN_SIZE cannot be used during startup");
+ if (options->minimum_size < 0)
+ elog(ERROR, "invalid minimum_size %zd for shared memory request for \"%s\"",
+ options->minimum_size, options->name);
if (options->maximum_size == SHMEM_ATTACH_UNKNOWN_SIZE)
elog(ERROR, "SHMEM_ATTACH_UNKNOWN_SIZE cannot be used during startup");
if (options->maximum_size < 0)
@@ -405,10 +418,20 @@ ShmemRequestInternal(ShmemStructOpts *options, ShmemRequestKind kind)
elog(ERROR, "invalid alignment %zu for shared memory request for \"%s\"",
options->alignment, options->name);
+ if (options->minimum_size > 0 && options->size != SHMEM_ATTACH_UNKNOWN_SIZE &&
+ options->minimum_size > options->size)
+ elog(ERROR, "resizable shared memory structure \"%s\" should have minimum size (%zd) less than or equal to size (%zd)",
+ options->name, options->minimum_size, options->size);
+
if (options->maximum_size > 0 && options->size > options->maximum_size)
elog(ERROR, "resizable shared memory structure \"%s\" should have maximum size (%zd) greater than size (%zd)",
options->name, options->maximum_size, options->size);
+ if (options->minimum_size > 0 && options->maximum_size > 0 &&
+ options->minimum_size > options->maximum_size)
+ elog(ERROR, "resizable shared memory structure \"%s\" should have minimum size (%zd) less than or equal to maximum size (%zd)",
+ options->name, options->minimum_size, options->maximum_size);
+
/* Check that we're in the right state */
if (shmem_request_state != SRS_REQUESTING)
elog(ERROR, "ShmemRequestStruct can only be called from a shmem_request callback");
@@ -614,12 +637,19 @@ InitShmemIndexEntry(ShmemRequest *request)
index_entry->allocated_size = allocated_size;
index_entry->location = structPtr;
index_entry->reserved_space = allocated_size;
- index_entry->maximum_size = request->options->maximum_size;
if (request->options->maximum_size > 0)
{
+ index_entry->minimum_size = request->options->minimum_size;
+ index_entry->maximum_size = request->options->maximum_size;
+
/* Adjust allocated size of a resizable structure. */
index_entry->allocated_size = EstimateAllocatedSize(index_entry);
}
+ else
+ {
+ index_entry->minimum_size = request->options->size;
+ index_entry->maximum_size = request->options->size;
+ }
/* Initialize depending on the kind of shmem area it is */
switch (request->kind)
@@ -674,14 +704,38 @@ AttachShmemIndexEntry(ShmemRequest *request, bool missing_ok)
name, index_entry->size, request->options->size)));
}
- if (index_entry->maximum_size != request->options->maximum_size &&
- request->options->maximum_size != SHMEM_ATTACH_UNKNOWN_SIZE)
+ /*
+ * For resizable structures, also check that minimum_size and maximum_size
+ * match. For fixed-size structures, these are derived (set to size) in
+ * the index entry and not meaningful in the request.
+ */
+ if (request->options->maximum_size != 0)
{
- ereport(ERROR,
- (errmsg("shared memory struct \"%s\" was created with" \
- " different maximum_size: existing %zu, requested %zu",
- name, index_entry->maximum_size,
- request->options->maximum_size)));
+ if (index_entry->minimum_size != request->options->minimum_size &&
+ request->options->minimum_size != SHMEM_ATTACH_UNKNOWN_SIZE)
+ {
+ ereport(ERROR,
+ (errmsg("shared memory struct \"%s\" was created with"
+ " different minimum_size: existing %zu, requested %zu",
+ name, index_entry->minimum_size,
+ request->options->minimum_size)));
+ }
+
+ if (index_entry->maximum_size != request->options->maximum_size &&
+ request->options->maximum_size != SHMEM_ATTACH_UNKNOWN_SIZE)
+ {
+ ereport(ERROR,
+ (errmsg("shared memory struct \"%s\" was created with"
+ " different maximum_size: existing %zu, requested %zu",
+ name, index_entry->maximum_size,
+ request->options->maximum_size)));
+ }
+ }
+ else
+ {
+ if (index_entry->minimum_size != index_entry->maximum_size)
+ elog(ERROR, "shared memory struct \"%s\" was created as resizable, but requested as fixed-size",
+ name);
}
/*
@@ -740,10 +794,11 @@ EstimateAllocatedSize(ShmemIndexEnt *entry)
/*
* ShmemResizeStruct() --- resize a resizable shared memory structure.
*
- * If the structure is being shrunk, the memory pages that are no longer needed
- * are freed. If the structure is being expanded, the memory pages that are
- * needed for the new size are allocated. See EstimateAllocatedSize() for
- * explanation of which pages are allocated for a resizable structure.
+ * The new size must be within [minimum_size, maximum_size]. If the structure
+ * is being shrunk, the memory pages that are no longer needed are freed. If
+ * the structure is being expanded, the memory pages that are needed for the
+ * new size are allocated. See EstimateAllocatedSize() for explanation of which
+ * pages are allocated for a resizable structure.
*/
void
ShmemResizeStruct(const char *name, Size new_size)
@@ -776,15 +831,22 @@ ShmemResizeStruct(const char *name, Size new_size)
Assert(result);
- if (result->maximum_size <= 0)
+ if (result->minimum_size == result->maximum_size)
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("shared memory struct \"%s\" is not resizable", name)));
+ if (new_size < result->minimum_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("cannot shrink shared memory structure \"%s\" below minimum size"
+ " (requested %zu bytes, minimum %zu bytes)",
+ name, new_size, result->minimum_size)));
+
if (result->maximum_size < new_size)
ereport(ERROR,
(errcode(ERRCODE_INSUFFICIENT_RESOURCES),
- errmsg("not enough address space is reserved for resizing structure \"%s\"" \
+ errmsg("not enough address space is reserved for resizing structure \"%s\""
"(required %zu bytes, reserved %zu bytes)",
name, new_size, result->maximum_size)));
@@ -925,7 +987,8 @@ InitShmemAllocator(PGShmemHeader *seghdr)
result->size = ShmemAllocator->index_size;
result->allocated_size = ShmemAllocator->index_size;
#ifdef HAVE_RESIZABLE_SHMEM
- result->maximum_size = 0;
+ result->minimum_size = result->size;
+ result->maximum_size = result->size;
result->reserved_space = result->allocated_size;
#endif
result->location = ShmemAllocator->index;
@@ -1271,7 +1334,7 @@ mul_size(Size s1, Size s2)
Datum
pg_get_shmem_allocations(PG_FUNCTION_ARGS)
{
-#define PG_GET_SHMEM_SIZES_COLS 6
+#define PG_GET_SHMEM_SIZES_COLS 7
ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
HASH_SEQ_STATUS hstat;
ShmemIndexEnt *ent;
@@ -1293,8 +1356,9 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS)
values[1] = Int64GetDatum((char *) ent->location - (char *) ShmemSegHdr);
values[2] = Int64GetDatum(ent->size);
values[3] = Int64GetDatum(ent->allocated_size);
- values[4] = Int64GetDatum(ent->maximum_size);
- values[5] = Int64GetDatum(ent->reserved_space);
+ values[4] = Int64GetDatum(ent->minimum_size);
+ values[5] = Int64GetDatum(ent->maximum_size);
+ values[6] = Int64GetDatum(ent->reserved_space);
/*
* Keep track of the total reserved space for named shmem areas, to be
@@ -1313,8 +1377,9 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS)
nulls[1] = true;
values[2] = Int64GetDatum(ShmemAllocator->free_offset - named_allocated);
values[3] = values[2];
- values[4] = Int64GetDatum(0);
+ values[4] = values[2];
values[5] = values[2];
+ values[6] = values[2];
tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
/* output as-of-yet unused shared memory */
@@ -1323,8 +1388,9 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS)
nulls[1] = false;
values[2] = Int64GetDatum(ShmemSegHdr->totalsize - ShmemAllocator->free_offset);
values[3] = values[2];
- values[4] = Int64GetDatum(0);
+ values[4] = values[2];
values[5] = values[2];
+ values[6] = values[2];
tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
LWLockRelease(ShmemIndexLock);
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 32945d73f36..5ff6c543305 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8702,8 +8702,8 @@
{ oid => '5052', descr => 'allocations from the main shared memory segment',
proname => 'pg_get_shmem_allocations', prorows => '50', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,int8,int8,int8,int8,int8}', proargmodes => '{o,o,o,o,o,o}',
- proargnames => '{name,off,size,allocated_size,maximum_size,reserved_space}',
+ proallargtypes => '{text,int8,int8,int8,int8,int8,int8}', proargmodes => '{o,o,o,o,o,o,o}',
+ proargnames => '{name,off,size,allocated_size,minimum_size,maximum_size,reserved_space}',
prosrc => 'pg_get_shmem_allocations',
proacl => '{POSTGRES=X,pg_read_all_stats=X}' },
diff --git a/src/include/storage/shmem.h b/src/include/storage/shmem.h
index 8140a0255ae..0e6d5a63f28 100644
--- a/src/include/storage/shmem.h
+++ b/src/include/storage/shmem.h
@@ -57,6 +57,12 @@ typedef struct ShmemStructOpts
*/
size_t alignment;
+ /*
+ * Minimum size this structure can shrink to. Should be set to 0 for
+ * fixed-size structures.
+ */
+ ssize_t minimum_size;
+
/*
* Maximum size this structure can grow upto in future. The memory is not
* allocated right away but the corresponding address space is reserved so
diff --git a/src/test/modules/resizable_shmem/resizable_shmem.c b/src/test/modules/resizable_shmem/resizable_shmem.c
index fb3dfd64b4b..d035d767a62 100644
--- a/src/test/modules/resizable_shmem/resizable_shmem.c
+++ b/src/test/modules/resizable_shmem/resizable_shmem.c
@@ -144,14 +144,17 @@ resizable_shmem_request(void *arg)
#ifdef HAVE_RESIZABLE_SHMEM
Size max_size = add_size(offsetof(TestResizableShmemStruct, data),
mul_size(test_max_entries, TEST_ENTRY_SIZE));
+ Size min_size = offsetof(TestResizableShmemStruct, data);
#else
Size max_size = 0;
+ Size min_size = 0;
#endif
/* Register our resizable shared memory structure */
ShmemRequestStruct(.name = "resizable_shmem",
.size = use_unknown_size ? SHMEM_ATTACH_UNKNOWN_SIZE : initial_size,
+ .minimum_size = min_size,
.maximum_size = max_size,
.ptr = (void **) &resizable_shmem,
);
diff --git a/src/test/modules/resizable_shmem/t/001_resizable_shmem.pl b/src/test/modules/resizable_shmem/t/001_resizable_shmem.pl
index a172cd0fd19..6a00ae0a194 100644
--- a/src/test/modules/resizable_shmem/t/001_resizable_shmem.pl
+++ b/src/test/modules/resizable_shmem/t/001_resizable_shmem.pl
@@ -96,7 +96,7 @@ sub run_resizable_tests
$session2->quit;
# Verify no other structure is resizable
- is($node->safe_psql('postgres', "SELECT count(*) FROM pg_shmem_allocations WHERE name <> 'resizable_shmem' AND maximum_size <> 0;"),
+ is($node->safe_psql('postgres', "SELECT count(*) FROM pg_shmem_allocations WHERE name <> 'resizable_shmem' AND maximum_size <> minimum_size;"),
'0', "$prefix: no other resizable structures");
# Resize to maximum
diff --git a/src/test/modules/test_shmem/t/001_late_shmem_alloc.pl b/src/test/modules/test_shmem/t/001_late_shmem_alloc.pl
index c89b140871f..472d4b121ae 100644
--- a/src/test/modules/test_shmem/t/001_late_shmem_alloc.pl
+++ b/src/test/modules/test_shmem/t/001_late_shmem_alloc.pl
@@ -67,6 +67,14 @@ if ($have_resizable_shmem)
like($stderr, qr/is not resizable/, "shrink error message mentions not resizable");
}
+###
+# Test that minimum_size and maximum_size equal size for a fixed-size structure
+# in pg_shmem_allocations.
+###
+is($node->safe_psql('postgres',
+ "SELECT minimum_size = size AND maximum_size = size FROM pg_shmem_allocations WHERE name = 'test_shmem area';"),
+ 't', "fixed-size structure has minimum_size = maximum_size = size");
+
$node->stop;
done_testing();
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 2bbbf48c96a..c42eb9c67a1 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1771,9 +1771,10 @@ pg_shmem_allocations| SELECT name,
off,
size,
allocated_size,
+ minimum_size,
maximum_size,
reserved_space
- FROM pg_get_shmem_allocations() pg_get_shmem_allocations(name, off, size, allocated_size, maximum_size, reserved_space);
+ FROM pg_get_shmem_allocations() pg_get_shmem_allocations(name, off, size, allocated_size, minimum_size, maximum_size, reserved_space);
pg_shmem_allocations_numa| SELECT name,
numa_node,
size
--
2.34.1
[text/x-patch] v20260406-0004-Avoid-creating-ABI-incompatibility-because.patch (14.4K, 4-v20260406-0004-Avoid-creating-ABI-incompatibility-because.patch)
download | inline diff:
From 45696d8f2371cceca1e38719ef49036cdd67c250 Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat <[email protected]>
Date: Mon, 6 Apr 2026 13:00:00 +0530
Subject: [PATCH v20260406 4/6] Avoid creating ABI incompatibility because of
HAVE_RESIZABLE_SHMEM
Per the suggestion at https://www.postgresql.org/message-id/CAEze2Wjn2cpQEPwzLajc0XdcMy8T=d1AWjg3UAmMUT2TmHkQkA@mail.gmail.com
---
src/backend/port/sysv_shmem.c | 14 +++++-
src/backend/port/win32_shmem.c | 22 +++++++++
src/backend/storage/ipc/shmem.c | 48 +++++++------------
src/include/storage/pg_shmem.h | 2 -
src/include/storage/shmem.h | 5 --
.../modules/resizable_shmem/resizable_shmem.c | 22 +++++----
src/test/modules/test_shmem/test_shmem.c | 6 ---
7 files changed, 63 insertions(+), 56 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 8d859dfbbfb..bb2a81417c6 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -1013,7 +1013,6 @@ PGSharedMemoryDetach(void)
}
}
-#ifdef HAVE_RESIZABLE_SHMEM
/*
* Make sure that the memory of given size from the given address is released.
*
@@ -1024,6 +1023,11 @@ PGSharedMemoryDetach(void)
void
PGSharedMemoryEnsureFreed(void *addr, Size size)
{
+#ifndef HAVE_RESIZABLE_SHMEM
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("resizable shared memory is not supported on this platform")));
+#else
if (!AnonymousShmem)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
@@ -1035,6 +1039,7 @@ PGSharedMemoryEnsureFreed(void *addr, Size size)
if (madvise(addr, size, MADV_REMOVE) == -1)
ereport(ERROR,
(errmsg("could not free shared memory: %m")));
+#endif
}
/*
@@ -1047,6 +1052,11 @@ PGSharedMemoryEnsureFreed(void *addr, Size size)
void
PGSharedMemoryEnsureAllocated(void *addr, Size size)
{
+#ifndef HAVE_RESIZABLE_SHMEM
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("resizable shared memory is not supported on this platform")));
+#else
if (!AnonymousShmem)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
@@ -1058,5 +1068,5 @@ PGSharedMemoryEnsureAllocated(void *addr, Size size)
if (madvise(addr, size, MADV_POPULATE_WRITE) == -1)
ereport(ERROR,
(errmsg("could not allocate shared memory: %m")));
+#endif
}
-#endif /* HAVE_RESIZABLE_SHMEM */
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index dc2ee018845..c1f30665e66 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -671,3 +671,25 @@ GetOSPageSize(void)
return os_page_size;
}
+
+/*
+ * PGSharedMemoryEnsureFreed / PGSharedMemoryEnsureAllocated
+ *
+ * Not supported on Windows. These are only meaningful on platforms with
+ * resizable shared memory (mmap + madvise).
+ */
+void
+PGSharedMemoryEnsureFreed(void *addr, Size size)
+{
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("resizable shared memory is not supported on this platform")));
+}
+
+void
+PGSharedMemoryEnsureAllocated(void *addr, Size size)
+{
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("resizable shared memory is not supported on this platform")));
+}
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 03de5d88d51..4a3e8a8769e 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -185,14 +185,12 @@ typedef struct
/*
* A convenient macro to get the space required for a shmem request consistently.
* A resizable structure, requested by non-zero maximum_size, requires space for
- * its maximum size.
+ * its maximum size. Please note that on the platforms that do not support
+ * resizable shmem, the maximum_size is ensured to be 0 i.e. all the structures
+ * are treated as fixed-size structures.
*/
-#ifdef HAVE_RESIZABLE_SHMEM
#define SHMEM_REQUEST_SPACE_SIZE(request) \
((request)->options->maximum_size > 0 ? (request)->options->maximum_size : (request)->options->size)
-#else
-#define SHMEM_REQUEST_SPACE_SIZE(request) ((request)->options->size)
-#endif
static List *pending_shmem_requests;
@@ -296,10 +294,8 @@ typedef struct
void *location; /* location in shared mem */
Size size; /* # bytes requested for the structure */
Size allocated_size; /* # bytes actually allocated */
-#ifdef HAVE_RESIZABLE_SHMEM
Size maximum_size; /* the maximum size the structure can grow to */
Size reserved_space; /* the total address space reserved */
-#endif
} ShmemIndexEnt;
/* To get reliable results for NUMA inquiry we need to "touch pages" once */
@@ -308,9 +304,7 @@ static bool firstNumaTouch = true;
static void CallShmemCallbacksAfterStartup(const ShmemCallbacks *callbacks);
static void InitShmemIndexEntry(ShmemRequest *request);
static bool AttachShmemIndexEntry(ShmemRequest *request, bool missing_ok);
-#ifdef HAVE_RESIZABLE_SHMEM
static Size EstimateAllocatedSize(ShmemIndexEnt *entry);
-#endif
Datum pg_numa_available(PG_FUNCTION_ARGS);
@@ -376,16 +370,22 @@ ShmemRequestInternal(ShmemStructOpts *options, ShmemRequestKind kind)
if (options->name == NULL)
elog(ERROR, "shared memory request is missing 'name' option");
+#ifndef HAVE_RESIZABLE_SHMEM
+ if (options->maximum_size > 0)
+ elog(ERROR, "resizable shared memory is not supported on this platform");
+#else
+ if (options->maximum_size > 0 && shared_memory_type != SHMEM_TYPE_MMAP)
+ elog(ERROR, "resizable shared memory requires shared_memory_type = mmap");
+#endif
+
if (IsUnderPostmaster)
{
if (options->size <= 0 && options->size != SHMEM_ATTACH_UNKNOWN_SIZE)
elog(ERROR, "invalid size %zd for shared memory request for \"%s\"",
options->size, options->name);
-#ifdef HAVE_RESIZABLE_SHMEM
if (options->maximum_size < 0 && options->maximum_size != SHMEM_ATTACH_UNKNOWN_SIZE)
elog(ERROR, "invalid maximum_size %zd for shared memory request for \"%s\"",
options->maximum_size, options->name);
-#endif
}
else
{
@@ -394,28 +394,21 @@ ShmemRequestInternal(ShmemStructOpts *options, ShmemRequestKind kind)
if (options->size <= 0)
elog(ERROR, "invalid size %zd for shared memory request for \"%s\"",
options->size, options->name);
-#ifdef HAVE_RESIZABLE_SHMEM
if (options->maximum_size == SHMEM_ATTACH_UNKNOWN_SIZE)
elog(ERROR, "SHMEM_ATTACH_UNKNOWN_SIZE cannot be used during startup");
if (options->maximum_size < 0)
elog(ERROR, "invalid maximum_size %zd for shared memory request for \"%s\"",
options->maximum_size, options->name);
-#endif
}
if (options->alignment != 0 && pg_nextpower2_size_t(options->alignment) != options->alignment)
elog(ERROR, "invalid alignment %zu for shared memory request for \"%s\"",
options->alignment, options->name);
-#ifdef HAVE_RESIZABLE_SHMEM
if (options->maximum_size > 0 && options->size > options->maximum_size)
elog(ERROR, "resizable shared memory structure \"%s\" should have maximum size (%zd) greater than size (%zd)",
options->name, options->maximum_size, options->size);
- if (options->maximum_size > 0 && shared_memory_type != SHMEM_TYPE_MMAP)
- elog(ERROR, "resizable shared memory requires shared_memory_type = mmap");
-#endif
-
/* Check that we're in the right state */
if (shmem_request_state != SRS_REQUESTING)
elog(ERROR, "ShmemRequestStruct can only be called from a shmem_request callback");
@@ -620,7 +613,6 @@ InitShmemIndexEntry(ShmemRequest *request)
index_entry->size = request->options->size;
index_entry->allocated_size = allocated_size;
index_entry->location = structPtr;
-#ifdef HAVE_RESIZABLE_SHMEM
index_entry->reserved_space = allocated_size;
index_entry->maximum_size = request->options->maximum_size;
if (request->options->maximum_size > 0)
@@ -628,7 +620,6 @@ InitShmemIndexEntry(ShmemRequest *request)
/* Adjust allocated size of a resizable structure. */
index_entry->allocated_size = EstimateAllocatedSize(index_entry);
}
-#endif
/* Initialize depending on the kind of shmem area it is */
switch (request->kind)
@@ -683,7 +674,6 @@ AttachShmemIndexEntry(ShmemRequest *request, bool missing_ok)
name, index_entry->size, request->options->size)));
}
-#ifdef HAVE_RESIZABLE_SHMEM
if (index_entry->maximum_size != request->options->maximum_size &&
request->options->maximum_size != SHMEM_ATTACH_UNKNOWN_SIZE)
{
@@ -693,7 +683,6 @@ AttachShmemIndexEntry(ShmemRequest *request, bool missing_ok)
name, index_entry->maximum_size,
request->options->maximum_size)));
}
-#endif
/*
* Re-establish the caller's pointer variable, or do other actions to
@@ -716,7 +705,6 @@ AttachShmemIndexEntry(ShmemRequest *request, bool missing_ok)
return true;
}
-#ifdef HAVE_RESIZABLE_SHMEM
/*
* Estimate the actual memory allocated for a resizable structure.
*
@@ -760,6 +748,11 @@ EstimateAllocatedSize(ShmemIndexEnt *entry)
void
ShmemResizeStruct(const char *name, Size new_size)
{
+#ifndef HAVE_RESIZABLE_SHMEM
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("resizable shared memory is not supported on this platform")));
+#else
ShmemIndexEnt *result;
bool found;
Size page_size = GetOSPageSize();
@@ -822,8 +815,8 @@ ShmemResizeStruct(const char *name, Size new_size)
result->allocated_size = EstimateAllocatedSize(result);
LWLockRelease(ShmemIndexLock);
+#endif
}
-#endif /* HAVE_RESIZABLE_SHMEM */
/*
* InitShmemAllocator() --- set up basic pointers to shared memory.
@@ -1300,7 +1293,6 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS)
values[1] = Int64GetDatum((char *) ent->location - (char *) ShmemSegHdr);
values[2] = Int64GetDatum(ent->size);
values[3] = Int64GetDatum(ent->allocated_size);
-#ifdef HAVE_RESIZABLE_SHMEM
values[4] = Int64GetDatum(ent->maximum_size);
values[5] = Int64GetDatum(ent->reserved_space);
@@ -1311,12 +1303,6 @@ pg_get_shmem_allocations(PG_FUNCTION_ARGS)
* the segment.
*/
named_allocated += ent->reserved_space;
-#else
- values[4] = Int64GetDatum(0);
- values[5] = Int64GetDatum(ent->allocated_size);
-
- named_allocated += ent->allocated_size;
-#endif
tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc,
values, nulls);
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 3d5aceba59c..f0efbf2aec1 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -89,10 +89,8 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-#ifdef HAVE_RESIZABLE_SHMEM
extern void PGSharedMemoryEnsureFreed(void *addr, Size size);
extern void PGSharedMemoryEnsureAllocated(void *addr, Size size);
-#endif
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
extern Size GetOSPageSize(void);
diff --git a/src/include/storage/shmem.h b/src/include/storage/shmem.h
index f356027e500..8140a0255ae 100644
--- a/src/include/storage/shmem.h
+++ b/src/include/storage/shmem.h
@@ -57,8 +57,6 @@ typedef struct ShmemStructOpts
*/
size_t alignment;
-#ifdef HAVE_RESIZABLE_SHMEM
-
/*
* Maximum size this structure can grow upto in future. The memory is not
* allocated right away but the corresponding address space is reserved so
@@ -68,7 +66,6 @@ typedef struct ShmemStructOpts
* structures.
*/
ssize_t maximum_size;
-#endif
/*
* When the shmem area is initialized or attached to, pointer to it is
@@ -181,9 +178,7 @@ typedef struct ShmemCallbacks
extern void RegisterShmemCallbacks(const ShmemCallbacks *callbacks);
extern bool ShmemAddrIsValid(const void *addr);
-#ifdef HAVE_RESIZABLE_SHMEM
extern void ShmemResizeStruct(const char *name, Size new_size);
-#endif
/*
* These macros provide syntactic sugar for calling the underlying functions
diff --git a/src/test/modules/resizable_shmem/resizable_shmem.c b/src/test/modules/resizable_shmem/resizable_shmem.c
index 66754582b32..fb3dfd64b4b 100644
--- a/src/test/modules/resizable_shmem/resizable_shmem.c
+++ b/src/test/modules/resizable_shmem/resizable_shmem.c
@@ -135,20 +135,24 @@ resizable_shmem_request(void *arg)
{
Size initial_size = add_size(offsetof(TestResizableShmemStruct, data),
mul_size(test_initial_entries, TEST_ENTRY_SIZE));
+
+/*
+ * Create resizable structure on the platforms which support it. Otherwise create
+ * as a fixed-size structure. Other way would be to conditionally include
+ * .maximum_size in the call to ShmemRequestStruct().
+ */
#ifdef HAVE_RESIZABLE_SHMEM
Size max_size = add_size(offsetof(TestResizableShmemStruct, data),
mul_size(test_max_entries, TEST_ENTRY_SIZE));
- /* A preprocessor macro to conditionally include the maximum_size field. */
-#define MAXIMUM_SIZE_ARG .maximum_size = max_size,
#else
-#define MAXIMUM_SIZE_ARG
+ Size max_size = 0;
#endif
/* Register our resizable shared memory structure */
ShmemRequestStruct(.name = "resizable_shmem",
.size = use_unknown_size ? SHMEM_ATTACH_UNKNOWN_SIZE : initial_size,
- MAXIMUM_SIZE_ARG
+ .maximum_size = max_size,
.ptr = (void **) &resizable_shmem,
);
}
@@ -172,11 +176,14 @@ resizable_shmem_shmem_init(void *arg)
/*
* Resize the shared memory structure to accommodate the specified number of
* entries.
+ *
+ * On the plaforms which do not support resizable shared memory,
+ * ShmemResizeStruct() will raise an error, so this function will fail if the
+ * caller tries to resize the structure.
*/
Datum
resizable_shmem_resize(PG_FUNCTION_ARGS)
{
-#ifdef HAVE_RESIZABLE_SHMEM
int32 new_entries = PG_GETARG_INT32(0);
Size new_size;
@@ -191,11 +198,6 @@ resizable_shmem_resize(PG_FUNCTION_ARGS)
resizable_shmem->num_entries = new_entries;
PG_RETURN_VOID();
-#else
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("resizable shared memory is not supported on this platform")));
-#endif
}
/*
diff --git a/src/test/modules/test_shmem/test_shmem.c b/src/test/modules/test_shmem/test_shmem.c
index fc2fd67887f..0dd469891ee 100644
--- a/src/test/modules/test_shmem/test_shmem.c
+++ b/src/test/modules/test_shmem/test_shmem.c
@@ -108,14 +108,8 @@ PG_FUNCTION_INFO_V1(test_shmem_resize_fixed);
Datum
test_shmem_resize_fixed(PG_FUNCTION_ARGS)
{
-#ifdef HAVE_RESIZABLE_SHMEM
int32 new_size = PG_GETARG_INT32(0);
ShmemResizeStruct("test_shmem area", new_size);
PG_RETURN_VOID();
-#else
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("resizable shared memory is not supported on this platform")));
-#endif
}
--
2.34.1
[text/x-patch] v20260406-0003-Add-more-diagnostics-about-shared-memory-s.patch (4.3K, 5-v20260406-0003-Add-more-diagnostics-about-shared-memory-s.patch)
download | inline diff:
From 1d1d85f1c36bb86b4648c5c5c3afb4b41b0f7c2a Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat <[email protected]>
Date: Mon, 6 Apr 2026 10:58:29 +0530
Subject: [PATCH v20260406 3/6] Add more diagnostics about shared memory
segments
NOT FOR FINAL COMMIT
Log size, RSS and Swap of every shared memory segment mapped by the backend.
This will be useful to understand the failure on CFBot where we are seeing about
10MB extra shared memory allocated that expected.
---
.../modules/resizable_shmem/resizable_shmem.c | 93 +++++++++++++++++--
1 file changed, 84 insertions(+), 9 deletions(-)
diff --git a/src/test/modules/resizable_shmem/resizable_shmem.c b/src/test/modules/resizable_shmem/resizable_shmem.c
index 2063d05053f..66754582b32 100644
--- a/src/test/modules/resizable_shmem/resizable_shmem.c
+++ b/src/test/modules/resizable_shmem/resizable_shmem.c
@@ -254,8 +254,11 @@ resizable_shmem_read(PG_FUNCTION_ARGS)
* backend.
*
* The VMA containing our resizable_shmem pointer is used to determine the main
- * memory segment. RSS + Swap (in bytes) for that VMS from /proc/self/smaps is
+ * memory segment. RSS + Swap (in bytes) for that VMA from /proc/self/smaps is
* returned.
+ *
+ * As a side effect, all shared-memory VMAs are logged with their name, RSS,
+ * and Swap values for diagnostic purposes.
*/
Datum
resizable_shmem_usage(PG_FUNCTION_ARGS)
@@ -268,6 +271,14 @@ resizable_shmem_usage(PG_FUNCTION_ARGS)
bool in_target_vma = false;
size_t result;
+ /* State for logging shared VMAs */
+ bool in_shared_vma = false;
+ char vma_name[256];
+ char vma_range[64];
+ int64 vma_size_kb = -1;
+ int64 vma_rss_kb = -1;
+ int64 vma_swap_kb = -1;
+
f = fopen("/proc/self/smaps", "r");
if (f == NULL)
ereport(ERROR,
@@ -278,22 +289,86 @@ resizable_shmem_usage(PG_FUNCTION_ARGS)
{
unsigned long start;
unsigned long end;
+ char perms[5];
+ unsigned long offset;
+ char dev[12];
+ unsigned long inode;
+ char pathname[256];
+ int nfields;
- if (sscanf(line, "%lx-%lx", &start, &end) == 2)
+ nfields = sscanf(line, "%lx-%lx %4s %lx %11s %lu %255[^\n]",
+ &start, &end, perms, &offset, dev, &inode, pathname);
+
+ if (nfields >= 6)
{
+ /*
+ * We've hit a new VMA header. First, log the previous shared VMA
+ * if we were tracking one.
+ */
+ if (in_shared_vma)
+ elog(LOG, "shared VMA %s %s: Size=%ld kB, Rss=%ld kB, Swap=%ld kB",
+ vma_range, vma_name,
+ (long) (vma_size_kb >= 0 ? vma_size_kb : 0),
+ (long) (vma_rss_kb >= 0 ? vma_rss_kb : 0),
+ (long) (vma_swap_kb >= 0 ? vma_swap_kb : 0));
+
+ /* Check if this VMA is a shared mapping (has 's' in perms) */
+ in_shared_vma = (perms[3] == 's');
+ if (in_shared_vma)
+ {
+ snprintf(vma_range, sizeof(vma_range), "%lx-%lx", start, end);
+ if (nfields >= 7)
+ strlcpy(vma_name, pathname, sizeof(vma_name));
+ else
+ strlcpy(vma_name, "(anonymous)", sizeof(vma_name));
+ vma_size_kb = -1;
+ vma_rss_kb = -1;
+ vma_swap_kb = -1;
+ }
+
+ /* Track the target VMA for our return value */
in_target_vma = (target >= start && target < end);
+ if (in_target_vma)
+ {
+ rss_kb = -1;
+ swap_kb = -1;
+ }
}
- else if (in_target_vma)
+ else
{
- if (rss_kb == -1)
- sscanf(line, "Rss: %ld kB", &rss_kb);
- if (swap_kb == -1)
- sscanf(line, "Swap: %ld kB", &swap_kb);
- if (rss_kb >= 0 && swap_kb >= 0)
- break;
+ /* Parse detail lines for the current VMA */
+ int64 val;
+
+ if (sscanf(line, "Size: %ld kB", &val) == 1)
+ {
+ if (in_shared_vma && vma_size_kb == -1)
+ vma_size_kb = val;
+ }
+ else if (sscanf(line, "Rss: %ld kB", &val) == 1)
+ {
+ if (in_target_vma && rss_kb == -1)
+ rss_kb = val;
+ if (in_shared_vma && vma_rss_kb == -1)
+ vma_rss_kb = val;
+ }
+ else if (sscanf(line, "Swap: %ld kB", &val) == 1)
+ {
+ if (in_target_vma && swap_kb == -1)
+ swap_kb = val;
+ if (in_shared_vma && vma_swap_kb == -1)
+ vma_swap_kb = val;
+ }
}
}
+ /* Log the last shared VMA if any */
+ if (in_shared_vma)
+ elog(LOG, "shared VMA %s %s: Size=%ld kB, Rss=%ld kB, Swap=%ld kB",
+ vma_range, vma_name,
+ (long) (vma_size_kb >= 0 ? vma_size_kb : 0),
+ (long) (vma_rss_kb >= 0 ? vma_rss_kb : 0),
+ (long) (vma_swap_kb >= 0 ? vma_swap_kb : 0));
+
fclose(f);
result = rss_kb >= 0 ? mul_size(rss_kb, 1024) : 0;
--
2.34.1
[text/x-patch] v20260406-0002-Use-smaps-instead-of-status-in-resizable_s.patch (6.1K, 6-v20260406-0002-Use-smaps-instead-of-status-in-resizable_s.patch)
download | inline diff:
From a982fd9491c914c67d674a21e4ba0ac746807811 Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat <[email protected]>
Date: Mon, 6 Apr 2026 10:43:18 +0530
Subject: [PATCH v20260406 2/6] Use smaps instead of status in
resizable_shmem_used()
/proc/self/status gives memory usages across all the VMAs of a process.
/proc/self/smaps gives memory usages for each VMA separately. Hence use smaps to
accurately estimate the memory allocated in the main shared memory segment.
---
.../resizable_shmem/resizable_shmem--1.0.sql | 6 +-
.../modules/resizable_shmem/resizable_shmem.c | 86 ++++++++-----------
.../resizable_shmem/t/001_resizable_shmem.pl | 2 +-
3 files changed, 41 insertions(+), 53 deletions(-)
diff --git a/src/test/modules/resizable_shmem/resizable_shmem--1.0.sql b/src/test/modules/resizable_shmem/resizable_shmem--1.0.sql
index c1bcb6117b6..b4b07336dc3 100644
--- a/src/test/modules/resizable_shmem/resizable_shmem--1.0.sql
+++ b/src/test/modules/resizable_shmem/resizable_shmem--1.0.sql
@@ -25,8 +25,10 @@ RETURNS boolean
AS 'MODULE_PATHNAME'
LANGUAGE C STRICT;
--- Function to report memory usage statistics of the calling backend
-CREATE FUNCTION resizable_shmem_usage(OUT rss_anon bigint, OUT rss_file bigint, OUT rss_shmem bigint, OUT vm_size bigint)
+-- Function to report memory mapped against the main shared memory segment in
+-- the backend where this function runs.
+CREATE FUNCTION resizable_shmem_usage()
+RETURNS bigint
AS 'MODULE_PATHNAME'
LANGUAGE C STRICT;
diff --git a/src/test/modules/resizable_shmem/resizable_shmem.c b/src/test/modules/resizable_shmem/resizable_shmem.c
index 5ae2d2e2d1d..2063d05053f 100644
--- a/src/test/modules/resizable_shmem/resizable_shmem.c
+++ b/src/test/modules/resizable_shmem/resizable_shmem.c
@@ -10,19 +10,17 @@
*/
#include "postgres.h"
+#include <limits.h>
+#include <stdio.h>
+
#include "commands/extension.h"
#include "fmgr.h"
-#include "funcapi.h"
#include "miscadmin.h"
#include "storage/shmem.h"
#include "storage/spin.h"
#include "utils/builtins.h"
#include "utils/guc.h"
#include "utils/memutils.h"
-#include "utils/timestamp.h"
-#include "access/htup_details.h"
-
-#include <stdio.h>
PG_MODULE_MAGIC;
@@ -252,68 +250,56 @@ resizable_shmem_read(PG_FUNCTION_ARGS)
}
/*
- * Report multiple memory usage statistics of the calling backend process
- * as reported by the kernel.
- * Returns RssAnon, RssFile, RssShmem, VmSize from /proc/self/status as a record.
+ * Return the memory mapped against the main shared memory segment in this
+ * backend.
*
- * The function assumes that these values will be available in
- * /proc/self/status, any system which also support madvise with MADV_REMOVE and
- * MADV_POPULATE_WRITE.
+ * The VMA containing our resizable_shmem pointer is used to determine the main
+ * memory segment. RSS + Swap (in bytes) for that VMS from /proc/self/smaps is
+ * returned.
*/
Datum
resizable_shmem_usage(PG_FUNCTION_ARGS)
{
FILE *f;
char line[256];
- int64 rss_anon_kb = -1;
- int64 rss_file_kb = -1;
- int64 rss_shmem_kb = -1;
- int64 vm_size_kb = -1;
- int found = 0;
- TupleDesc tupdesc;
- Datum values[4];
- bool nulls[4];
- HeapTuple tuple;
-
- /* Open /proc/self/status to read memory information */
- f = fopen("/proc/self/status", "r");
+ int64 rss_kb = -1;
+ int64 swap_kb = -1;
+ uintptr_t target = (uintptr_t) resizable_shmem;
+ bool in_target_vma = false;
+ size_t result;
+
+ f = fopen("/proc/self/smaps", "r");
if (f == NULL)
ereport(ERROR,
(errcode_for_file_access(),
- errmsg("could not open /proc/self/status: %m")));
+ errmsg("could not open /proc/self/smaps: %m")));
- /* Look for the memory usage lines */
- while (fgets(line, sizeof(line), f) != NULL && found < 4)
+ while (fgets(line, sizeof(line), f) != NULL)
{
- if (rss_anon_kb == -1 && sscanf(line, "RssAnon: %ld kB", &rss_anon_kb) == 1)
- found++;
- else if (rss_file_kb == -1 && sscanf(line, "RssFile: %ld kB", &rss_file_kb) == 1)
- found++;
- else if (rss_shmem_kb == -1 && sscanf(line, "RssShmem: %ld kB", &rss_shmem_kb) == 1)
- found++;
- else if (vm_size_kb == -1 && sscanf(line, "VmSize: %ld kB", &vm_size_kb) == 1)
- found++;
+ unsigned long start;
+ unsigned long end;
+
+ if (sscanf(line, "%lx-%lx", &start, &end) == 2)
+ {
+ in_target_vma = (target >= start && target < end);
+ }
+ else if (in_target_vma)
+ {
+ if (rss_kb == -1)
+ sscanf(line, "Rss: %ld kB", &rss_kb);
+ if (swap_kb == -1)
+ sscanf(line, "Swap: %ld kB", &swap_kb);
+ if (rss_kb >= 0 && swap_kb >= 0)
+ break;
+ }
}
fclose(f);
- /* Build tuple descriptor for our result type */
- if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("function returning record called in context "
- "that cannot accept a record")));
-
- /* Build the result tuple */
- values[0] = Int64GetDatum(rss_anon_kb >= 0 ? rss_anon_kb * 1024 : 0);
- values[1] = Int64GetDatum(rss_file_kb >= 0 ? rss_file_kb * 1024 : 0);
- values[2] = Int64GetDatum(rss_shmem_kb >= 0 ? rss_shmem_kb * 1024 : 0);
- values[3] = Int64GetDatum(vm_size_kb >= 0 ? vm_size_kb * 1024 : 0);
-
- nulls[0] = nulls[1] = nulls[2] = nulls[3] = false;
+ result = rss_kb >= 0 ? mul_size(rss_kb, 1024) : 0;
+ result = add_size(result, swap_kb >= 0 ? mul_size(swap_kb, 1024) : 0);
- tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_INT64(result);
}
/*
diff --git a/src/test/modules/resizable_shmem/t/001_resizable_shmem.pl b/src/test/modules/resizable_shmem/t/001_resizable_shmem.pl
index 6d45b1eccdc..a172cd0fd19 100644
--- a/src/test/modules/resizable_shmem/t/001_resizable_shmem.pl
+++ b/src/test/modules/resizable_shmem/t/001_resizable_shmem.pl
@@ -19,7 +19,7 @@ sub check_shmem_usage
{
my ($session, $label, $node) = @_;
- my $rss_shmem = $session->query_safe('SELECT rss_shmem FROM resizable_shmem_usage();',
+ my $rss_shmem = $session->query_safe('SELECT resizable_shmem_usage();',
verbose => 0);
my $total_alloc = $node->safe_psql('postgres',
"SELECT sum(allocated_size) FROM pg_shmem_allocations;");
--
2.34.1
[text/x-patch] v20260406-0006-Add-support-to-protect-unused-resizable_sh.patch (11.8K, 7-v20260406-0006-Add-support-to-protect-unused-resizable_sh.patch)
download | inline diff:
From ef321b327e7ff0a9abc1eb34f11910d7b71d32dd Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat <[email protected]>
Date: Mon, 6 Apr 2026 19:00:14 +0530
Subject: [PATCH v20260406 6/6] Add support to protect unused resizable_shmem
structure
Add APIs to make the portion of resizable_shmem structure beyond its current
size inaccessible.
Author: Ashutosh Bapat <[email protected]>
Suggested-by: Matthias van de Meent <[email protected]>
---
doc/src/sgml/xfunc.sgml | 13 +++-
src/backend/port/sysv_shmem.c | 55 +++++++++++++++++
src/backend/port/win32_shmem.c | 8 +++
src/backend/storage/ipc/shmem.c | 60 +++++++++++++++++++
src/include/storage/pg_shmem.h | 1 +
src/include/storage/shmem.h | 1 +
.../modules/resizable_shmem/resizable_shmem.c | 51 ++++++++++++++++
7 files changed, 186 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/xfunc.sgml b/doc/src/sgml/xfunc.sgml
index a6c7b8b1b22..62b33366f30 100644
--- a/doc/src/sgml/xfunc.sgml
+++ b/doc/src/sgml/xfunc.sgml
@@ -3781,9 +3781,16 @@ my_shmem_init(void *arg)
shared structure. Also accessing the memory beyond the current size of the
structure will not cause any segmentation fault or a bus error. Memory will
be allocated during such a write access. 0s will be returned on such a read
- access if memory is not allocated yet. The additional synchronization may
- use mprotect() with PROT_NONE in every backend that may access this memory
- to ensure that such an access results in a fault.
+ access if memory is not allocated yet.
+ </para>
+
+ <para>
+ <function>ShmemProtectStruct</function> can be called when resizing the
+ structure to make the unused portion of the structure inaccessible and the
+ used portion accessible. These protections work only at the memory page
+ level, so some unused portion may still remain accessible. Please note that
+ the function modifies the protections only in the backend where it is run.
+ It needs to be called from every backend that may access the structure.
</para>
<para>
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index bb2a81417c6..14b6fa7f7e6 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -1065,8 +1065,63 @@ PGSharedMemoryEnsureAllocated(void *addr, Size size)
Assert(addr == (void *) TYPEALIGN(GetOSPageSize(), addr));
Assert(size == TYPEALIGN(GetOSPageSize(), size));
+ /*
+ * Ensure that MADV_POPULATE_WRITE can initialize the newly allocated
+ * pages.
+ */
+ if (mprotect(addr, size, PROT_READ | PROT_WRITE) != 0)
+ ereport(ERROR,
+ (errmsg("could not protect shared memory: %m")));
+
if (madvise(addr, size, MADV_POPULATE_WRITE) == -1)
ereport(ERROR,
(errmsg("could not allocate shared memory: %m")));
#endif
}
+
+/*
+ * Set memory protection on the given region of shared memory.
+ *
+ * Makes [rw_start, rw_end) readable and writable, and [rw_end, prot_end)
+ * inaccessible.
+ *
+ * All addresses are expected to be page aligned.
+ *
+ * Only supported on platforms that support resizable shared memory.
+ */
+void
+PGSharedMemoryProtect(void *rw_start, void *rw_end, void *prot_end)
+{
+#ifndef HAVE_RESIZABLE_SHMEM
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("resizable shared memory is not supported on this platform")));
+#else
+
+ if (!AnonymousShmem)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("only anonymous shared memory can be protected at runtime")));
+
+ Assert(rw_start == (void *) TYPEALIGN(GetOSPageSize(), rw_start));
+ Assert(rw_end == (void *) TYPEALIGN(GetOSPageSize(), rw_end));
+ Assert(prot_end == (void *) TYPEALIGN(GetOSPageSize(), prot_end));
+ Assert(rw_end >= rw_start);
+
+ if (rw_end > rw_start)
+ {
+ if (mprotect(rw_start, (char *) rw_end - (char *) rw_start,
+ PROT_READ | PROT_WRITE) != 0)
+ ereport(ERROR,
+ (errmsg("could not protect shared memory: %m")));
+ }
+
+ if (prot_end > rw_end)
+ {
+ if (mprotect(rw_end, (char *) prot_end - (char *) rw_end,
+ PROT_NONE) != 0)
+ ereport(ERROR,
+ (errmsg("could not protect shared memory: %m")));
+ }
+#endif
+}
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index c1f30665e66..b5396e4a5e8 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -693,3 +693,11 @@ PGSharedMemoryEnsureAllocated(void *addr, Size size)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("resizable shared memory is not supported on this platform")));
}
+
+void
+PGSharedMemoryProtect(void *rw_start, void *rw_end, void *prot_end)
+{
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("resizable shared memory is not supported on this platform")));
+}
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 115c543d36a..a3ed082e4d9 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -880,6 +880,66 @@ ShmemResizeStruct(const char *name, Size new_size)
#endif
}
+/*
+ * ShmemProtectStruct() --- protect the unused portion of a resizable structure.
+ *
+ * Makes the region beyond the current size up to maximum_size inaccessible, and
+ * ensures the region up to the current size is readable and writable. Depending
+ * upon the platform, the protection honours the page boundaries. So it may be
+ * more permissible than strictly needed.
+ *
+ * Only works for resizable structures. Should be called in every backend that
+ * may access the resizable structure while resizing it.
+ */
+void
+ShmemProtectStruct(const char *name)
+{
+#ifndef HAVE_RESIZABLE_SHMEM
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("resizable shared memory is not supported on this platform")));
+#else
+ ShmemIndexEnt *result;
+ bool found;
+ Size page_size = GetOSPageSize();
+ char *rw_start;
+ char *rw_end;
+ char *prot_end;
+
+ LWLockAcquire(ShmemIndexLock, LW_SHARED);
+ result = (ShmemIndexEnt *) hash_search(ShmemIndex, name, HASH_FIND, &found);
+ if (!found)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("shmem struct \"%s\" is not initialized", name)));
+
+ if (result->minimum_size == result->maximum_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("shared memory struct \"%s\" is not resizable", name)));
+
+ /* Resizable structures are only supported with mmap-based shared memory. */
+ Assert(shared_memory_type == SHMEM_TYPE_MMAP);
+
+ /* Make at least [location, location+size) readable and writable */
+ rw_start = (char *) TYPEALIGN_DOWN(page_size, result->location);
+ rw_end = (char *) TYPEALIGN(page_size,
+ (char *) result->location + result->size);
+
+ /*
+ * Make remaining portion inaccessible while making sure that the portion
+ * after maximum_size is not affected since it may be used by other
+ * structures.
+ */
+ prot_end = (char *) TYPEALIGN_DOWN(page_size,
+ (char *) result->location + result->maximum_size);
+
+ LWLockRelease(ShmemIndexLock);
+
+ PGSharedMemoryProtect(rw_start, rw_end, prot_end);
+#endif
+}
+
/*
* InitShmemAllocator() --- set up basic pointers to shared memory.
*
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index f0efbf2aec1..5165b815cc1 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -91,6 +91,7 @@ extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void PGSharedMemoryEnsureFreed(void *addr, Size size);
extern void PGSharedMemoryEnsureAllocated(void *addr, Size size);
+extern void PGSharedMemoryProtect(void *rw_start, void *rw_end, void *prot_end);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
extern Size GetOSPageSize(void);
diff --git a/src/include/storage/shmem.h b/src/include/storage/shmem.h
index 0e6d5a63f28..f8ddb0dd7c0 100644
--- a/src/include/storage/shmem.h
+++ b/src/include/storage/shmem.h
@@ -185,6 +185,7 @@ typedef struct ShmemCallbacks
extern void RegisterShmemCallbacks(const ShmemCallbacks *callbacks);
extern bool ShmemAddrIsValid(const void *addr);
extern void ShmemResizeStruct(const char *name, Size new_size);
+extern void ShmemProtectStruct(const char *name);
/*
* These macros provide syntactic sugar for calling the underlying functions
diff --git a/src/test/modules/resizable_shmem/resizable_shmem.c b/src/test/modules/resizable_shmem/resizable_shmem.c
index d035d767a62..c02ba54896a 100644
--- a/src/test/modules/resizable_shmem/resizable_shmem.c
+++ b/src/test/modules/resizable_shmem/resizable_shmem.c
@@ -56,10 +56,12 @@ static bool use_unknown_size = false;
static void resizable_shmem_request(void *arg);
static void resizable_shmem_shmem_init(void *arg);
+static void resizable_shmem_shmem_attach(void *arg);
static ShmemCallbacks shmem_callbacks = {
.request_fn = resizable_shmem_request,
.init_fn = resizable_shmem_shmem_init,
+ .attach_fn = resizable_shmem_shmem_attach,
};
/* SQL-callable functions */
@@ -172,10 +174,32 @@ resizable_shmem_shmem_init(void *arg)
*/
Assert(resizable_shmem != NULL);
+#ifdef HAVE_RESIZABLE_SHMEM
+ /* Protect the shared memory structure in this backend. */
+ ShmemProtectStruct("resizable_shmem");
+#endif
+
resizable_shmem->num_entries = test_initial_entries;
memset(resizable_shmem->data, 0, mul_size(test_initial_entries, TEST_ENTRY_SIZE));
}
+/*
+ * Protect the shared memory structure memory after attaching.
+ */
+static void
+resizable_shmem_shmem_attach(void *arg)
+{
+ /*
+ * Shared memory structure should have been already allocated. Initialize
+ * it.
+ */
+ Assert(resizable_shmem != NULL);
+
+#ifdef HAVE_RESIZABLE_SHMEM
+ ShmemProtectStruct("resizable_shmem");
+#endif
+}
+
/*
* Resize the shared memory structure to accommodate the specified number of
* entries.
@@ -198,6 +222,7 @@ resizable_shmem_resize(PG_FUNCTION_ARGS)
new_size = add_size(offsetof(TestResizableShmemStruct, data),
mul_size(new_entries, TEST_ENTRY_SIZE));
ShmemResizeStruct("resizable_shmem", new_size);
+ ShmemProtectStruct("resizable_shmem");
resizable_shmem->num_entries = new_entries;
PG_RETURN_VOID();
@@ -217,6 +242,19 @@ resizable_shmem_write(PG_FUNCTION_ARGS)
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("resizable_shmem is not initialized")));
+#ifdef HAVE_RESIZABLE_SHMEM
+
+ /*
+ * Ideally the structure should be protected through a synchronization
+ * cycle across all the backends that may access the structure. But we
+ * don't implement any such synchronization in this test module to keep it
+ * simple. Given that ProcSignalBarrier mechanism is not extensible, we
+ * may not be able to do that as well here. Hence add protect just before
+ * accessing the structure.
+ */
+ ShmemProtectStruct("resizable_shmem");
+#endif
+
/* Write the value to all current entries */
for (i = 0; i < resizable_shmem->num_entries; i++)
resizable_shmem->data[i] = entry_value;
@@ -245,6 +283,19 @@ resizable_shmem_read(PG_FUNCTION_ARGS)
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("entry_count %d is out of range (0..%d)", entry_count, resizable_shmem->num_entries)));
+#ifdef HAVE_RESIZABLE_SHMEM
+
+ /*
+ * Ideally the structure should be protected through a synchronization
+ * cycle across all the backends that may access the structure. But we
+ * don't implement any such synchronization in this test module to keep it
+ * simple. Given that ProcSignalBarrier mechanism is not extensible, we
+ * may not be able to do that as well here. Hence add protect just before
+ * accessing the structure.
+ */
+ ShmemProtectStruct("resizable_shmem");
+#endif
+
for (i = 0; i < entry_count; i++)
{
if (resizable_shmem->data[i] != entry_value)
--
2.34.1
view thread (82+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: Better shared data structure management and resizable shared data structures
In-Reply-To: <CAExHW5u5LOgB3Y4Ee3VWVEurYN2GwnkbVEcfrGCQLgcGgf_zKw@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox