public inbox for [email protected]
help / color / mirror / Atom feedFrom: Heikki Linnakangas <[email protected]>
To: Ashutosh Bapat <[email protected]>
Cc: Matthias van de Meent <[email protected]>
Cc: Robert Haas <[email protected]>
Cc: Andres Freund <[email protected]>
Cc: pgsql-hackers <[email protected]>
Cc: [email protected]
Subject: Re: Better shared data structure management and resizable shared data structures
Date: Tue, 7 Apr 2026 22:48:17 +0300
Message-ID: <[email protected]> (raw)
In-Reply-To: <CAExHW5v5muT_SKV2NCxxVmvC=_38Rw0aiv-wU4CGzHaBCRYzqA@mail.gmail.com>
References: <CAExHW5vM1bneLYfg0wGeAa=52UiJ3z4vKd3AJ72X8Fw6k3KKrg@mail.gmail.com>
<CAExHW5tCC0T1ky=Jnq-AvMxa67Adaw7aQ4iQAO=BSdHcbSNBVg@mail.gmail.com>
<[email protected]>
<CAExHW5tS7GncN90oJWOSzW_3F1EHL9xwe59L7Req3nUVgmObUw@mail.gmail.com>
<[email protected]>
<CAEze2WhMOHVgH2Xeyzx=VEk-Ta_YnQUqT+TdBiv5Lx8ESn2WZA@mail.gmail.com>
<CAExHW5s6h=c_q2m72Nvyj1ghMEhPkOBkeN5Htn7YR=1BrNN-Sw@mail.gmail.com>
<[email protected]>
<CAEze2WjQZff3znd6CtG-OBzYZMMqy5TyQSoAo=QTFT38tDndeQ@mail.gmail.com>
<[email protected]>
<CAEze2WjgCROMMXY0+j8FFdm3iFcr7By-+6Mwiz=PgGSEydiW3A@mail.gmail.com>
<[email protected]>
<[email protected]>
<CAExHW5u5LOgB3Y4Ee3VWVEurYN2GwnkbVEcfrGCQLgcGgf_zKw@mail.gmail.com>
<CAExHW5uLau-i0T-ybPa=g5FLQo6OR1U9MR3GNijm6TDv12aLPw@mail.gmail.com>
<CAExHW5v5muT_SKV2NCxxVmvC=_38Rw0aiv-wU4CGzHaBCRYzqA@mail.gmail.com>
On 07/04/2026 17:46, Ashutosh Bapat wrote:
> Here are patches with the test modules merged.
>
> The merged module looks a bit rough to me and so does 0006. For
> example, I am not sure whether calling ShmemStructProtect() from
> init_fn is a good idea. See [1] for example. But init_fn is the last
> chance for the subsystem to touch and setup the resizable structure
> before it's opened to the wild. So, in the current infrastructure, I
> don't see any better place to call ShmemStructProtect() either. If you
> run tests after applying patch 0006, you will need to apply patch
> attached to [1] as well; otherwise the test will hang.
I haven't really looked at these resizeable patches before, except for
how they would fit with the new shmem allocation API, so I have some
very basic, high-level design questions:
> +/*
> + * ShmemResizeStruct() --- resize a resizable shared memory structure.
> + *
> + * The new size must be within [minimum_size, maximum_size]. If the structure
> + * is being shrunk, the memory pages that are no longer needed are freed. If
> + * the structure is being expanded, the memory pages that are needed for the
> + * new size are allocated. See EstimateAllocatedSize() for explanation of which
> + * pages are allocated for a resizable structure.
> + */
> +void
> +ShmemResizeStruct(const char *name, Size new_size)
This interface only allows shrinking and growing the allocated region at
the end, but the underlying mechanism is madvise(MADV_REMOVE) and
madvise(MADV_WRITE_POPULATE), which supports also "punching holes", i.e.
freeing memory in the middle of a region. Do we gain anything by
restricting ourselves to changing the size at the end? It seems to me
that it could be handy to punch holes for some use cases.
What's the portability story? I understand that this is Linux-only at
the moment, but what platforms can we support in the future, and what's
the effort? I think BSD's have similar capabilities with plain mmap()
and MADV_FREE if I read the man pages right. What about macOS and
Windows? This doesn't necessarily need to be fully portable, if some
OS's don't have the capabilities we need, but would be nice to know
what's possible.
- Heikki
view thread (82+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: Better shared data structure management and resizable shared data structures
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox