Message-ID: <7d3ba240-9350-4dfc-bbe1-be6584aee236@iki.fi>
Date: Sat, 4 Apr 2026 03:45:18 +0300
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Better shared data structure management and resizable shared data
 structures
To: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>,
 Matthias van de Meent <boekewurm+postgres@gmail.com>
Cc: Robert Haas <robertmhaas@gmail.com>, Andres Freund <andres@anarazel.de>,
 pgsql-hackers <pgsql-hackers@postgresql.org>, chaturvedipalak1911@gmail.com
References: 
 <CAExHW5vM1bneLYfg0wGeAa=52UiJ3z4vKd3AJ72X8Fw6k3KKrg@mail.gmail.com>
 <8a6799be-bd42-49fb-8914-856c97bb1977@iki.fi>
 <CA+Tgmob8ot7qam7a650nx_4jyEoFLs8s_kPMc1+EB7-dr56=WA@mail.gmail.com>
 <cc3ff976-c303-45cd-9a8d-a90483170e37@iki.fi>
 <CA+TgmoauqYXm8iA3FGRAVKxYShUxWBiS_MSLmQpTrmO7wNHamw@mail.gmail.com>
 <113724ab-0028-493f-9605-6e8570f0939f@iki.fi>
 <CAExHW5uTNWOSxJDWQAUnS0tZawob2_J3dRAtc67NHNZ98X4_xA@mail.gmail.com>
 <CAExHW5t439y61YD9bc7d5wZWHp6J=M43Qu3eEZOBPguZML7o2A@mail.gmail.com>
 <CAExHW5v5FVZbsO9sLzztMZ11C3hgGStE=HkkV2bQkCyncess4w@mail.gmail.com>
 <b2451225-0a10-4ded-abef-4e9efbde7a8e@iki.fi>
 <CAExHW5tCC0T1ky=Jnq-AvMxa67Adaw7aQ4iQAO=BSdHcbSNBVg@mail.gmail.com>
 <791c3f18-f4de-4d84-ac6b-c7ccc074dd38@iki.fi>
 <CAExHW5tS7GncN90oJWOSzW_3F1EHL9xwe59L7Req3nUVgmObUw@mail.gmail.com>
 <9d919bd9-94dd-4bda-8ccf-ebced4178c53@iki.fi>
 <CAEze2WhMOHVgH2Xeyzx=VEk-Ta_YnQUqT+TdBiv5Lx8ESn2WZA@mail.gmail.com>
 <CAExHW5s6h=c_q2m72Nvyj1ghMEhPkOBkeN5Htn7YR=1BrNN-Sw@mail.gmail.com>
Content-Language: en-US
From: Heikki Linnakangas <hlinnaka@iki.fi>
In-Reply-To: 
 <CAExHW5s6h=c_q2m72Nvyj1ghMEhPkOBkeN5Htn7YR=1BrNN-Sw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Archived-At: 
 <https://www.postgresql.org/message-id/7d3ba240-9350-4dfc-bbe1-be6584aee236%40iki.fi>
Precedence: bulk

On 03/04/2026 16:12, Ashutosh Bapat wrote:
> On Fri, Apr 3, 2026 at 3:40 AM Matthias van de Meent
> <boekewurm+postgres@gmail.com> wrote:
>> While I do think it's an improvement over the current APIs, the
>> improvement seems to be mostly concentrated in the RequestStruct/Hash
>> department, with only marginal improvements in RegisterShmemCallbacks.
>> I feel like it's missing the important part: I'd like
>> direct-from-_PG_init() ShmemRequestStruct/Hash calls. If
>> ShmemRequestStruct/Hash had a size callback as alternative to the size
>> field (which would then be called after preload_libraries finishes)
>> then that would be sufficient for most shmem allocations, and it'd
>> simplify shmem management for most subsystems.
>> We'd still need the shmem lifecycle hooks/RegisterShmemCallbacks to
>> allow conditionally allocated shmem areas (e.g. those used in aio),
>> but I think that, in general, we shouldn't need a separate callback
>> function just to get started registering shmem structures.
>>
>> I also noticed that ShmemCallbacks.%_arg are generally undocumented,
>> and I couldn't find any users in core (at the end of the patchset)
>> that actually use the argument. Could it be I missed something?

None of the current code currently uses it, that's correct. I felt it 
might become very handy in the future or in extensions, if you wanted to 
reuse the same function for initializing different shmem areas, for 
example. It's a pretty common pattern to have an opaque pointer like 
that in any callbacks.

>> I don't understand the use of ShmemStructDesc. They generally/always
>> are private to request_fn(), and their fields are used exclusively
>> inside the shmem mechanisms, with no reads of its fields that can't
>> already be deduced from context. Why do we need that struct
>> everywhere?
> 
> My resizable shared memory structure patches use it as a handle to the
> structure to be resized.

Right. And hash tables and SLRUs use a desc-like object already, so for 
symmetry it feels natural to have it for plain structs too.

I wonder if we should make it optional though, for the common case that 
you have no intention of doing anything more with the shmem region that 
you'd need a desc for. I'm thinking you could just pass NULL for the 
desc pointer:

     ShmemRequestStruct(NULL,
         .name = "pg_stat_statements",
         .size = sizeof(pgssSharedState),
         .ptr = (void **) &pgss,
     };

- Heikki