Message-ID: <b441d0e6-807f-4790-b41f-ab1212c54466@iki.fi>
Date: Sun, 5 Apr 2026 21:58:15 +0300
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Better shared data structure management and resizable shared data
 structures
To: Matthias van de Meent <boekewurm+postgres@gmail.com>
Cc: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>,
 Robert Haas <robertmhaas@gmail.com>, Andres Freund <andres@anarazel.de>,
 pgsql-hackers <pgsql-hackers@postgresql.org>, chaturvedipalak1911@gmail.com
References: 
 <CAExHW5vM1bneLYfg0wGeAa=52UiJ3z4vKd3AJ72X8Fw6k3KKrg@mail.gmail.com>
 <CA+TgmobqwCHp1w6xkTJmZiq4C8Pr0O3tNUvSFZcAgYLkvF45bg@mail.gmail.com>
 <8a6799be-bd42-49fb-8914-856c97bb1977@iki.fi>
 <CA+Tgmob8ot7qam7a650nx_4jyEoFLs8s_kPMc1+EB7-dr56=WA@mail.gmail.com>
 <cc3ff976-c303-45cd-9a8d-a90483170e37@iki.fi>
 <CA+TgmoauqYXm8iA3FGRAVKxYShUxWBiS_MSLmQpTrmO7wNHamw@mail.gmail.com>
 <113724ab-0028-493f-9605-6e8570f0939f@iki.fi>
 <CAExHW5uTNWOSxJDWQAUnS0tZawob2_J3dRAtc67NHNZ98X4_xA@mail.gmail.com>
 <CAExHW5t439y61YD9bc7d5wZWHp6J=M43Qu3eEZOBPguZML7o2A@mail.gmail.com>
 <CAExHW5v5FVZbsO9sLzztMZ11C3hgGStE=HkkV2bQkCyncess4w@mail.gmail.com>
 <b2451225-0a10-4ded-abef-4e9efbde7a8e@iki.fi>
 <CAExHW5tCC0T1ky=Jnq-AvMxa67Adaw7aQ4iQAO=BSdHcbSNBVg@mail.gmail.com>
 <791c3f18-f4de-4d84-ac6b-c7ccc074dd38@iki.fi>
 <CAExHW5tS7GncN90oJWOSzW_3F1EHL9xwe59L7Req3nUVgmObUw@mail.gmail.com>
 <9d919bd9-94dd-4bda-8ccf-ebced4178c53@iki.fi>
 <CAEze2WhMOHVgH2Xeyzx=VEk-Ta_YnQUqT+TdBiv5Lx8ESn2WZA@mail.gmail.com>
Content-Language: en-US
From: Heikki Linnakangas <hlinnaka@iki.fi>
In-Reply-To: 
 <CAEze2WhMOHVgH2Xeyzx=VEk-Ta_YnQUqT+TdBiv5Lx8ESn2WZA@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/b441d0e6-807f-4790-b41f-ab1212c54466%40iki.fi>
Precedence: bulk

On 03/04/2026 01:10, Matthias van de Meent wrote:
> While I do think it's an improvement over the current APIs, the
> improvement seems to be mostly concentrated in the RequestStruct/Hash
> department, with only marginal improvements in RegisterShmemCallbacks.
> I feel like it's missing the important part: I'd like
> direct-from-_PG_init() ShmemRequestStruct/Hash calls. If
> ShmemRequestStruct/Hash had a size callback as alternative to the size
> field (which would then be called after preload_libraries finishes)
> then that would be sufficient for most shmem allocations, and it'd
> simplify shmem management for most subsystems.
> We'd still need the shmem lifecycle hooks/RegisterShmemCallbacks to
> allow conditionally allocated shmem areas (e.g. those used in aio),
> but I think that, in general, we shouldn't need a separate callback
> function just to get started registering shmem structures.

I kind of started from that thought too, but the design has since 
evolved to what it is. Robert in particular was skeptical of that 
approach, and I think I've come around on most of his feedback.

A per-struct size callback isn't very ergonomic in places like 
LockManagerShmemRequest(), which derives the size of two different 
things from the same calculated value. You'd need to repeat the 
calculation for each one, or pass it through global variables or 
something. PredicateLockShmemInit() is another extreme example. It 
already has that problem to some extent, and already uses global 
variables (I'm all ears if you have suggestions to improve it!) but I 
feel that with a size callback this kind of stuff would get even harder.

Another reason is that it's good to have only one way of doing the 
initialization, instead of one simple way and a different way for more 
complex scenarios. If the simple way was vastly simpler, it might be 
worth it, but I don't think there's that much difference here.


BTW, I also considered another way of initializing structs for simple 
cases: instead of providing an init callback function, you could provide 
the struct contents directly in the ShmemRequestStruct() call. To pick a 
random example, slotsync.c currently looks like this:

static void
SlotSyncShmemRequest(void *arg)
{
     ShmemRequestStruct(.name = "Slot Sync Data",
                        .size = sizeof(SlotSyncCtxStruct),
                        .ptr = (void **) &SlotSyncCtx,
     );
}

static void
SlotSyncShmemInit(void *arg)
{
     memset(SlotSyncCtx, 0, sizeof(SlotSyncCtxStruct));
     SlotSyncCtx->pid = InvalidPid;
     SpinLockInit(&SlotSyncCtx->mutex);
}


But it could look like this instead:

static void
SlotSyncShmemRequest(void *arg)
{
     SlotSyncCtxStruct init_content;

     memset(SlotSyncCtx, 0, sizeof(SlotSyncCtxStruct));
     init_content.pid
     SpinLockInit(&SlotSyncCtx->mutex);

     ShmemRequestStruct(.name = "Slot Sync Data",
                        .size = sizeof(SlotSyncCtxStruct),
                        .ptr = (void **) &SlotSyncCtx,
                        .init_content = &init_content,
		);
}

That'd only work for small, simple structs, though. Since the initial 
contents would be copied in this model, it won't work for anything with 
pointers to itself, for example. And it's not much less code after all.

- Heikki