Message-ID: <c7b4dafc-7a6c-43e4-8b9e-20807396590b@iki.fi>
Date: Thu, 12 Mar 2026 21:21:19 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Better shared data structure management and resizable shared data
 structures
To: Robert Haas <robertmhaas@gmail.com>
Cc: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>,
 Andres Freund <andres@anarazel.de>,
 pgsql-hackers <pgsql-hackers@postgresql.org>, chaturvedipalak1911@gmail.com
References: 
 <CAExHW5vM1bneLYfg0wGeAa=52UiJ3z4vKd3AJ72X8Fw6k3KKrg@mail.gmail.com>
 <5a37c2e3-619d-4816-84d7-0b27e3e6797f@iki.fi>
 <CAExHW5s9Vp+-vJi020UJ+otyccBBo7eT1g6bttdRKL6HAvscyQ@mail.gmail.com>
 <mlsruptoxgm2nqtdfyfsowjklzxl5zltsjb3y5bmywtigm474l@5tsonk4t3kia>
 <CAExHW5uEK+eeG7e2g6uWh7POrFpfp+dqfaa=_3miMN17zgeaJw@mail.gmail.com>
 <CAExHW5vz+PUHHUuzGRwtyx-mPLQk3nCZXxrFqnruRadEFrO5Xg@mail.gmail.com>
 <CAExHW5so6VSxBC-1V=35229Z1+dw5vhw8HxHg9ry7UzceKcXzA@mail.gmail.com>
 <26c766d6-db0f-43d3-a618-44f8d40a3121@iki.fi>
 <CA+TgmoaoSha9KnSdrA7DDMYD9+Uor4OPEaQ88bbEC3QuVE5Mcg@mail.gmail.com>
 <62b8dc23-8f6a-4cac-91ff-f74bb5bc159a@iki.fi>
 <CA+TgmoboxAaS6MCRf__LRhf8Ji3B0O9_=d7KzRS12QhOuocnTQ@mail.gmail.com>
Content-Language: en-US
From: Heikki Linnakangas <hlinnaka@iki.fi>
In-Reply-To: 
 <CA+TgmoboxAaS6MCRf__LRhf8Ji3B0O9_=d7KzRS12QhOuocnTQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Archived-At: 
 <https://www.postgresql.org/message-id/c7b4dafc-7a6c-43e4-8b9e-20807396590b%40iki.fi>
Precedence: bulk

On 12/03/2026 20:56, Robert Haas wrote:
> On Thu, Mar 12, 2026 at 2:41 PM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
>> shmem_startup_hook() is too late. The shmem structs need to be
>> registered at postmaster startup before the shmem segment is allocated,
>> so that we can calculate the total size needed.
> 
> Sorry, I meant shmem_request_hook.

Ah ok

>> I'm currently leaning towards _PG_init(), except for allocations that
>> depend on MaxBackends. For those, you can install a shmem_request_hook
>> that sets the size in the descriptor. In other words, you can leave the
>> 'size' as empty in _PG_init(), but set it later in the shmem_request_hook.
> 
> Why can't you just do the whole thing later?

shmem_request_hook won't work in EXEC_BACKEND mode, because in 
EXEC_BACKEND mode, ShmemRegisterStruct() also needs to be called at 
backend startup.

One of my design goals is to avoid EXEC_BACKEND specific steps so that 
if you write your extension oblivious to EXEC_BACKEND mode, it will 
still usually work with EXEC_BACKEND. For example, if it was necessary 
to call a separate AttachShmem() function for every shmem struct in 
EXEC_BACKEND mode, but which was not needed on Unix, that would be bad.

>> Except that you'd still need them for RequestNamedLWLockTranche(). I
>> wonder if we should recommend extensions to embed the LWLock struct into
>> their shared memory struct and use the LWLockInitialize() and
>> LWLockNewTrancheId() functions instead. That fits the new
>> ShmemRegisterStruct() API a little better than RequestNamedLWLockTranche().
> 
> Yeah, I think RequestNamedLWLockTranche() might be fine if you just
> need LWLocks, but if you need a bunch of resources, putting them all
> into the same chunk of memory seems cleaner.

Agreed. Then again, how often do you need just a LWLock (or multiple 
LWLocks)? Surely you have a struct you want to protect with the lock. I 
guess having shmem hash table but no struct would be pretty common, though.

- Heikki