Message-ID: <ea001640-3818-4023-b0d5-17ce05c20701@iki.fi>
Date: Tue, 21 Apr 2026 10:40:14 +0300
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Better shared data structure management and resizable shared data
 structures
To: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Cc: =?UTF-8?Q?Dagfinn_Ilmari_Manns=C3=A5ker?= <ilmari@ilmari.org>,
 Robert Haas <robertmhaas@gmail.com>, Andres Freund <andres@anarazel.de>,
 pgsql-hackers <pgsql-hackers@postgresql.org>, chaturvedipalak1911@gmail.com
References: 
 <CAExHW5vM1bneLYfg0wGeAa=52UiJ3z4vKd3AJ72X8Fw6k3KKrg@mail.gmail.com>
 <cc3ff976-c303-45cd-9a8d-a90483170e37@iki.fi>
 <CA+TgmoauqYXm8iA3FGRAVKxYShUxWBiS_MSLmQpTrmO7wNHamw@mail.gmail.com>
 <113724ab-0028-493f-9605-6e8570f0939f@iki.fi>
 <CAExHW5uTNWOSxJDWQAUnS0tZawob2_J3dRAtc67NHNZ98X4_xA@mail.gmail.com>
 <CAExHW5t439y61YD9bc7d5wZWHp6J=M43Qu3eEZOBPguZML7o2A@mail.gmail.com>
 <CAExHW5v5FVZbsO9sLzztMZ11C3hgGStE=HkkV2bQkCyncess4w@mail.gmail.com>
 <b2451225-0a10-4ded-abef-4e9efbde7a8e@iki.fi>
 <CAExHW5tCC0T1ky=Jnq-AvMxa67Adaw7aQ4iQAO=BSdHcbSNBVg@mail.gmail.com>
 <791c3f18-f4de-4d84-ac6b-c7ccc074dd38@iki.fi>
 <CAExHW5tS7GncN90oJWOSzW_3F1EHL9xwe59L7Req3nUVgmObUw@mail.gmail.com>
 <9d919bd9-94dd-4bda-8ccf-ebced4178c53@iki.fi>
 <CAExHW5sYgt=XekOoAE-Tu_Dv8obWZCjCqAPT9vtN2D4m=M=drQ@mail.gmail.com>
 <470e7ebe-0971-49f6-8e46-9b8f6395f88b@iki.fi>
 <87y0iz2c1v.fsf@wibble.ilmari.org>
 <4297b4ee-5248-415f-ae12-abf2a23f4ab6@iki.fi>
 <CAExHW5uMQGvQH6GKaBZVtH4S9O13TwN+_0Vy1gUpAW=_T_AmRA@mail.gmail.com>
Content-Language: en-US
From: Heikki Linnakangas <hlinnaka@iki.fi>
In-Reply-To: 
 <CAExHW5uMQGvQH6GKaBZVtH4S9O13TwN+_0Vy1gUpAW=_T_AmRA@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/ea001640-3818-4023-b0d5-17ce05c20701%40iki.fi>
Precedence: bulk

On 07/04/2026 17:19, Ashutosh Bapat wrote:
> Hi Heikki,
> CallShmemCallbacksAfterStartup() holds ShmemIndexLock while invoking
> init_fn/attach_fn callbacks. That looks wrong. Before this commit,
> init or attach code was not run with the lock held. Any reason the
> lock is held while calling init and attach callbacks. Since these
> function can come from extensions, we don't have control on what goes
> in those functions, and thus looks problematic. Further, it will
> serialize all the attach_fn executions across backends, since each
> will be run under the lock.

This was intentional, I added a note in the docs about it:

       When <function>RegisterShmemCallbacks()</function> is called after
       startup, it will immediately call the appropriate callbacks, 
depending
       on whether the requested memory areas were already initialized by
       another backend. The callbacks will be called while holding an 
internal
       lock, which prevents concurrent two backends from initializing the
       memory area concurrently.

That "internal lock" is ShmemIndexLock. I piggybacked on that since the 
code needs to acquire it anyway for the hash table lookups.

With the old ShmemInitStruct() interface, extensions needed to do the 
locking themselves, usually by holding AddinShmemInitLock.

(Now that I read that again, the grammar on the last sentence sounds 
awkward...)

> In my case, the init_fn was performing ShmemIndex lookup which
> deadlocked. It's questionable whether init function should lookup
> ShmemIndex but, it's not something that needs to be prohibited
> either.
Yeah I'm curious what the use case is. We could easily introduce another 
lock or reuse AddinShmemInitLock for this.

- Heikki