Content-Type: text/plain;
	charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.700.81\))
Subject: Re: Optimize LISTEN/NOTIFY
From: Chao Li <li.evan.chao@gmail.com>
In-Reply-To: <dab234b5-a10c-4fb5-a2b1-ce725d3e3020@app.fastmail.com>
Date: Tue, 18 Nov 2025 16:15:23 +0800
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <798A8F4A-DC41-4FE9-9BED-38E0DF2193F3@gmail.com>
References: <6899c044-4a82-49be-8117-e6f669765f7e@app.fastmail.com>
 <165530.1752362320@sss.pgh.pa.us>
 <02a7cd37-e2fc-4212-8b19-f8c239c95fb8@app.fastmail.com>
 <e396ecf3-4227-4918-b9ff-e9568dcebcf0@app.fastmail.com>
 <b6a427fe-2ed6-4568-85d9-207a68172617@app.fastmail.com>
 <CAK80=jhmE40KVqQ3ho37MArS7cAED1p9m7uikDxcnDmqdW7t8A@mail.gmail.com>
 <96f00bf1-cc9d-4520-9d02-9e14e7767c88@app.fastmail.com>
 <30c2aa7d-dd6c-4b68-a2e4-f217a1a34acf@app.fastmail.com>
 <CA+hUKGLrMGkWDB0cwTa0RqD+AF7O-Ywgck8aVYKwOQnZgYRRug@mail.gmail.com>
 <0b4d402a-9ac2-4aa8-acf8-8231dbe579ea@app.fastmail.com>
 <c8e84bb2-d2e5-4a77-b547-3a93c5778e7e@app.fastmail.com>
 <3095599.1758644879@sss.pgh.pa.us>
 <0dc6a2cc-5216-4dc1-9dd2-430cafc6095b@app.fastmail.com>
 <E7D8A5CD-C445-4713-B6B7-6CC1D3A71161@gmail.com>
 <baf71275-9083-41be-ae13-f80821541a1a@app.fastmail.com>
 <52CC167F-763B-4ECA-B0B4-DAB381816828@gmail.com>
 <d35a114d-204a-4b40-9284-06749532b724@app.fastmail.com>
 <9186C6D0-F7A9-482A-9183-89E530B57E36@gmail.com>
 <da0996!!!!!!d8-ecf9-490d-bbd0-2a5bdb5b115a@app.fastmail.com>
 <D1CF7CA5-213A-4FC1-86DB-052D58904A0A@gmail.com>
 <cad78ff4-aae9-4c1c-8e16-ff0ccd4957f1@app.fastmail.com>
 <1073593.1759423179@sss.pgh.pa.us>
 <4bd5e6c4-6fa7-44bb-869d-59a32a331fa8@app.fastmail.com>
 <CAFY6G8dap-bCnAnMG-2Gzew8yv2Vbi9gsx9+yszKMmd57ygfvA@mail.gmail.com>
 <85828f29-e72e-4400-94f3-9a69bc8dc239@app.fastmail.com>
 <2495353.1759860890@sss.pgh.pa.us>
 <8aeae418-92a6-4bbd-9c06-9574c79e59f7@app.fastmail.com>
 <2531672.1759868124@sss.pgh.pa.us>
 <474efa78-337c-41cd-a73a-f845a0115109@app.fastmail.com>
 <2749343.1759949176@sss.pgh.pa.us>
 <8bfca2be-1ec0-4e15-aafb-0b7b661fe936@app.fastmail.com>
 <9eba307f-f2fb-48f0-9507-2e197f39ef9e@app.fastmail.com>
 <cde2a91d-d47a-43b5-ab27-f98aa739d38e@app.fastmail.com>
 <8c71183a-0d28-4bcf-a806-78446ff95404@app.fastmail.com>
 <1009807.1760476747@sss.pgh.pa.us>
 <1F7227F5-C33D-4E2C-8511-33F1468590D0@gmail.com>
 <0a5a20d3-4621-46b3-b2ab-903f63a20dea@app.fastmail.com>
 <6F913129-ABEF-4004-AAF3-F22FC34!29AE8@gmail.com>
 <cbabbb45-ec27-47f1-a8f3-eb29073e9aef@app.fastmail.com>
 <ceab2c54-3463-4029-a7d8-895a617bb457@app.fastmail.com>
 <1547585.1760645808@sss.pgh.pa.us>
 <b8df8040-1b15-4a1a-b5f6-74acebbd2d0c@app.fastmail.com>
 <CAE7r3MLivh1sHWF06hrVXkiQbw-KChPcQsh+9CheXprm5vRVMQ@mail.gmail.com>
 <14865EB6-0BF4-462B-9072-10BDAC10C052@gmail.com>
 <CAE7r3MK-3AOdh1mpZ8hw9h6F_i0D5RMoAy7CttnfCJRpB8GJDA@mail.gmail.com>
 <0BCA1C2D-B92C-459E-B1A6-6D06BA4C62CF@gmail.com>
 <55d24cbb-e9ef-491f-a99b-b3dbd7cecdf9@app.fastmail.com>
 <38574cad-e90d-47b7-a015-753bb6bbc360@app.fastmail.com>
 <66631FB7-5BEA-4ED5-A694-9AD8B9CCFEE8@gmail.com>
 <4b7b49a5-5e1a-44a8-93e0-60457d15cb1d@app.fastmail.com>
 <82DEA2B6-6FC5-4A79-BDE3-1FD72F104A6E@gmail.com>
 <38de1036-d8cf-420c-b845-edb5a946b191@app.fastmail.com>
 <87E40BF8-8877-4DBD-9040-99AF8A4E6358@gmail.com>
 <d29ae0a8-b28c-4450-bd9f-c56657e57f45@app.fastmail.com>
 <DF4BDA30-CC41-4BAF-9852-E399C3F273EC@gmail.com>
 <7556f0d4-03fd-451a-bd34-5f62b424319a@app.fastmail.com>
 <290910DE-9A03-4AE6-B348-073D5DA96ACC@gmail.com>
 <CAE7r3MJxxycYwDFJdTJ+HmzeNJ_JGKyr4HpT5njN1ouVX56OUg@mail.gmail.com>
 <4B243750-12BE-4C16-A769-A803268F40E3@gmail.com>
 <CAE7r3MJ0VoAJzdLzX0dgfPJpBJxW+wg_pYaCi6mQJi47+qukhg@mail.gmail.com>
 <4605CAD6-69D5-4082-B96C-91FC0DE5399D@gmail.com>
 <dc5ebc6f-6dde-499a-8d6b-c32172b84186@app.fastmail.com>
 <bacdfabb-a2a8-4379-ac97-adfc302d626a@app.fastmail.com>
 <294e1641-d658-4d43-8671-60e8ff860532@app.fastmail.com>
 <26d4dd6a-cfc5-4efc-9704-9cd3216ed712@app.fastmail.com>
 <7456ec96-7a9c-45a0-988e-ba1c7f9ec937@app.fastmail.com>
 <CAE7r3MJyitsHSd9A-y42Cmzs5gmG=zJH2pqMytSVzmzF4e_K9A@mail.gmail.com>
 <0253b822-e8fd-4067-ab24-23493c115a2a@app.fastmail.com>
 <CAE7r3MJXH8Phf1s5KKvh5zUG4=jnzeM+EnAx=KakvPd_djjrtw@mail.gmail.com>
 <a0215c3a-ffd0-41f1-900c-ddc74e1ffd6d@app.fastmail.com>
 <2eeea4f1-1b4f-430c-8571-544da04f08dc@app.fastmail.com>
 <dab234b5-a10c-4fb5-a2b1-ce725d3e3020@app.fastmail.com>
To: Joel Jacobson <joel@compiler.org>
Archived-At: 
 <https://www.postgresql.org/message-id/798A8F4A-DC41-4FE9-9BED-38E0DF2193F3%40gmail.com>
Precedence: bulk

Hi Joel,


> On Nov 16, 2025, at 05:53, Joel Jacobson <joel@compiler.org> wrote:
>=20
> The attached v28 is the same as v27, except some comments have been
> fixed to accurately reflect the code.
>=20
> =
/Joel<0001-optimize_listen_notify-v28.patch><0002-optimize_listen_notify-v=
28.patch>

Thanks for the continuous effort on this patch. Finally, I got some =
time, after revisiting v28 throughoutly, I think it=E2=80=99s much =
better now. Just got 2 more comments:

1
```
+	if (asyncQueueControl->channelHashDSH =3D=3D =
DSHASH_HANDLE_INVALID)
+	{
+		/* Initialize dynamic shared hash table for channel hash =
*/
+		channelDSA =3D =
dsa_create(LWTRANCHE_NOTIFY_CHANNEL_HASH);
+		dsa_pin(channelDSA);
+		dsa_pin_mapping(channelDSA);
+		channelHash =3D dshash_create(channelDSA, =
&channelDSHParams, NULL);
+
+		/* Store handles in shared memory for other backends to =
use */
+		asyncQueueControl->channelHashDSA =3D =
dsa_get_handle(channelDSA);
+		asyncQueueControl->channelHashDSH =3D
+			dshash_get_hash_table_handle(channelHash);
+	}
+	else if (!channelHash)
+	{
+		/* Attach to existing dynamic shared hash table */
+		channelDSA =3D =
dsa_attach(asyncQueueControl->channelHashDSA);
+		dsa_pin_mapping(channelDSA);
+		channelHash =3D dshash_attach(channelDSA, =
&channelDSHParams,
+									=
asyncQueueControl->channelHashDSH,
+									=
NULL);
+	}
```

DSA is created and pinned by the first backend and every backend =
isa_in_mapping, but I don=E2=80=99t see any unpin, is it a problem? If =
unpin is not needed, why are they provided?

2
```
+			entry =3D dshash_find(channelHash, &key, false);
+		}
+
+		if (entry =3D=3D NULL)
+			continue;			/* No listeners =
registered for this channel */
+
+		listeners =3D (ProcNumber *) dsa_get_address(channelDSA,
+										=
		   entry->listenersArray);
+
+		for (int j =3D 0; j < entry->numListeners; j++)
+		{
+			ProcNumber	i =3D listeners[j];
+			int32		pid;
+			QueuePosition pos;
+
+			if (QUEUE_BACKEND_WAKEUP_PENDING(i))
+				continue;
+
+			pos =3D QUEUE_BACKEND_POS(i);
+			pid =3D QUEUE_BACKEND_PID(i);
+
+			/* Skip if caught up */
 			if (QUEUE_POS_EQUAL(pos, QUEUE_HEAD))
 				continue;
+
+			Assert(pid !=3D InvalidPid);
+
+			QUEUE_BACKEND_WAKEUP_PENDING(i) =3D true;
+			pids[count] =3D pid;
+			procnos[count] =3D i;
+			count++;
 		}
-		else
+
+		dshash_release_lock(channelHash, entry);
```

SignalBackends() now holds the dshash entry lock for long time, while =
other backend=E2=80=99s LISTEN/UNLISTEN all needs to acquire the lock. =
So, my suggestion is to copy the listeners array to local then quickly =
release the lock.


Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/