Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1v6mC2-009Xwi-Mm for pgsql-hackers@arkaria.postgresql.org; Thu, 09 Oct 2025 08:40:10 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1v6mBz-001xdU-QT for pgsql-hackers@arkaria.postgresql.org; Thu, 09 Oct 2025 08:40:08 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1v6mBz-001xdL-Ff for pgsql-hackers@lists.postgresql.org; Thu, 09 Oct 2025 08:40:08 +0000 Received: from mail-pj1-x1032.google.com ([2607:f8b0:4864:20::1032]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1v6mBx-001CFR-2z for pgsql-hackers@postgresql.org; Thu, 09 Oct 2025 08:40:08 +0000 Received: by mail-pj1-x1032.google.com with SMTP id 98e67ed59e1d1-3306d93e562so801539a91.1 for ; Thu, 09 Oct 2025 01:40:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759999199; x=1760603999; darn=postgresql.org; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:from:to:cc:subject:date:message-id:reply-to; bh=S7kqCngpxy8xwTGonEDTkQMolZIk2DRlSchwfOnSyh8=; b=XM0ijx0bU4MryOME9YFpd4PluVCqFUnKw/+T24XE8lrwu6u4fJR4FwlJ8KO+PCIG6z f18uoCh7OIXvPyIOlDxRzcNyRS4zqDMjYYf64JrkeW06VV/+QHdg/KW7ulhWYNzxDHGH lkbh8srg5cqPY3mRXGi3hURpUnXMY/5H8l1YM2tFImXrLo21Kic8ejGBBLsHjfftos4f aIgyDm26h96+/AXrOzqGikGMkyPLtt5SYT7pHANVWbOxpr08R4u9UQE70/ktx55c4Z2W 4IA6Zurwn1JscsdxdQIvwMY9c40r6AXKb64nU932zKanK8mwiPlPWi9hshYi5H4ioQ8v 5xEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759999199; x=1760603999; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=S7kqCngpxy8xwTGonEDTkQMolZIk2DRlSchwfOnSyh8=; b=ok0skozdSXAXJFTiUfLOgEvxPTtS/wngf7gdQRjwG8vAZC/hAr+rqeCGkqjYL0gi4p Svvy6Xxi6/lQYMnQpJncHSNnqycWykqGPQa9G6yQ43YOmQLLdwXk6vBjdlpUX2tRapg5 b5PViYnLCllQL1moHCajs+rGArVIB7MCLeHiMyKHJ3eIvaSiASGrJQWqF3Hhr8HbLADR WCgHq4jBOkK++HjMPMJzorjqAP0e5aAu8AgXCSYrrJ+vD+xCsQ+WbLRfjezaIff9TUaD OVEbUHLK/loZ/gaefROTPg4rLTG+YMKhDkNcyhioy3NpTKg9oooDDuBXnbVJlYox44FT azug== X-Forwarded-Encrypted: i=1; AJvYcCV98Mc8sRdQNc//eJpym51Ne4Qiju0qE4GU4fNYaj9nq10VH3ds2mpuy0dw6r6Ld8cyaL5FJ6gROErw9sJV@postgresql.org X-Gm-Message-State: AOJu0Yy3r+mfiO46STOLv0H5ss02rwEi38S7vwbq4dXNQ8HUmRhXMk9b 2yQESvozJecDZjGdNxFxl6bgz6k7zVbkyMp0J/CPTxMMod2OO4eMZQQ/ X-Gm-Gg: ASbGncs3NhPH4Iio5XGpqAibVi+b9s+Yx9muqRwwM2TzL7G8vCfcLFHUtjAYwQm/Zh9 AkKvQhGZyBxsoyon2N0rpiZa6A9ErN55AHKtedaBW1LRVLMqTZLO+LS2MAHxputAHBMqwq1J5Md YsVIHDPejEc7Ze08WrEd3Ls4matuvWX5etolVVHPc3f//+sK3skB268rMzatlRUNdxquaXAX+WZ FZiRwOn/qkeDQYSVW9IEimWflrmvTNvB0PvOuCG4ENWcz0WZRzFZCQE4bLC2ZwTCqE9skHv9txE /dUIREZFgxqXKEdTG9oRkFhZ8mriM+OjvUkhBCmX6RWMLqywi0896OBrOVpDHftr833Zl0X89n2 IT+jfFz4rWVREYPJwNq26Uw+Hq2KuNSSI1j/N7b0H9Bpwg6mpXMno1JdsykzehwaI30Na X-Google-Smtp-Source: AGHT+IHlm4NKCXzLeyFhX0zlDc2uQoXv6YvrjmXfAchoDMfkAcBi7ZWDKyaEhvY88IcD8ReyBiqaDQ== X-Received: by 2002:a17:90b:4ad2:b0:338:26e3:ffb6 with SMTP id 98e67ed59e1d1-33b513ceddemr10048526a91.26.1759999198907; Thu, 09 Oct 2025 01:39:58 -0700 (PDT) Received: from smtpclient.apple ([170.178.170.211]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-33b510ff683sm6208733a91.8.2025.10.09.01.39.56 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Oct 2025 01:39:58 -0700 (PDT) From: Chao Li Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_2BF8AF4A-18D8-4835-A583-8BDF48A0E1F6" Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.700.81\)) Subject: Re: Optimize LISTEN/NOTIFY Date: Thu, 9 Oct 2025 16:39:43 +0800 In-Reply-To: <1db48c52-aa22-4e9a-a9ec-e80ad0129753@app.fastmail.com> Cc: Tom Lane , Matheus Alcantara , pgsql-hackers To: Joel Jacobson References: <6899c044-4a82-49be-8117-e6f669765f7e@app.fastmail.com> <165530.1752362320@sss.pgh.pa.us> <02a7cd37-e2fc-4212-8b19-f8c239c95fb8@app.fastmail.com> <96f00bf1-cc9d-4520-9d02-9e14e7767c88@app.fastmail.com> <30c2aa7d-dd6c-4b68-a2e4-f217a1a34acf@app.fastmail.com> <0b4d402a-9ac2-4aa8-acf8-8231dbe579ea@app.fastmail.com> <3095599.1758644879@sss.pgh.pa.us> <0dc6a2cc-5216-4dc1-9dd2-430cafc6095b@app.fastmail.com> <52CC167F-763B-4ECA-B0B4-DAB381816828@gmail.com> <9186C6D0-F7A9-482A-9183-89E530B57E36@gmail.com> <1073593.1759423179@sss.pgh.pa.us> <4bd5e6c4-6fa7-44bb-869d-59a32a331fa8@app.fastmail.com> <85828f29-e72e-4400-94f3-9a69bc8dc239@app.fastmail.com> <2495353.1759860890@sss.pgh.pa.us> <8aeae418-92a6-4bbd-9c06-9574c79e59f7@app.fastmail.com> <2790345f-03c3-4cae-8f14-886ed9079319@app.fastmail.com> <1db48c52-aa22-4e9a-a9ec-e80ad0129753@app.fastmail.com> X-Mailer: Apple Mail (2.3826.700.81) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --Apple-Mail=_2BF8AF4A-18D8-4835-A583-8BDF48A0E1F6 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Oct 9, 2025, at 16:07, Joel Jacobson wrote: >=20 >>>> ``` >>>> /* >>>> @@ -1865,6 +2087,7 @@ asyncQueueReadAllNotifications(void) >>>> LWLockAcquire(NotifyQueueLock, LW_SHARED); >>>> /* Assert checks that we have a valid state entry */ >>>> Assert(MyProcPid =3D=3D QUEUE_BACKEND_PID(MyProcNumber)); >>>> + QUEUE_BACKEND_WAKEUP_PENDING(MyProcNumber) =3D false; >>>> ``` >>>>=20 >>>> This piece of code originally only read the shared memory, so it = can=20 >>>> use LW_SHARED lock mode, but now it writes to the shared memory, do = we=20 >>>> need to change the lock mode to =E2=80=9Cexclusive=E2=80=9D? >>>=20 >>> No, LW_SHARED is sufficient here, since the backend only modifies = its own state, >>> and no other backend could do that, without holding an exclusive = lock. >>=20 >> Yes, the backend only modifies its own state to =E2=80=9Cfalse=E2=80=9D= , but other=20 >> backends may set its state to =E2=80=9Ctrue=E2=80=9D, that is a race = condition. So I=20 >> still think an exclusive lock is needed. >=20 > No, other backends cannot alter our state without holding an exclusive = lock, > and they cannot obtain an exclusive lock on our backend until we've = released > the shared lock we're holding. >=20 Ah=E2=80=A6 That=E2=80=99s true. This comment is resolved. >>>>=20 >>=20 >> The hash function channel_hash_func() is defined by your own code, it=20= >> use strnlen() to get length of channel name, so that bytes after = =E2=80=98\0=E2=80=99=20 >> won=E2=80=99t be used. >=20 > No, the hash function is not used for comparison. > We're using the default dshash_memcmp for comparison: >=20 > ``` > /* parameters for the channel hash table */ > static const dshash_parameters channelDSHParams =3D { > sizeof(ChannelHashKey), > sizeof(ChannelEntry), > dshash_memcmp, > channelHashFunc, > dshash_memcpy, > LWTRANCHE_NOTIFY_CHANNEL_HASH > }; > ``` >=20 > Looking at its implementation, we can see it's using memcmp under the = hood: >=20 > ``` > /* > * A compare function that forwards to memcmp. > */ > int > dshash_memcmp(const void *a, const void *b, size_t size, void *arg) > { > return memcmp(a, b, size); > } > ``` >=20 > Here, the input parameter `size` comes from `sizeof(ChannelHashKey)`, > so it will include all bytes in the comparison. >=20 Okay, I think I misunderstood hash_function. So, this comment is also = resolved. I am thinking loudly. When a hash key is created, it has been memset to = 0, meaning that in key->channel, all bytes after =E2=80=98\0=E2=80=99 = are also 0, there should not be any random bytes in hash key, so that in = channelHashFunc(), we don=E2=80=99t need to to use strnlen() anymore, = which improves performance a little bit. Like this: h =3D DatumGetUInt32(hash_uint32(k->dboid)); h ^=3D DatumGetUInt32(hash_any((const unsigned char *) k->channel, sizeof(k->channel))); Best regards, -- Chao Li (Evan) HighGo Software Co., Ltd. https://www.highgo.com/ --Apple-Mail=_2BF8AF4A-18D8-4835-A583-8BDF48A0E1F6 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8

On Oct 9, 2025, at 16:07, Joel Jacobson = <joel@compiler.org> wrote:

```
/*
@@ -1865,6 +2087,7 @@ = asyncQueueReadAllNotifications(void)
LWLockAcquire(NotifyQueueLock, = LW_SHARED);
/* Assert checks that we have a valid state entry = */
Assert(MyProcPid =3D=3D QUEUE_BACKEND_PID(MyProcNumber));
+ = QUEUE_BACKEND_WAKEUP_PENDING(MyProcNumber) =3D false;
```

This = piece of code originally only read the shared memory, so it can
use = LW_SHARED lock mode, but now it writes to the shared memory, do we =
need to change the lock mode to = =E2=80=9Cexclusive=E2=80=9D?

No, LW_SHARED is = sufficient here, since the backend only modifies its own state,
and = no other backend could do that, without holding an exclusive = lock.

Yes, the backend only modifies its own state = to =E2=80=9Cfalse=E2=80=9D, but other
backends may set its state to = =E2=80=9Ctrue=E2=80=9D, that is a race condition. So I
still think = an exclusive lock is needed.

No, other backends = cannot alter our state without holding an exclusive lock,
and they = cannot obtain an exclusive lock on our backend until we've = released
the shared lock we're = holding.


Ah=E2=80=A6 = That=E2=80=99s true. This comment is = resolved.




The hash function = channel_hash_func() is defined by your own code, it
use strnlen() to = get length of channel name, so that bytes after =E2=80=98\0=E2=80=99 =
won=E2=80=99t be used.

No, the hash function is = not used for comparison.
We're using the default dshash_memcmp for = comparison:

```
/* parameters for the channel hash table = */
static const dshash_parameters channelDSHParams =3D {
= sizeof(ChannelHashKey),
sizeof(ChannelEntry),
= dshash_memcmp,
channelHashFunc,
= dshash_memcpy,
= LWTRANCHE_NOTIFY_CHANNEL_HASH
};
```

Looking at its = implementation, we can see it's using memcmp under the = hood:

```
/*
* A compare function that forwards to = memcmp.
*/
int
dshash_memcmp(const void *a, const void *b, = size_t size, void *arg)
{
return memcmp(a, b, = size);
}
```

Here, the input parameter `size` comes from = `sizeof(ChannelHashKey)`,
so it will include all bytes in the = comparison.


Okay= , I think I misunderstood hash_function. So, this comment is also = resolved.

I am thinking loudly. When a hash key = is created, it has been memset to 0, meaning that in key->channel, = all bytes after =E2=80=98\0=E2=80=99 are also 0, there should not be any = random bytes in hash key, so that in channelHashFunc(), we don=E2=80=99t = need to to use strnlen() anymore, which improves performance a little = bit. Like this:

h =3D DatumGetUInt32(hash_uint32(k->dboid));
h ^=3D DatumGetUInt32(hash_any((const = unsigned char *) k->channel,
sizeof(k->channel)));

Best = regards,
--
Chao Li (Evan)
HighGo Software = Co., Ltd.
https://www.highgo.com/




= --Apple-Mail=_2BF8AF4A-18D8-4835-A583-8BDF48A0E1F6--