Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qzMwB-00Fn62-Vu for pgsql-hackers@arkaria.postgresql.org; Sat, 04 Nov 2023 20:08:08 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1qzMw8-007BmZ-Qi for pgsql-hackers@arkaria.postgresql.org; Sat, 04 Nov 2023 20:08:04 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qzMw8-007BmR-6E for pgsql-hackers@lists.postgresql.org; Sat, 04 Nov 2023 20:08:04 +0000 Received: from forwardcorp1c.mail.yandex.net ([2a02:6b8:c03:500:1:45:d181:df01]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qzMw3-003yDS-6I for pgsql-hackers@lists.postgresql.org; Sat, 04 Nov 2023 20:08:02 +0000 Received: from mail-nwsmtp-smtp-corp-canary-81.sas.yp-c.yandex.net (mail-nwsmtp-smtp-corp-canary-81.sas.yp-c.yandex.net [IPv6:2a02:6b8:c08:ba1:0:640:375a:0]) by forwardcorp1c.mail.yandex.net (Yandex) with ESMTP id 41F0D6155B; Sat, 4 Nov 2023 23:07:54 +0300 (MSK) Received: from smtpclient.apple (unknown [2a02:6b8:b081:7218::1:3a]) by mail-nwsmtp-smtp-corp-canary-81.sas.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id q7gvKX4OiGk0-I1sbWPw8; Sat, 04 Nov 2023 23:07:53 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1699128473; bh=Goyc/u4ylYdu1pF0bog0ZS/YBGeYDi5SHupftk1h1Cg=; h=References:To:Cc:In-Reply-To:Date:From:Message-Id:Subject; b=kwxuYQeP7fkwuduJQr8PW4eyYtKFbge6MSngxFyD+6rGfsK+Cc9s6Tyg0wIhbt5b7 TGnqXpzOdaFC4nuVKoxnYIc9HovYlL1MnZdFo4a4i+Je++LEL+FBFrvoAPxj53eT2v zKULq2bWLAZ9x7aypaga6B90m/Uwl6wdNua6XyIc= Authentication-Results: mail-nwsmtp-smtp-corp-canary-81.sas.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: "Andrey M. Borodin" Message-Id: <96C73B61-70B4-4E56-BFC7-4FBAE45DED51@yandex-team.ru> Content-Type: multipart/alternative; boundary="Apple-Mail=_7DC27B21-06BB-4EA4-B644-86B9AD328A45" Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.4\)) Subject: Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock Date: Sat, 4 Nov 2023 23:07:52 +0300 In-Reply-To: Cc: Alvaro Herrera , PostgreSQL Hackers To: Dilip Kumar References: <202310241604.bvjwhasbhjg7@alvherre.pgsql> X-Mailer: Apple Mail (2.3696.120.41.1.4) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --Apple-Mail=_7DC27B21-06BB-4EA4-B644-86B9AD328A45 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On 30 Oct 2023, at 09:20, Dilip Kumar wrote: >=20 > changed the logic of SlruAdjustNSlots() in 0002, such that now it > starts with the next power of 2 value of the configured slots and > keeps doubling the number of banks until we reach the number of banks > to the max SLRU_MAX_BANKS(128) and bank size is bigger than > SLRU_MIN_BANK_SIZE (8). By doing so, we will ensure we don't have too > many banks There was nothing wrong with having too many banks. Until bank-wise = locks and counters were added in later patchsets. Having hashtable to find SLRU page in the buffer IMV is too slow. Some = comments on this approach can be found here [0]. I'm OK with having HTAB for that if we are sure performance does not = degrade significantly, but I really doubt this is the case. I even think SLRU buffers used HTAB in some ancient times, but I could = not find commit when it was changed to linear search. Maybe we could decouple locks and counters from SLRU banks? Banks were = meant to be small to exploit performance of local linear search. Lock = partitions have to be bigger for sure. > On 30 Oct 2023, at 09:20, Dilip Kumar wrote: >=20 > I have taken 0001 and 0002 from [1], done some bug fixes in 0001 BTW can you please describe in more detail what kind of bugs? Thanks for working on this! Best regards, Andrey Borodin. [0] = https://www.postgresql.org/message-id/flat/CA%2BhUKGKVqrxOp82zER1%3DXN%3Dy= PwV_-OCGAg%3Dez%3D1iz9rG%2BA7Smw%40mail.gmail.com#b60f1cb73d350cf686338d4e= 800e12a2= --Apple-Mail=_7DC27B21-06BB-4EA4-B644-86B9AD328A45 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii

On 30 Oct 2023, at 09:20, Dilip Kumar <dilipbalaut@gmail.com> wrote:

changed the logic of SlruAdjustNSlots() in 0002, such that = now it
starts with = the next power of 2 value of the configured slots and
keeps doubling the number of = banks until we reach the number of banks
to the max SLRU_MAX_BANKS(128) and bank size is bigger = than
SLRU_MIN_BANK_SIZE (8).  By doing so, we will ensure we = don't have too
many banks
There was nothing = wrong with having too many banks. Until bank-wise locks and counters = were added in later patchsets.
Having hashtable to find = SLRU page in the buffer IMV is too slow. Some comments on this approach = can be found here [0].
I'm OK with having HTAB = for that if we are sure performance does not degrade significantly, but = I really doubt this is the case.
I even = think SLRU buffers used HTAB in some ancient times, but I could not find = commit when it was changed to linear search.

Maybe we could decouple locks and = counters from SLRU banks? Banks were meant to be small to exploit = performance of local linear search. Lock partitions have to be bigger = for sure.



On 30 Oct 2023, at 09:20, Dilip Kumar <dilipbalaut@gmail.com> wrote:

I have taken 0001 and = 0002 from [1], done some bug fixes in = 0001

BTW can you please = describe in more detail what kind of bugs?


Thanks for working on this!


Best= regards, Andrey Borodin.


[0] https://www.postgresql.org/message-id/flat/CA%2BhUKGKVqrxOp82zE= R1%3DXN%3DyPwV_-OCGAg%3Dez%3D1iz9rG%2BA7Smw%40mail.gmail.com#b60f1cb73d350= cf686338d4e800e12a2
= --Apple-Mail=_7DC27B21-06BB-4EA4-B644-86B9AD328A45--