From: Nathan Bossart <nathandbossart@gmail.com>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: Dilip Kumar <dilipbalaut@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock
Date: Tue, 24 Oct 2023 12:04:51 -0500
Message-ID: <20231024170451.GC871220@nathanxps13> (raw)
In-Reply-To: <202310241604.bvjwhasbhjg7@alvherre.pgsql>
References: <CAFiTN-taYmir2_g+aWeEVQopvLB5K-0316KAFsBEsXx5OiaM9A@mail.gmail.com>
<202310241604.bvjwhasbhjg7@alvherre.pgsql>
On Tue, Oct 24, 2023 at 06:04:13PM +0200, Alvaro Herrera wrote:
> Like everybody else, I like having less GUCs to configure, but going
> this far to avoid them looks rather disastrous to me. IMO we should
> just use Munro's older patches that gave one GUC per SLRU, and users
> only need to increase the one that shows up in pg_wait_event sampling.
> Someday we will get the (much more complicated) patches to move these
> buffers to steal memory from shared buffers, and that'll hopefully let
> use get rid of all this complexity.
+1
> On the other hand, here's a somewhat crazy idea. What if, instead of
> stealing buffers from shared_buffers (which causes a lot of complexity),
> we allocate a common pool for all SLRUs to use? We provide a single
> knob -- say, non_relational_buffers=32MB as default -- and we use a LRU
> algorithm (or something) to distribute that memory across all the SLRUs.
> So the ratio to use for this SLRU or that one would depend on the nature
> of the workload: maybe more for multixact in this server here, but more
> for subtrans in that server there; it's just the total amount that the
> user would have to configure, side by side with shared_buffers (and
> perhaps scale with it like wal_buffers), and the LRU would handle the
> rest. The "only" problem here is finding a distribution algorithm that
> doesn't further degrade performance, of course ...
I think it's worth a try. It does seem simpler, and it might allow us to
sidestep some concerns about scaling when the SLRU pages are in
shared_buffers [0].
[0] https://postgr.es/m/ZPsaEGRvllitxB3v%40tamriel.snowman.net
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: nathandbossart@gmail.com, alvherre@alvh.no-ip.org, dilipbalaut@gmail.com, pgsql-hackers@lists.postgresql.org
Subject: Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock
In-Reply-To: <20231024170451.GC871220@nathanxps13>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox