From: Jack Ng <Jack.Ng@huawei.com>
To: Dmitry Dolgov <9erthalion6@gmail.com>
CC: Andres Freund <andres@anarazel.de>, Thom Brown <thom@linux.com>, "Ashutosh
 Bapat" <ashutosh.bapat.oss@gmail.com>, Tomas Vondra <tomas@vondra.me>,
	"Thomas Munro" <thomas.munro@gmail.com>, PostgreSQL-development
	<pgsql-hackers@postgresql.org>, Ni Ku <jakkuniku@gmail.com>
Subject: RE: Changing shared_buffers without restart
Thread-Topic: Changing shared_buffers without restart
Thread-Index: 
 AQHb4czjArbykOnCnUaEQCqOWcY4q7QfGxSAgAJTTACAAPSigIADCKUAgAAFeACAC1ixAIAArMiAgAA2XgCAAARBgIAACBkAgAAIcACAAAIegIAAOSQAgAADOQCAAAGrAIAAAZKAgAAGWACAAAVUAIAABgaAgAAEhICAAAj0AP//vbtQgABY5oCAAbaCAA==
Date: Tue, 15 Jul 2025 22:52:01 +0000
Message-ID: <ecf5907548a44c4c84d8916ccfe00a42@huawei.com>
References: <ndv2aa7z6qxzp7xehoqi4xz2sujfm4mgwictalr5ovtfhwv2pp@fjzdg5pj3r6t>
 <pdhm6tcvwhmnodv7rnmev2ikd2jy47f72rnygt6majew2at62o@6ac7byvkpiko>
 <afnt6ptmwx5zef46wmvxnqa2e57fwh57yg53met3hrbmsswavr@i663vegqqcal>
 <vslqe4duatd5hp4cw6eom3il5umpfrixylmpoh2ar27x3rstel@iengs2ijyflq>
 <eww3iiu2b64oay6qnz3f2kptccc3fxljm4bas2a2i2l5jhi4cd@kojmlzsq5uq2>
 <pkjfmei3j6yqmdi76cwzsmn4z34zg5yanb7rfv2rq4aik6ija3@xq6nrkvpya76>
 <fkm2z3idtem4raycfvxhuxwytxkugcqhjiryzub2hstjkbjele@cn2v5fzvvj2k>
 <y3dv2iuwh7tai5pztshiezdtffpfzt6t3nj4vbs6jmywhyy4zo@wxqvktyq5ix5>
 <1ABF6EF8-8556-4C1D-BA34-142DC4813CED@anarazel.de>
 <aac96524d79245b29522bc39db2c1262@huawei.com>
 <6r37jr5f5cfwve3tycgvmasxdfih3xv7jfua24ewt7ebnh4sn2@vzxf5ngdkgmb>
In-Reply-To: <6r37jr5f5cfwve3tycgvmasxdfih3xv7jfua24ewt7ebnh4sn2@vzxf5ngdkgmb>
Accept-Language: en-CA, zh-CN, en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Archived-At: 
 <https://www.postgresql.org/message-id/ecf5907548a44c4c84d8916ccfe00a42%40huawei.com>
Precedence: bulk

>> On Mon, Jul 14, 2025 at 03:18:10PM +0000, Jack Ng wrote:
>> Just brain-storming here... would moving NBuffers to shared memory solve
>this specific issue? Though I'm pretty sure that would open up a new set o=
f
>synchronization issues elsewhere, so I'm not sure if there's a net gain.
>
>It's in fact already happening, there is a shared structure that described=
 the
>resize status. But if I get everything right, it doesn't solve all the pro=
blems.

Hi Dmitry,=20

Just to clarify, you're not only referring to the ShmemControl::NSharedBuff=
ers
and related logic in the current patches, but actually getting rid of per-p=
rocess
NBuffers completely and use ShmemControl::NSharedBuffers everywhere instead=
 (or
something along those lines)? So that when the coordinator updates
ShmemControl::NSharedBuffers, everyone sees the new value right away.
I guess this is part of the "simplified design" you mentioned several posts=
 earlier?

I also thought about that approach more, and there seems to be new synchron=
ization
issues we would need to deal with, like:

1. Mid-execution change of NBuffers in functions like BufferSync and BgBuff=
erSync,
which could cause correctness and performance issues. I suppose most of the=
m
are solvable with atomics and shared r/w locks etc, but at the cost of high=
er
performance overheads.

2. NBuffers becomes inconsistent with the underlying shared memory mappings=
 for a
period of time for each process. Currently both are updated in AnonymousShm=
emResize
and AdjustShmemSize "atomically" for a process, so I wonder if letting them=
 get
out-of-sync (even for a brief period) could be problematic.

I agree it doesn't seem to solve all the problems. It can simplify certain =
aspects
of the design, but may also introduce new issues. Overall not a "silver bul=
let" :)

Jack