agora inbox for pgsql-hackers@postgresql.org
help / color / mirror / Atom feedFrom: Jack Ng <Jack.Ng@huawei.com>
To: Dmitry Dolgov <9erthalion6@gmail.com>
Cc: Andres Freund <andres@anarazel.de>
Cc: Thom Brown <thom@linux.com>
Cc: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Cc: Tomas Vondra <tomas@vondra.me>
Cc: Thomas Munro <thomas.munro@gmail.com>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Cc: Ni Ku <jakkuniku@gmail.com>
Subject: RE: Changing shared_buffers without restart
Date: Tue, 15 Jul 2025 22:52:01 +0000
Message-ID: <ecf5907548a44c4c84d8916ccfe00a42@huawei.com> (raw)
In-Reply-To: <6r37jr5f5cfwve3tycgvmasxdfih3xv7jfua24ewt7ebnh4sn2@vzxf5ngdkgmb>
References: <ndv2aa7z6qxzp7xehoqi4xz2sujfm4mgwictalr5ovtfhwv2pp@fjzdg5pj3r6t>
<pdhm6tcvwhmnodv7rnmev2ikd2jy47f72rnygt6majew2at62o@6ac7byvkpiko>
<afnt6ptmwx5zef46wmvxnqa2e57fwh57yg53met3hrbmsswavr@i663vegqqcal>
<vslqe4duatd5hp4cw6eom3il5umpfrixylmpoh2ar27x3rstel@iengs2ijyflq>
<eww3iiu2b64oay6qnz3f2kptccc3fxljm4bas2a2i2l5jhi4cd@kojmlzsq5uq2>
<pkjfmei3j6yqmdi76cwzsmn4z34zg5yanb7rfv2rq4aik6ija3@xq6nrkvpya76>
<fkm2z3idtem4raycfvxhuxwytxkugcqhjiryzub2hstjkbjele@cn2v5fzvvj2k>
<y3dv2iuwh7tai5pztshiezdtffpfzt6t3nj4vbs6jmywhyy4zo@wxqvktyq5ix5>
<1ABF6EF8-8556-4C1D-BA34-142DC4813CED@anarazel.de>
<aac96524d79245b29522bc39db2c1262@huawei.com>
<6r37jr5f5cfwve3tycgvmasxdfih3xv7jfua24ewt7ebnh4sn2@vzxf5ngdkgmb>
>> On Mon, Jul 14, 2025 at 03:18:10PM +0000, Jack Ng wrote:
>> Just brain-storming here... would moving NBuffers to shared memory solve
>this specific issue? Though I'm pretty sure that would open up a new set of
>synchronization issues elsewhere, so I'm not sure if there's a net gain.
>
>It's in fact already happening, there is a shared structure that described the
>resize status. But if I get everything right, it doesn't solve all the problems.
Hi Dmitry,
Just to clarify, you're not only referring to the ShmemControl::NSharedBuffers
and related logic in the current patches, but actually getting rid of per-process
NBuffers completely and use ShmemControl::NSharedBuffers everywhere instead (or
something along those lines)? So that when the coordinator updates
ShmemControl::NSharedBuffers, everyone sees the new value right away.
I guess this is part of the "simplified design" you mentioned several posts earlier?
I also thought about that approach more, and there seems to be new synchronization
issues we would need to deal with, like:
1. Mid-execution change of NBuffers in functions like BufferSync and BgBufferSync,
which could cause correctness and performance issues. I suppose most of them
are solvable with atomics and shared r/w locks etc, but at the cost of higher
performance overheads.
2. NBuffers becomes inconsistent with the underlying shared memory mappings for a
period of time for each process. Currently both are updated in AnonymousShmemResize
and AdjustShmemSize "atomically" for a process, so I wonder if letting them get
out-of-sync (even for a brief period) could be problematic.
I agree it doesn't seem to solve all the problems. It can simplify certain aspects
of the design, but may also introduce new issues. Overall not a "silver bullet" :)
Jack
view thread (167+ messages) latest in thread
Message-ID: <ecf5907548a44c4c84d8916ccfe00a42@huawei.com>
Permalink: ../ecf5907548a44c4c84d8916ccfe00a42@huawei.com/
Also on: postgresql.org/message-id/ecf5907548a44c4c84d8916ccfe00a42@huawei.com
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: Jack.Ng@huawei.com, 9erthalion6@gmail.com, andres@anarazel.de, thom@linux.com, ashutosh.bapat.oss@gmail.com, tomas@vondra.me, thomas.munro@gmail.com, jakkuniku@gmail.com
Subject: RE: Changing shared_buffers without restart
In-Reply-To: <ecf5907548a44c4c84d8916ccfe00a42@huawei.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox