agora inbox for pgsql-hackers@postgresql.org
help / color / mirror / Atom feedFrom: Jack Ng <Jack.Ng@huawei.com>
To: Andres Freund <andres@anarazel.de>
To: Dmitry Dolgov <9erthalion6@gmail.com>
Cc: Thom Brown <thom@linux.com>
Cc: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Cc: Tomas Vondra <tomas@vondra.me>
Cc: Thomas Munro <thomas.munro@gmail.com>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Cc: Ni Ku <jakkuniku@gmail.com>
Subject: RE: Changing shared_buffers without restart
Date: Mon, 14 Jul 2025 15:10:50 +0000
Message-ID: <39b5afc524174c8a8fa6ef058676a596@huawei.com> (raw)
In-Reply-To: <fkm2z3idtem4raycfvxhuxwytxkugcqhjiryzub2hstjkbjele@cn2v5fzvvj2k>
References: <CAExHW5tztw-ApbcsAHv+=vTZTO6w9hC+Q7OtUEWWfEgvDFHWxA@mail.gmail.com>
<nu3pggzvuqomroda5cliicehtuam4kd4sxfm27d5rqpejnqutf@micq2wwi66jc>
<CAA-aLv7p=9jCy_-67+Wj2vrRL1QCV-X0ZFZmpAxBJqLPp-ho+A@mail.gmail.com>
<s2eqkawv2m3knhtr32cut6zvyebas25cszpnmlh6amjw6fjjzf@jyzij4mrquqa>
<ndv2aa7z6qxzp7xehoqi4xz2sujfm4mgwictalr5ovtfhwv2pp@fjzdg5pj3r6t>
<pdhm6tcvwhmnodv7rnmev2ikd2jy47f72rnygt6majew2at62o@6ac7byvkpiko>
<afnt6ptmwx5zef46wmvxnqa2e57fwh57yg53met3hrbmsswavr@i663vegqqcal>
<vslqe4duatd5hp4cw6eom3il5umpfrixylmpoh2ar27x3rstel@iengs2ijyflq>
<eww3iiu2b64oay6qnz3f2kptccc3fxljm4bas2a2i2l5jhi4cd@kojmlzsq5uq2>
<pkjfmei3j6yqmdi76cwzsmn4z34zg5yanb7rfv2rq4aik6ija3@xq6nrkvpya76>
<fkm2z3idtem4raycfvxhuxwytxkugcqhjiryzub2hstjkbjele@cn2v5fzvvj2k>
If I understanding correctly, putting a new buffer in the freelist before updating NBuffers could break existing logic that calls BufferIsValid(bufnum) and asserts bufnum <= NBuffers? (since a backend can grab the new buffer and checks its validity before the coordinator can add it to the freelist.)
But it seems updating NBuffers before adding new elements to the freelist could be problematic too? Like if a new buffer is already chosen as a victim and then the coordinator adds it to the freelist, would that lead to "double-use"? (seems possible at least with current logic and serialization in StrategyGetBuffer). If that's a valid concern, would something like this work?
1) initialize buffer headers, with a new state/flag to indicate "add-pending"
2) update NBuffers
-- add a check in clock-sweep logic for "add-pending" and skip them
3) put them onto the freelist
4) when a new element is grabbed from freelist, check for and reset add-pending flag.
This ensure the new element is always obtained from the freelist first I think.
Jack
>-----Original Message-----
>From: Andres Freund <andres@anarazel.de>
>Sent: Monday, July 14, 2025 10:23 AM
>To: Dmitry Dolgov <9erthalion6@gmail.com>
>Cc: Thom Brown <thom@linux.com>; Ashutosh Bapat
><ashutosh.bapat.oss@gmail.com>; Tomas Vondra <tomas@vondra.me>;
>Thomas Munro <thomas.munro@gmail.com>; PostgreSQL-development <pgsql-
>hackers@postgresql.org>; Jack Ng <Jack.Ng@huawei.com>; Ni Ku
><jakkuniku@gmail.com>
>Subject: Re: Changing shared_buffers without restart
>
>Hi,
>
>On 2025-07-14 16:01:50 +0200, Dmitry Dolgov wrote:
>> > On Mon, Jul 14, 2025 at 09:42:46AM -0400, Andres Freund wrote:
>> > What on earth would be the point of putting a buffer on the freelist
>> > but not make it reachable by the clock sweep? To me that's just nonsensical.
>>
>> To clarify, we're not talking about this scenario as "that's how it
>> would work after the resize". The point is that to expand shared
>> buffers they need to be initialized, included into the whole buffer
>> machinery (freelist, clock sweep, etc.) and NBuffers has to be updated.
>
>It seems pretty obvious to that the order has to be
>
>1) initialize buffer headers
>2) update NBuffers
>3) put them onto the freelist
>
>(with 3) hopefully becoming obsolete)
>
>
>> Those steps are separated in time, and I'm currently trying to
>> understand what are the consequences of performing them in different
>> order and whether there are possible concurrency issues under various
>> scenarios. Does this make more sense, or still not?
>
>I still don't understand why it'd ever make sense to put a buffer onto the freelist
>before updating NBuffers first.
>
>Greetings,
>
>Andres Freund
view thread (167+ messages) latest in thread
Message-ID: <39b5afc524174c8a8fa6ef058676a596@huawei.com>
Permalink: ../39b5afc524174c8a8fa6ef058676a596@huawei.com/
Also on: postgresql.org/message-id/39b5afc524174c8a8fa6ef058676a596@huawei.com
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: Jack.Ng@huawei.com, andres@anarazel.de, 9erthalion6@gmail.com, thom@linux.com, ashutosh.bapat.oss@gmail.com, tomas@vondra.me, thomas.munro@gmail.com, jakkuniku@gmail.com
Subject: RE: Changing shared_buffers without restart
In-Reply-To: <39b5afc524174c8a8fa6ef058676a596@huawei.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox