From: Jack Ng <Jack.Ng@huawei.com>
To: Dmitry Dolgov <9erthalion6@gmail.com>
CC: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>,
	"pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>, Robert Haas
	<robertmhaas@gmail.com>, Ni Ku <jakkuniku@gmail.com>
Subject: RE: Changing shared_buffers without restart
Thread-Topic: Changing shared_buffers without restart
Thread-Index: AQHbl7DrFzUaSyTydEunWI983IaxOrNc1X6AgGMtZVaABTbmwIAAlRyAgAEig+A=
Date: Wed, 7 May 2025 05:34:37 +0000
Message-ID: <dcd396b2e89c4b83a34adaaec5838476@huawei.com>
References: 
 <CAExHW5soDB=08jeyqn-W-=+Y86v-a5f_NJRjAdV9_yFrKpJcCw@mail.gmail.com>
 <eqs6v4rsboazl67xz3wxc6xjkgrpfybitpl45y3lmb2br67wbj@o7czebb3rlgd>
 <CAExHW5sTB1Ow0a1JtVy4PGCPckM7P1CSngJ4+LX3w+yz7rwQfQ@mail.gmail.com>
 <ywyt5gecsfjzxlvxbvingd2dps2dedpwq5l4xhfmbcci6xta2e@h7ygoxvep725>
 <CAExHW5s1b-_9GFGGOxSVimrt8oe2Y0a7vRhENrd2BSo+0Tkp-A@mail.gmail.com>
 <gzb2oqqabzqidlnumrbau2k2fscb5jo6nlpdjnf6pu5y4bk5mo@xoujqgtiqvgr>
 <CAExHW5uGozdiDm_BefS7jpQzaisCs4_nLTMF0+S_Pds49pRG_w@mail.gmail.com>
 <CAPuPUJwDa_WEVmkknyYHYub5qvBuwKyLLOOiUpyiq6Wd=9FQ_g@mail.gmail.com>
 <a2b6laawu4iya4zu737lhgdbvi5bcspxkxqinzogw6o7e2yhke@ay4qtrdlyzhq>
 <fbdc6674ae54420d96368079bf76dd74@huawei.com>
 <urny6igvtdqmbumsjmnpvw2vkynbjsxcyunov57gzp4bsikrh4@3pmdg245fkbe>
In-Reply-To: <urny6igvtdqmbumsjmnpvw2vkynbjsxcyunov57gzp4bsikrh4@3pmdg245fkbe>
Accept-Language: en-CA, zh-CN, en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Archived-At: 
 <https://www.postgresql.org/message-id/dcd396b2e89c4b83a34adaaec5838476%40huawei.com>
Precedence: bulk

> all the possible scenarios. But now I'm reworking it along the lines sugg=
ested
> by Thomas, and will address those as well. Thanks!

Thanks for the info, Dmitry.
Just want to confirm my understanding of Thomas' suggestion and your discus=
sions... I think the simpler and more portable solution goes something like=
 the following?=20

* For each BP resource segment (main, desc, buffers, etc):
    1. create an anonymous file as backing
    2. mmap a large reserved shared memory area with PROTO_READ/WRITE + MAP=
_NORESERVE using the anon fd
    3. use ftruncate to back the in-use region (and maybe posix_fallocate t=
oo to avoid SIGBUS on alloc failure during first-touch), but no need to cre=
ate a memory mapping for it
    4. also no need to create a separate mapping for the reserved region (a=
lready covered by the mapping created in 2.)

|-- Memory mapping (MAP_NORESERVE) for BUFFER --|
|-- In-use region --|----- Reserved region -----|

* During resize, simply calculate the new size and call ftruncate on each s=
egment to adjust memory accordingly, no need to mmap/munmap or modify any m=
emory mapping.

I tried this approach with a test program (with huge pages), and both expan=
d and shrink seem to work as expected --for shrink, the memory is freed rig=
ht after the resize ftruncate.

Regards,

Jack Ng