agora inbox for pgsql-hackers@postgresql.org
help / color / mirror / Atom feedFrom: Jack Ng <Jack.Ng@huawei.com>
To: Dmitry Dolgov <9erthalion6@gmail.com>
Cc: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Cc: pgsql-hackers@postgresql.org <pgsql-hackers@postgresql.org>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Ni Ku <jakkuniku@gmail.com>
Subject: RE: Changing shared_buffers without restart
Date: Wed, 7 May 2025 05:34:37 +0000
Message-ID: <dcd396b2e89c4b83a34adaaec5838476@huawei.com> (raw)
In-Reply-To: <urny6igvtdqmbumsjmnpvw2vkynbjsxcyunov57gzp4bsikrh4@3pmdg245fkbe>
References: <CAExHW5soDB=08jeyqn-W-=+Y86v-a5f_NJRjAdV9_yFrKpJcCw@mail.gmail.com>
<eqs6v4rsboazl67xz3wxc6xjkgrpfybitpl45y3lmb2br67wbj@o7czebb3rlgd>
<CAExHW5sTB1Ow0a1JtVy4PGCPckM7P1CSngJ4+LX3w+yz7rwQfQ@mail.gmail.com>
<ywyt5gecsfjzxlvxbvingd2dps2dedpwq5l4xhfmbcci6xta2e@h7ygoxvep725>
<CAExHW5s1b-_9GFGGOxSVimrt8oe2Y0a7vRhENrd2BSo+0Tkp-A@mail.gmail.com>
<gzb2oqqabzqidlnumrbau2k2fscb5jo6nlpdjnf6pu5y4bk5mo@xoujqgtiqvgr>
<CAExHW5uGozdiDm_BefS7jpQzaisCs4_nLTMF0+S_Pds49pRG_w@mail.gmail.com>
<CAPuPUJwDa_WEVmkknyYHYub5qvBuwKyLLOOiUpyiq6Wd=9FQ_g@mail.gmail.com>
<a2b6laawu4iya4zu737lhgdbvi5bcspxkxqinzogw6o7e2yhke@ay4qtrdlyzhq>
<fbdc6674ae54420d96368079bf76dd74@huawei.com>
<urny6igvtdqmbumsjmnpvw2vkynbjsxcyunov57gzp4bsikrh4@3pmdg245fkbe>
> all the possible scenarios. But now I'm reworking it along the lines suggested
> by Thomas, and will address those as well. Thanks!
Thanks for the info, Dmitry.
Just want to confirm my understanding of Thomas' suggestion and your discussions... I think the simpler and more portable solution goes something like the following?
* For each BP resource segment (main, desc, buffers, etc):
1. create an anonymous file as backing
2. mmap a large reserved shared memory area with PROTO_READ/WRITE + MAP_NORESERVE using the anon fd
3. use ftruncate to back the in-use region (and maybe posix_fallocate too to avoid SIGBUS on alloc failure during first-touch), but no need to create a memory mapping for it
4. also no need to create a separate mapping for the reserved region (already covered by the mapping created in 2.)
|-- Memory mapping (MAP_NORESERVE) for BUFFER --|
|-- In-use region --|----- Reserved region -----|
* During resize, simply calculate the new size and call ftruncate on each segment to adjust memory accordingly, no need to mmap/munmap or modify any memory mapping.
I tried this approach with a test program (with huge pages), and both expand and shrink seem to work as expected --for shrink, the memory is freed right after the resize ftruncate.
Regards,
Jack Ng
view thread (167+ messages) latest in thread
Message-ID: <dcd396b2e89c4b83a34adaaec5838476@huawei.com>
Permalink: ../dcd396b2e89c4b83a34adaaec5838476@huawei.com/
Also on: postgresql.org/message-id/dcd396b2e89c4b83a34adaaec5838476@huawei.com
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: Jack.Ng@huawei.com, 9erthalion6@gmail.com, ashutosh.bapat.oss@gmail.com, robertmhaas@gmail.com, jakkuniku@gmail.com
Subject: RE: Changing shared_buffers without restart
In-Reply-To: <dcd396b2e89c4b83a34adaaec5838476@huawei.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox