From: Andres Freund <andres@anarazel.de>
To: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Cc: Heikki Linnakangas <hlinnaka@iki.fi>
Cc: Tomas Vondra <tomas@vondra.me>
Cc: Peter Eisentraut <peter@eisentraut.org>
Cc: Thomas Munro <thomas.munro@gmail.com>
Cc: Dmitry Dolgov <9erthalion6@gmail.com>
Cc: pgsql-hackers@postgresql.org, Robert Haas <robertmhaas@gmail.com>
Cc: chaturvedipalak1911@gmail.com
Subject: Re: Changing shared_buffers without restart
Date: Thu, 12 Feb 2026 15:12:50 -0500
Message-ID: <aY4v1oSmokXNpQMX@alap3.anarazel.de> (raw)
In-Reply-To: <CAExHW5ui9-aKtS-vwUYMQ4uSmi54cvLpvqrf29B8L69ZJa766g@mail.gmail.com>
References: <CAExHW5sYg_d4O7oGRqbomnVODeqR3YNAeYAa526n1dsWCM=+Fg@mail.gmail.com>
<CAExHW5vB8sAmDtkEN5dcYYeBok3D8eAzMFCOH1k+krxht1yFjA@mail.gmail.com>
<qltuzcdxapofdtb5mrd4em3bzu2qiwhp3cdwdsosmn7rhrtn4u@yaogvphfwc4h>
<CAExHW5sOu8+9h6t7jsA5jVcQ--N-LCtjkPnCw+rpoN0ovT6PHg@mail.gmail.com>
<CAExHW5sVxEwQsuzkgjjJQP9-XVe0H2njEVw1HxeYFdT7u7J+eQ@mail.gmail.com>
<CAExHW5s8s=UhjqNa_Tz1PFCRLzt3=5nvd5vD1wFdKWMQCmFySQ@mail.gmail.com>
<9ac6082a-2e30-4462-a260-1507452aa962@iki.fi>
<CAExHW5tSw8r06RLAArvf923cO4NGetitPhQ7AO0o7hsKx8jsNw@mail.gmail.com>
<aYpea9zn_MDhVA38@alap3.anarazel.de>
<CAExHW5ui9-aKtS-vwUYMQ4uSmi54cvLpvqrf29B8L69ZJa766g@mail.gmail.com>
Hi,
On 2026-02-10 11:02:13 +0530, Ashutosh Bapat wrote:
> > I still don't see what the point of having multiple mappings and using memfd
> > is. We need to reserve the address space for the maximum sized allocation in
> > postmaster, otherwise there's absolutely no guarantee that it's available at
> > those addresses in all the children - which you do as you explain
> > here. Therefore, the maximum size of each "suballocation" needs to be reserved
> > ahead of time. At which point I don't see the point of having multiple
> > mmaps. It just makes things more complicated and expensive (each mmap makes
> > fork & exit slower).
> >
> > Even if we decide to use memfd, because we consider MADV_DONTNEED to not be
> > suitable for some reason, what's the point of having more than one mapping
> > using memfd?
(this should reference MADV_REMOVE, not MADV_DONTNEED)
> There are just two mappings now compared to 6 earlier. If I am reading
> Jakub's benchmarking correctly, even 6 segments didn't show much
> regression in his benchmarks. Having just two should not see much
> regression. If we use multiple mappings we could control the
> properties of each segment separately - e.g. use huge pages for some
> (buffer blocks) and not for others. In Windows it seems it is easy to
> create multiple segments than punching holes in an existing segments.
> When we port the feature to Windows or other platforms, being able to
> treat all the segments in the same way would be an advantage.
> Said that I am not discarding the idea of using a single fd and then
> punching holes using fallocate() altogether; we will use it if
> multiple mappings do not bring any advantages. Let's also see how the
> on-demand shared memory segment feature being discussed in this thread
> with Heikki gets shaped.
I think the multiple memory mappings approach is just too restrictive. If we
e.g. eventually want to make some of the other major allocations that depend
on NBuffers react to resizing shared buffers, it's very easy to do if all it
requires is calling
madvise(TYPEALIGN(start, page_size), MADV_REMOVE, TYPEALIGN_DOWN(end, page_size));
There are several cases that are pretty easy to handle that way:
- Buffer Blocks
- Buffer Descriptors
- Sync request queue (part of the "Checkpointer Data" allocation)
- Checkpoint BufferIds (for sorting the to-be-checkpointed data)
- Buffer IO Condition Variables
But if you want to support making these resizable with the separate mappings
approach, it gets considerably more complicated and the number of mappings
increases more substantially.
We also don't need a lot less infrastructure in shmem.c that way. We could
e.g. make ShmemInitStruct() reservere the entire requested size (to avoid OOM
killer issues) and have a ShmemInitStructExt() that allows the caller choose
whether to reserve. No different segment IDs etc are needed.
Greetings,
Andres Freund
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: andres@anarazel.de, ashutosh.bapat.oss@gmail.com, hlinnaka@iki.fi, tomas@vondra.me, peter@eisentraut.org, thomas.munro@gmail.com, 9erthalion6@gmail.com, robertmhaas@gmail.com, chaturvedipalak1911@gmail.com
Subject: Re: Changing shared_buffers without restart
In-Reply-To: <aY4v1oSmokXNpQMX@alap3.anarazel.de>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox