Feedback-ID: id4a34324:Fastmail
Date: Thu, 12 Feb 2026 15:12:50 -0500
From: Andres Freund <andres@anarazel.de>
To: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Cc: Heikki Linnakangas <hlinnaka@iki.fi>, Tomas Vondra <tomas@vondra.me>,
	Peter Eisentraut <peter@eisentraut.org>,
 Thomas Munro <thomas.munro@gmail.com>,
	Dmitry Dolgov <9erthalion6@gmail.com>, pgsql-hackers@postgresql.org,
	Robert Haas <robertmhaas@gmail.com>, chaturvedipalak1911@gmail.com
Subject: Re: Changing shared_buffers without restart
Message-ID: <aY4v1oSmokXNpQMX@alap3.anarazel.de>
References: 
 <CAExHW5sYg_d4O7oGRqbomnVODeqR3YNAeYAa526n1dsWCM=+Fg@mail.gmail.com>
 <CAExHW5vB8sAmDtkEN5dcYYeBok3D8eAzMFCOH1k+krxht1yFjA@mail.gmail.com>
 <qltuzcdxapofdtb5mrd4em3bzu2qiwhp3cdwdsosmn7rhrtn4u@yaogvphfwc4h>
 <CAExHW5sOu8+9h6t7jsA5jVcQ--N-LCtjkPnCw+rpoN0ovT6PHg@mail.gmail.com>
 <CAExHW5sVxEwQsuzkgjjJQP9-XVe0H2njEVw1HxeYFdT7u7J+eQ@mail.gmail.com>
 <CAExHW5s8s=UhjqNa_Tz1PFCRLzt3=5nvd5vD1wFdKWMQCmFySQ@mail.gmail.com>
 <9ac6082a-2e30-4462-a260-1507452aa962@iki.fi>
 <CAExHW5tSw8r06RLAArvf923cO4NGetitPhQ7AO0o7hsKx8jsNw@mail.gmail.com>
 <aYpea9zn_MDhVA38@alap3.anarazel.de>
 <CAExHW5ui9-aKtS-vwUYMQ4uSmi54cvLpvqrf29B8L69ZJa766g@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
 <CAExHW5ui9-aKtS-vwUYMQ4uSmi54cvLpvqrf29B8L69ZJa766g@mail.gmail.com>
Archived-At: 
 <https://www.postgresql.org/message-id/aY4v1oSmokXNpQMX%40alap3.anarazel.de>
Precedence: bulk

Hi,

On 2026-02-10 11:02:13 +0530, Ashutosh Bapat wrote:
> > I still don't see what the point of having multiple mappings and using memfd
> > is. We need to reserve the address space for the maximum sized allocation in
> > postmaster, otherwise there's absolutely no guarantee that it's available at
> > those addresses in all the children - which you do as you explain
> > here. Therefore, the maximum size of each "suballocation" needs to be reserved
> > ahead of time. At which point I don't see the point of having multiple
> > mmaps. It just makes things more complicated and expensive (each mmap makes
> > fork & exit slower).
> >
> > Even if we decide to use memfd, because we consider MADV_DONTNEED to not be
> > suitable for some reason, what's the point of having more than one mapping
> > using memfd?

(this should reference MADV_REMOVE, not MADV_DONTNEED)


> There are just two mappings now compared to 6 earlier. If I am reading
> Jakub's benchmarking correctly, even 6 segments didn't show much
> regression in his benchmarks. Having just two should not see much
> regression. If we use multiple mappings we could control the
> properties of each segment separately - e.g. use huge pages for some
> (buffer blocks) and not for others. In Windows it seems it is easy to
> create multiple segments than punching holes in an existing segments.
> When we port the feature to Windows or other platforms, being able to
> treat all the segments in the same way would be an advantage.

> Said that I am not discarding the idea of using a single fd and then
> punching holes using fallocate() altogether; we will use it if
> multiple mappings do not bring any advantages. Let's also see how the
> on-demand shared memory segment feature being discussed in this thread
> with Heikki gets shaped.

I think the multiple memory mappings approach is just too restrictive. If we
e.g. eventually want to make some of the other major allocations that depend
on NBuffers react to resizing shared buffers, it's very easy to do if all it
requires is calling
   madvise(TYPEALIGN(start, page_size), MADV_REMOVE, TYPEALIGN_DOWN(end, page_size));

There are several cases that are pretty easy to handle that way:
- Buffer Blocks
- Buffer Descriptors
- Sync request queue (part of the "Checkpointer Data" allocation)
- Checkpoint BufferIds (for sorting the to-be-checkpointed data)
- Buffer IO Condition Variables

But if you want to support making these resizable with the separate mappings
approach, it gets considerably more complicated and the number of mappings
increases more substantially.

We also don't need a lot less infrastructure in shmem.c that way. We could
e.g. make ShmemInitStruct() reservere the entire requested size (to avoid OOM
killer issues) and have a ShmemInitStructExt() that allows the caller choose
whether to reserve. No different segment IDs etc are needed.

Greetings,

Andres Freund