Re: Changing shared_buffers without restart

pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  
help / color / mirror / Atom feed

From: Yuhang Qiu <iamqyh@gmail.com>
To: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Cc: pgsql-hackers@postgresql.org, Heikki Linnakangas <heikki.linnakangas@databricks.com>
Cc: Haoyu Huang <haoyu.huang@databricks.com>
Cc: Tomas Vondra <tomas@vondra.me>
Cc: Peter Eisentraut <peter@eisentraut.org>
Cc: Thomas Munro <thomas.munro@gmail.com>
Cc: Dmitry Dolgov <9erthalion6@gmail.com>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: chaturvedipalak1911@gmail.com, Andres Freund <andres@anarazel.de>
Cc: Jakub Wartak <jakub.wartak@enterprisedb.com>
Subject: Re: Changing shared_buffers without restart
Date: Thu, 20 Aug 2026 19:13:22 +0800
Message-ID: <B6CC6AF0-F7B3-4389-9740-DD1288EC15DB@gmail.com> (raw)
In-Reply-To: <CAExHW5ts93Rnof7pjFFYrY9aTOmCo+xsdQazqeqN1BaKozTBvA@mail.gmail.com>
References: <gzb2oqqabzqidlnumrbau2k2fscb5jo6nlpdjnf6pu5y4bk5mo@xoujqgtiqvgr>
	<CAExHW5uGozdiDm_BefS7jpQzaisCs4_nLTMF0+S_Pds49pRG_w@mail.gmail.com>
	<ufa7sghbf3fv7jobgd4yc5am4lzqf7bn2cigqv6cpi6vtkmbah@flj6n76hdgab>
	<CA+hUKG+KFbOhPudfcuHaa4TPsZxQ053LnXri2vYfeFNTo+WYjQ@mail.gmail.com>
	<CA+hUKGLfdc6gbXc0bE6R_TukCMzbKLvgjZsLtfyPxQOnOaGivw@mail.gmail.com>
	<kwvbsat7reeopjhwopfypwdhsfrcev7nmmltec5ec6zzkjol5o@itpqaokfw7kb>
	<CA+hUKGLQhsZ1dEf5Zo6JuPbs6n-qX=cTGy49feKf1iFA_TBP1g@mail.gmail.com>
	<CAExHW5v9cE+ETusTafZyvy+eVpnVvoayVm7ZOk4Ddq8fxY270A@mail.gmail.com>
	<CAExHW5sYg_d4O7oGRqbomnVODeqR3YNAeYAa526n1dsWCM=+Fg@mail.gmail.com>
	<CAExHW5vB8sAmDtkEN5dcYYeBok3D8eAzMFCOH1k+krxht1yFjA@mail.gmail.com>
	<qltuzcdxapofdtb5mrd4em3bzu2qiwhp3cdwdsosmn7rhrtn4u@yaogvphfwc4h>
	<CAExHW5sOu8+9h6t7jsA5jVcQ--N-LCtjkPnCw+rpoN0ovT6PHg@mail.gmail.com>
	<CAExHW5sVxEwQsuzkgjjJQP9-XVe0H2njEVw1HxeYFdT7u7J+eQ@mail.gmail.com>
	<CAExHW5s8s=UhjqNa_Tz1PFCRLzt3=5nvd5vD1wFdKWMQCmFySQ@mail.gmail.com>
	<CAKZiRmwxVqEbp7JgOed=BCT6cq8RNuHk3N0vuwro65Tsw9E8NA@mail.gmail.com>
	<CAExHW5vEfDQuqgV0Z_8=5htZTt186VioD+d2YtszywegAag5=Q@mail.gmail.com>
	<CAKZiRmy-ou3p9r7=ejARZMFEa8-eXwOPEVJd2Q8XqHrhSnTRag@mail.gmail.com>
	<CAExHW5stjJP7eOspBJEhkPb4ODYALSkMZyurb5AaWwOt8_umSw@mail.gmail.com>
	<CAKZiRmxfT7n0whHkLqtPkyUq_coH=UYP9fQ3V6Xv+2bug-XJSw@mail.gmail.com>
	<CAExHW5uX2LGwWHaeZu5XqK-83NwzUzpSAXn6hT+xx9X=21tweA@mail.gmail.com>
	<CAExHW5tYUYY13b4S1fv-zdxNsEEJzTy1EW6mRUj+FFEpA4Q0=g@mail.gmail.com>
	<CAExHW5ts93Rnof7pjFFYrY9aTOmCo+xsdQazqeqN1BaKozTBvA@mail.gmail.com>

Hi Ashutosh,

I reviewed the v20260817 patches, and here is what I found:

0003 / 0004:

BufTableGetContents() holds all mapping partition locks for the whole scan,
with no CHECK_FOR_INTERRUPTS. pg_buffercache_lookup_table is a view whose name
ends in "table", which is ambiguous. What about pg_buffercache_mappings?

With a resize pending, pg_settings.setting reads "16384 (pending: 32768)",
could break pg_size_bytes(current_setting('shared_buffers')). Maybe we need a
new GUC rather than a composite one.

0006:

configure hasn't been regenerated. The feature might not be compiled by default.

Shrinking PANICs, whenever max_shared_buffers > shared_buffers.
The range handed to madvise(MADV_REMOVE) in ShmemResizeStruct() ends at
maximum_size rather than at the current size, so it covers the PROT_NONE tail
and fails with EACCES.

There are now six read-only shared_memory_* values, and the names are getting
long. A function might fit better than that many GUCs.
shared_memory_size_in_huge_pages is gone, it might break compatibility.

MADV_POPULATE_WRITE requires a new OS kernel version. #ifdef is needed in
PGSharedMemoryEnsureAllocated.

"could not protect shared memory" is emitted from two places, so it's not
possible to tell which one failed.

0007:

EvictExtraBuffers() checks BM_VALID, but BM_TAG_VALID is the flag that means
there's a mapping table entry, so buffers with IO in flight are skipped by the
precheck.

"shared buffer resizing to %d buffers failed" doesn't say why it failed.

Some values like MaxProportionalPins aren't recomputed on resize.

001_resize_fault_tolerance.pl never reaches madvise(): with
max_shared_buffers = 32 buffers and huge pages still on for the TAP cluster,
the range rounds away at 2MB granularity, so everything passes with nothing
freed. That is also why the shrink problem above goes unnoticed.

Best Regards,
Yuhang Qiu.

view thread (167+ messages)  latest in thread

Message-ID: <B6CC6AF0-F7B3-4389-9740-DD1288EC15DB@gmail.com>
Permalink:  ../B6CC6AF0-F7B3-4389-9740-DD1288EC15DB@gmail.com/
Also on:    postgresql.org/message-id/B6CC6AF0-F7B3-4389-9740-DD1288EC15DB@gmail.com

 ·

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: iamqyh@gmail.com, ashutosh.bapat.oss@gmail.com, heikki.linnakangas@databricks.com, haoyu.huang@databricks.com, tomas@vondra.me, peter@eisentraut.org, thomas.munro@gmail.com, 9erthalion6@gmail.com, robertmhaas@gmail.com, andres@anarazel.de, jakub.wartak@enterprisedb.com
  Subject: Re: Changing shared_buffers without restart
  In-Reply-To: <B6CC6AF0-F7B3-4389-9740-DD1288EC15DB@gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox