agora inbox for pgsql-hackers@postgresql.org  

help / color / mirror / Atom feed

From: Dmitrii Dolgov <9erthalion6@gmail.com>
Subject: [PATCH v2 5/6] Allow to resize shared memory without restart
Date: Thu, 20 Feb 2025 21:12:26 +0100

Add assing hook for shared_buffers to resize shared memory using space,
introduced in the previous commits without requiring PostgreSQL restart.
Essentially the implementation is based on two mechanisms: a global
Barrier to coordinate backends that simultaneously change
shared_buffers, and pieces in shared memory to coordinate backends that
are too late to the party for some reason.

The resize process looks like this:

* The GUC assign hook sets a flag to let the Postmaster know that resize
  was requested.

* Postmaster verifies the flag in the event loop, and starts the resize
  by emitting a ProcSignal barrier. Afterwards it does shared memory
  resize itself.

* All the backends, that participate in ProcSignal mechanism,
  recalculate shared memory size based on the new NBuffers and extend it
  using mremap.

* When finished, a backend waits on a global ShmemControl barrier,
  untill all backends will be finished as well. This way we ensure three
  stages with clear boundaries: before the resize, when all processes
  use old NBuffers; during the resize, when processes have mix of old
  and new NBuffers, and wait until it's done; after the resize, when all
  processes use new NBuffers.

* After all backends are using new value, one backend will initialize
  new shared structures (buffer blocks, descriptors, etc) as needed and
  broadcast new value of NBuffers via ShmemControl in shared memory.
  Other backends are waiting for this operation to finish as well. Then
  the barrier is lifted and everything goes as usual.

Here is how it looks like after raising shared_buffers from 128 MB to
512 MB and calling pg_reload_conf():

    -- 128 MB
    7f5a2bd04000-7f5a32e52000  /dev/zero (deleted)
    7f5a39252000-7f5a4030e000  /dev/zero (deleted)
    7f5a4670e000-7f5a4d7ba000  /dev/zero (deleted)
    7f5a53bba000-7f5a5ad26000  /dev/zero (deleted)
    7f5a9ad26000-7f5aa9d94000  /dev/zero (deleted)
    ^ buffers mapping, ~240 MB
    7f5d29d94000-7f5d30e00000  /dev/zero (deleted)

    -- 512 MB
    7f5a2bd04000-7f5a33274000  /dev/zero (deleted)
    7f5a39252000-7f5a4057e000  /dev/zero (deleted)
    7f5a4670e000-7f5a4d9fa000  /dev/zero (deleted)
    7f5a53bba000-7f5a5b1a6000  /dev/zero (deleted)
    7f5a9ad26000-7f5ac1f14000  /dev/zero (deleted)
    ^ buffers mapping, ~625 MB
    7f5d29d94000-7f5d30f80000  /dev/zero (deleted)

The implementation supports only increasing of shared_buffers. For
decreasing the value a similar procedure is needed. But the buffer
blocks with data have to be drained first, so that the actual data set
fits into the new smaller space.

view thread (425+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: 9erthalion6@gmail.com
  Subject: Re: [PATCH v2 5/6] Allow to resize shared memory without restart
  In-Reply-To: <no-message-id-1108646@localhost>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox