Subject: Adding basic NUMA awareness

From: Tomas Vondra <[email-masked]>
Date: Tue, 1 Jul 2025 21:07:00 +0200

Hi,

This is a WIP version of a patch series I'm working on, adding some
basic NUMA awareness for a couple parts of our shared memory (shared
buffers, etc.). It's based on Andres' experimental patches he spoke
about at pgconf.eu 2024 [1], and while it's improved and polished in
various ways, it's still experimental.

But there's a recent thread aiming to do something similar [2], so
better to share it now so that we can discuss both approaches. This
patch set is a bit more ambitious, handling NUMA in a way to allow
smarter optimizations later, so I'm posting it in a separate thread.

The series is split into patches addressing different parts of the
shared memory, starting (unsurprisingly) from shared buffers, then
buffer freelists and ProcArray. There's a couple additional parts, but
those are smaller / addressing miscellaneous stuff.

Each patch has a numa_ GUC, intended to enable/disable that part. This
is meant to make development easier, not as a final interface. I'm not
sure how exactly that should look. It's possible some combinations of
GUCs won't work, etc.

Each patch should have a commit message explaining the intent and
implementation, and then also detailed comments explaining various
challenges and open questions.

But let me go over the basics, and discuss some of the design choices
and open questions that need solving.

1) v1-0001-NUMA-interleaving-buffers.patch

This is the main thing when people think about NUMA - making sure the
shared buffers are allocated evenly on all the nodes, not just on a
single node (which can happen easily with warmup). The regular memory
interleaving would address this, but it also has some disadvantages.

Firstly, it's oblivious to the contents of the shared memory segment,
and we may not want to interleave everything. It's also oblivious to
alignment of the items (a buffer can easily end up "split" on multiple
NUMA nodes), or relationship between different parts (e.g. there's a
BufferBlock and a related BufferDescriptor, and those might again end up
on different nodes).

So the patch handles this by explicitly mapping chunks of shared buffers
to different nodes - a bit like interleaving, but in larger chunks.
Ideally each node gets (1/N) of shared buffers, as a contiguous chunk.

It's a bit more complicated, because the patch distributes both the
blocks and descriptors, in the same way. So a buffer and it's descriptor
always end on the same NUMA node. This is one of the reasons why we need
to map larger chunks, because NUMA works on page granularity, and the
descriptors are tiny - many fit on a memory page.

There's a secondary benefit of explicitly assigning buffers to nodes,
using this simple scheme - it allows quickly determining the node ID
given a buffer ID. This is helpful later, when building freelist.

The patch is fairly simple. Most of the complexity is about picking the
chunk size, and aligning the arrays (so that it nicely aligns with
memory pages).

The patch has a GUC "numa_buffers_interleave", with "off" by default.

2) v1-0002-NUMA-localalloc.patch

This simply sets "localalloc" when initializing a backend, so that all
memory allocated later is local, not interleaved. Initially this was
necessary because the patch set the allocation policy to interleaving
before initializing shared memory, and we didn't want to interleave the
private memory. But that's no longer the case - the explicit mapping to
nodes does not have this issue. I'm keeping the patch for convenience,
it allows experimenting with numactl etc.

The patch has a GUC "numa_localalloc", with "off" by default.

3) v1-0003-freelist-Don-t-track-tail-of-a-freelist.patch

Minor optimization. Andres noticed we're tracking the tail of buffer
freelist, without using it. So the patch removes that.

4) v1-0004-NUMA-partition-buffer-freelist.patch

Right now we have a single freelist, and in busy instances that can be
quite contended. What's worse, the freelist may trash between different
CPUs, NUMA nodes, etc. So the idea is to have multiple freelists on
subsets of buffers. The patch implements multiple strategies how the
list can be split (configured using "numa_partition_freelist" GUC), for
experimenting:

* node - One list per NUMA node. This is the most natural option,
because we now know which buffer is on which node, so we can ensure a
list for a node only has buffers from that list.

* cpu - One list per CPU. Pretty simple, each CPU gets it's own list.

* pid - Similar to "cpu", but the processes are mapped to lists based on
PID, not CPU ID.

* none - nothing, sigle freelist

Ultimately, I think we'll want to go with "node", simply because it
aligns with the buffer interleaving. But there are improvements needed.

The main challenge is that with multiple smaller lists, a process can't
really use the whole shared buffers. So a single backed will only use
part of the memory. The more lists there are, the worse this effect is.
This is also why I think we won't use the other partitioning options,
because there's going to be more CPUs than NUMA nodes.

Obviously, this needs solving even with NUMA nodes - we need to allow a
single backend to utilize the whole shared buffers if needed. There
should be a way to "steal" buffers from other freelists (if the
"regular" freelist is empty), but the patch does not implement this.
Shouldn't be hard, I think.

The other missing part is clocksweep - there's still just a single
instance of clocksweep, feeding buffers to all the freelists. But that's
clearly a problem, because the clocksweep returns buffers from all NUMA
nodes. The clocksweep really needs to be partitioned the same way as a
freelists, and each partition will operate on a subset of buffers (from
the right NUMA node).

I do have a separate experimental patch doing something like that, I
need to make it part of this branch.

5) v1-0005-NUMA-interleave-PGPROC-entries.patch

Another area that seems like it might benefit from NUMA is PGPROC, so I
gave it a try. It turned out somewhat challenging. Similarly to buffers
we have two pieces that need to be located in a coordinated way - PGPROC
entries and fast-path arrays. But we can't use the same approach as for
buffers/descriptors, because

(a) Neither of those pieces aligns with memory page size (PGPROC is
~900B, fast-path arrays are variable length).

(b) We could pad PGPROC entries e.g. to 1KB, but that'd still require
rather high max_connections before we use multiple huge pages.

The fast-path arrays are less of a problem, because those tend to be
larger, and are accessed through pointers, so we can just adjust that.

So what I did instead is splitting the whole PGPROC array into one array
per NUMA node, and one array for auxiliary processes and 2PC xacts. So
with 4 NUMA nodes there are 5 separate arrays, for example. Each array
is a multiple of memory pages, so we may waste some of the memory. But
that's simply how NUMA works - page granularity.

This however makes one particular thing harder - in a couple places we
accessed PGPROC entries through PROC_HDR->allProcs, which was pretty
much just one large array. And GetNumberFromPGProc() relied on array
arithmetics to determine procnumber. With the array partitioned, this
can't work the same way.

But there's a simple solution - if we turn allProcs into an array of
*pointers* to PGPROC arrays, there's no issue. All the places need a
pointer anyway. And then we need an explicit procnumber field in PGPROC,
instead of calculating it.

There's a chance this have negative impact on code that accessed PGPROC
very often, but so far I haven't seen such cases. But if you can come up
with such examples, I'd like to see those.

There's another detail - when obtaining a PGPROC entry in InitProcess(),
we try to get an entry from the same NUMA node. And only if that doesn't
work, we grab the first one from the list (there's still just one PGPROC
freelist, I haven't split that - maybe we should?).

This has a GUC "numa_procs_interleave", again "off" by default. It's not
quite correct, though, because the partitioning happens always. It only
affects the PGPROC lookup. (In a way, this may be a bit broken.)

6) v1-0006-NUMA-pin-backends-to-NUMA-nodes.patch

This is an experimental patch, that simply pins the new process to the
NUMA node obtained from the freelist.

Driven by GUC "numa_procs_pin" (default: off).

Summary
-------

So this is what I have at the moment. I've tried to organize the patches
in the order of importance, but that's just my guess. It's entirely
possible there's something I missed, some other order might make more
sense, etc.

There's also the question how this is related to other patches affecting
shared memory - I think the most relevant one is the "shared buffers
online resize" by Ashutosh, simply because it touches the shared memory.

I don't think the splitting would actually make some things simpler, or
maybe more flexible - in particular, it'd allow us to enable huge pages
only for some regions (like shared buffers), and keep the small pages
e.g. for PGPROC. So that'd be good.

But there'd also need to be some logic to "rework" how shared buffers
get mapped to NUMA nodes after resizing. It'd be silly to start with
memory on 4 nodes (25% each), resize shared buffers to 50% and end up
with memory only on 2 of the nodes (because the other 2 nodes were
originally assigned the upper half of shared buffers).

I don't have a clear idea how this would be done, but I guess it'd
require a bit of code invoked sometime after the resize. It'd already
need to rebuild the freelists in some way, I guess.

The other thing I haven't thought about very much is determining on
which CPUs/nodes the instance is allowed to run. I assume we'd start by
simply inherit/determine that at the start through libnuma, not through
some custom PG configuration (which the patch [2] proposed to do).

regards

[1] https://www.youtube.com/watch?v=V75KpACdl6E

[2]
https://www.postgresql.org/message-id/CAKZiRmw6i1W1AwXxa-Asrn8wrVcVH3TO715g_MCoowTS9rkGyw%40mail.gmail.com

-- 
Tomas Vondra
From 9712e50d6d15c18ea2c5fcf457972486b0d4ef53 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 6 May 2025 21:12:21 +0200
Subject: [PATCH v1 1/6] NUMA: interleaving buffers

Ensure shared buffers are allocated from all NUMA nodes, in a balanced
way, instead of just using the node where Postgres initially starts, or
where the kernel decides to migrate the page, etc. With pre-warming
performed by a single backend, this can easily result in severely
unbalanced memory distribution (with most from a single NUMA node).

The kernel would eventually move some of the memory to other nodes
(thanks to zone_reclaim), but that tends to take a long time. So this
patch improves predictability, reduces the time needed for warmup
during benchmarking, etc.  It's less dependent on what the CPU
scheduler does, etc.

Furthermore, the buffers are mapped to NUMA nodes in a deterministic
way, so this also allows further improvements like backends using
buffers from the same NUMA node.

The effect is similar to

     numactl --interleave=all

but there's a number of important differences.

Firstly, it's applied only to shared buffers (and also to descriptors),
not to the whole shared memory segment. It's not clear we'd want to use
interleaving for all parts, storing entries with different sizes and
life cycles (e.g. ProcArray may need different approach).

Secondly, it considers the page and block size, and makes sure not to
split a buffer on different NUMA nodes (which with the regular
interleaving is guaranteed to happen, unless when using huge pages). The
patch performs "explicit" interleaving, so that buffers are not split
like this.

The patch maps both buffers and buffer descriptors, so that the buffer
and it's buffer descriptor end up on the same NUMA node.

The mapping happens in larger chunks (see choose_chunk_items). This is
required to handle buffer descriptors (which are smaller than buffers),
and it should also help to reduce the number of mappings. Most NUMA
systems will use 1GB chunks, unless using very small shared buffers.

Notes:

* The feature is enabled by numa_buffers_interleave GUC (false by default)

* It's not clear we want to enable interleaving for all shared memory.
  We probably want that for shared buffers, but maybe not for ProcArray
  or freelists.

* Similar questions are about huge pages - in general it's a good idea,
  but maybe it's not quite good for ProcArray. It's somewhate separate
  from NUMA, but not entirely because NUMA works on page granularity.
  PGPROC entries are ~8KB, so too large for interleaving with 4K pages,
  as we don't want to split the entry to multiple nodes. But could be
  done explicitly, by specifying which node to use for the pages.

* We could partition ProcArray, with one partition per NUMA node, and
  then at connection time pick a node from the same node. The process
  could migrate to some other node later, especially for long-lived
  connections, but there's no perfect solution, Maybe we could set
  affinity to cores from the same node, or something like that?
---
 src/backend/storage/buffer/buf_init.c | 384 +++++++++++++++++++++++++-
 src/backend/storage/buffer/bufmgr.c   |   1 +
 src/backend/utils/init/globals.c      |   3 +
 src/backend/utils/misc/guc_tables.c   |  10 +
 src/bin/pgbench/pgbench.c             |  67 ++---
 src/include/miscadmin.h               |   2 +
 src/include/storage/bufmgr.h          |   1 +
 7 files changed, 427 insertions(+), 41 deletions(-)

diff --git a/src/backend/storage/buffer/buf_init.c b/src/backend/storage/buffer/buf_init.c
index ed1dc488a42..2ad34624c49 100644
--- a/src/backend/storage/buffer/buf_init.c
+++ b/src/backend/storage/buffer/buf_init.c
@@ -14,9 +14,17 @@
  */
 #include "postgres.h"
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
+#include "port/pg_numa.h"
 #include "storage/aio.h"
 #include "storage/buf_internals.h"
 #include "storage/bufmgr.h"
+#include "storage/pg_shmem.h"
+#include "storage/proc.h"
 
 BufferDescPadded *BufferDescriptors;
 char	   *BufferBlocks;
@@ -25,6 +33,19 @@ WritebackContext BackendWritebackContext;
 CkptSortItem *CkptBufferIds;

+static Size get_memory_page_size(void);
+static int64 choose_chunk_buffers(int NBuffers, Size mem_page_size, int num_nodes);
+static void pg_numa_interleave_memory(char *startptr, char *endptr,
+									  Size mem_page_size, Size chunk_size,
+									  int num_nodes);
+
+/* number of buffers allocated on the same NUMA node */
+static int64 numa_chunk_buffers = -1;
+
+/* number of NUMA nodes (as returned by numa_num_configured_nodes) */
+static int	numa_nodes = -1;
+
+
 /*
  * Data Structures:
  *		buffers live in a freelist and a lookup data structure.
@@ -71,18 +92,80 @@ BufferManagerShmemInit(void)
 				foundDescs,
 				foundIOCV,
 				foundBufCkpt;
+	Size		mem_page_size;
+	Size		buffer_align;
+
+	/*
+	 * XXX A bit weird. Do we need to worry about postmaster? Could this even
+	 * run outside postmaster? I don't think so.
+	 *
+	 * XXX Another issue is we may get different values than when sizing the
+	 * the memory, because at that point we didn't know if we get huge pages,
+	 * so we assumed we will. Shouldn't cause crashes, but we might allocate
+	 * shared memory and then not use some of it (because of the alignment
+	 * that we don't actually need). Not sure about better way, good for now.
+	 */
+	if (IsUnderPostmaster)
+		mem_page_size = pg_get_shmem_pagesize();
+	else
+		mem_page_size = get_memory_page_size();
+
+	/*
+	 * With NUMA we need to ensure the buffers are properly aligned not just
+	 * to PG_IO_ALIGN_SIZE, but also to memory page size, because NUMA works
+	 * on page granularity, and we don't want a buffer to get split to
+	 * multiple nodes (when using multiple memory pages).
+	 *
+	 * We also don't want to interfere with other parts of shared memory,
+	 * which could easily happen with huge pages (e.g. with data stored before
+	 * buffers).
+	 *
+	 * We do this by aligning to the larger of the two values (we know both
+	 * are power-of-two values, so the larger value is automatically a
+	 * multiple of the lesser one).
+	 *
+	 * XXX Maybe there's a way to use less alignment?
+	 *
+	 * XXX Maybe with (mem_page_size > PG_IO_ALIGN_SIZE), we don't need to
+	 * align to mem_page_size? Especially for very large huge pages (e.g. 1GB)
+	 * that doesn't seem quite worth it. Maybe we should simply align to
+	 * BLCKSZ, so that buffers don't get split? Still, we might interfere with
+	 * other stuff stored in shared memory that we want to allocate on a
+	 * particular NUMA node (e.g. ProcArray).
+	 *
+	 * XXX Maybe with "too large" huge pages we should just not do this, or
+	 * maybe do this only for sufficiently large areas (e.g. shared buffers,
+	 * but not ProcArray).
+	 */
+	buffer_align = Max(mem_page_size, PG_IO_ALIGN_SIZE);
+
+	/* one page is a multiple of the other */
+	Assert(((mem_page_size % PG_IO_ALIGN_SIZE) == 0) ||
+		   ((PG_IO_ALIGN_SIZE % mem_page_size) == 0));
 
-	/* Align descriptors to a cacheline boundary. */
+	/*
+	 * Align descriptors to a cacheline boundary, and memory page.
+	 *
+	 * We want to distribute both to NUMA nodes, so that each buffer and it's
+	 * descriptor are on the same NUMA node. So we align both the same way.
+	 *
+	 * XXX The memory page is always larger than cacheline, so the cacheline
+	 * reference is a bit unnecessary.
+	 *
+	 * XXX In principle we only need to do this with NUMA, otherwise we could
+	 * still align just to cacheline, as before.
+	 */
 	BufferDescriptors = (BufferDescPadded *)
-		ShmemInitStruct("Buffer Descriptors",
-						NBuffers * sizeof(BufferDescPadded),
-						&foundDescs);
+		TYPEALIGN(buffer_align,
+				  ShmemInitStruct("Buffer Descriptors",
+								  NBuffers * sizeof(BufferDescPadded) + buffer_align,
+								  &foundDescs));
 
 	/* Align buffer pool on IO page size boundary. */
 	BufferBlocks = (char *)
-		TYPEALIGN(PG_IO_ALIGN_SIZE,
+		TYPEALIGN(buffer_align,
 				  ShmemInitStruct("Buffer Blocks",
-								  NBuffers * (Size) BLCKSZ + PG_IO_ALIGN_SIZE,
+								  NBuffers * (Size) BLCKSZ + buffer_align,
 								  &foundBufs));
 
 	/* Align condition variables to cacheline boundary. */
@@ -112,6 +195,63 @@ BufferManagerShmemInit(void)
 	{
 		int			i;
 
+		/*
+		 * Assign chunks of buffers and buffer descriptors to the available
+		 * NUMA nodes. We can't use the regular interleaving, because with
+		 * regular memory pages (smaller than BLCKSZ) we'd split all buffers
+		 * to multiple NUMA nodes. And we don't want that.
+		 *
+		 * But even with huge pages it seems like a good idea to not have
+		 * mapping for each page.
+		 *
+		 * So we always assign a larger contiguous chunk of buffers to the
+		 * same NUMA node, as calculated by choose_chunk_buffers(). We try to
+		 * keep the chunks large enough to work both for buffers and buffer
+		 * descriptors, but not too large. See the comments at
+		 * choose_chunk_buffers() for details.
+		 *
+		 * Thanks to the earlier alignment (to memory page etc.), we know the
+		 * buffers won't get split, etc.
+		 *
+		 * This also makes it easier / straightforward to calculate which NUMA
+		 * node a buffer belongs to (it's a matter of divide + mod). See
+		 * BufferGetNode().
+		 */
+		if (numa_buffers_interleave)
+		{
+			char	   *startptr,
+					   *endptr;
+			Size		chunk_size;
+
+			numa_nodes = numa_num_configured_nodes();
+
+			numa_chunk_buffers
+				= choose_chunk_buffers(NBuffers, mem_page_size, numa_nodes);
+
+			elog(LOG, "BufferManagerShmemInit num_nodes %d chunk_buffers %ld",
+				 numa_nodes, numa_chunk_buffers);
+
+			/* first map buffers */
+			startptr = BufferBlocks;
+			endptr = startptr + ((Size) NBuffers) * BLCKSZ;
+			chunk_size = (numa_chunk_buffers * BLCKSZ);
+
+			pg_numa_interleave_memory(startptr, endptr,
+									  mem_page_size,
+									  chunk_size,
+									  numa_nodes);
+
+			/* now do the same for buffer descriptors */
+			startptr = (char *) BufferDescriptors;
+			endptr = startptr + ((Size) NBuffers) * sizeof(BufferDescPadded);
+			chunk_size = (numa_chunk_buffers * sizeof(BufferDescPadded));
+
+			pg_numa_interleave_memory(startptr, endptr,
+									  mem_page_size,
+									  chunk_size,
+									  numa_nodes);
+		}
+
 		/*
 		 * Initialize all the buffer headers.
 		 */
@@ -144,6 +284,11 @@ BufferManagerShmemInit(void)
 		GetBufferDescriptor(NBuffers - 1)->freeNext = FREENEXT_END_OF_LIST;
 	}
 
+	/*
+	 * As this point we have all the buffers in a single long freelist. With
+	 * freelist partitioning we rebuild them in StrategyInitialize.
+	 */
+
 	/* Init other shared buffer-management stuff */
 	StrategyInitialize(!foundDescs);
 
@@ -152,24 +297,72 @@ BufferManagerShmemInit(void)
 						 &backend_flush_after);
 }
 
+/*
+ * Determine the size of memory page.
+ *
+ * XXX This is a bit tricky, because the result depends at which point we call
+ * this. Before the allocation we don't know if we succeed in allocating huge
+ * pages - but we have to size everything for the chance that we will. And then
+ * if the huge pages fail (with 'huge_pages=try'), we'll use the regular memory
+ * pages. But at that point we can't adjust the sizing.
+ *
+ * XXX Maybe with huge_pages=try we should do the sizing twice - first with
+ * huge pages, and if that fails, then without them. But not for this patch.
+ * Up to this point there was no such dependency on huge pages.
+ */
+static Size
+get_memory_page_size(void)
+{
+	Size		os_page_size;
+	Size		huge_page_size;
+
+#ifdef WIN32
+	SYSTEM_INFO sysinfo;
+
+	GetSystemInfo(&sysinfo);
+	os_page_size = sysinfo.dwPageSize;
+#else
+	os_page_size = sysconf(_SC_PAGESIZE);
+#endif
+
+	/* assume huge pages get used, unless HUGE_PAGES_OFF */
+	if (huge_pages_status != HUGE_PAGES_OFF)
+		GetHugePageSize(&huge_page_size, NULL);
+	else
+		huge_page_size = 0;
+
+	return Max(os_page_size, huge_page_size);
+}
+
 /*
  * BufferManagerShmemSize
  *
  * compute the size of shared memory for the buffer pool including
  * data pages, buffer descriptors, hash tables, etc.
+ *
+ * XXX Called before allocation, so we don't know if huge pages get used yet.
+ * So we need to assume huge pages get used, and use get_memory_page_size()
+ * to calculate the largest possible memory page.
  */
 Size
 BufferManagerShmemSize(void)
 {
 	Size		size = 0;
+	Size		mem_page_size;
+
+	/* XXX why does IsUnderPostmaster matter? */
+	if (IsUnderPostmaster)
+		mem_page_size = pg_get_shmem_pagesize();
+	else
+		mem_page_size = get_memory_page_size();
 
 	/* size of buffer descriptors */
 	size = add_size(size, mul_size(NBuffers, sizeof(BufferDescPadded)));
 	/* to allow aligning buffer descriptors */
-	size = add_size(size, PG_CACHE_LINE_SIZE);
+	size = add_size(size, Max(mem_page_size, PG_IO_ALIGN_SIZE));
 
 	/* size of data pages, plus alignment padding */
-	size = add_size(size, PG_IO_ALIGN_SIZE);
+	size = add_size(size, Max(mem_page_size, PG_IO_ALIGN_SIZE));
 	size = add_size(size, mul_size(NBuffers, BLCKSZ));
 
 	/* size of stuff controlled by freelist.c */
@@ -186,3 +379,178 @@ BufferManagerShmemSize(void)
 
 	return size;
 }
+
+/*
+ * choose_chunk_buffers
+ *		choose the number of buffers allocated to a NUMA node at once
+ *
+ * We don't map shared buffers to NUMA nodes one by one, but in larger chunks.
+ * This is both for efficiency reasons (fewer mappings), and also because we
+ * want to map buffer descriptors too - and descriptors are much smaller. So
+ * we pick a number that's high enough for descriptors to use whole pages.
+ *
+ * We also want to keep buffers somehow evenly distributed on nodes, with
+ * about NBuffers/nodes per node. So we don't use chunks larger than this,
+ * to keep it as fair as possible (the chunk size is a possible difference
+ * between memory allocated to different NUMA nodes).
+ *
+ * It's possible shared buffers are so small this is not possible (i.e.
+ * it's less than chunk_size). But sensible NUMA systems will use a lot
+ * of memory, so this is unlikely.
+ *
+ * We simply print a warning about the misbalance, and that's it.
+ *
+ * XXX It'd be good to ensure the chunk size is a power-of-2, because then
+ * we could calculate the NUMA node simply by shift/modulo, while now we
+ * have to do a division. But we don't know how many buffers and buffer
+ * descriptors fits into a memory page. It may not be a power-of-2.
+ */
+static int64
+choose_chunk_buffers(int NBuffers, Size mem_page_size, int num_nodes)
+{
+	int64		num_items;
+	int64		max_items;
+
+	/* make sure the chunks will align nicely */
+	Assert(BLCKSZ % sizeof(BufferDescPadded) == 0);
+	Assert(mem_page_size % sizeof(BufferDescPadded) == 0);
+	Assert(((BLCKSZ % mem_page_size) == 0) || ((mem_page_size % BLCKSZ) == 0));
+
+	/*
+	 * The minimum number of items to fill a memory page with descriptors and
+	 * blocks. The NUMA allocates memory in pages, and we need to do that for
+	 * both buffers and descriptors.
+	 *
+	 * In practice the BLCKSZ doesn't really matter, because it's much larger
+	 * than BufferDescPadded, so the result is determined buffer descriptors.
+	 * But it's clearer this way.
+	 */
+	num_items = Max(mem_page_size / sizeof(BufferDescPadded),
+					mem_page_size / BLCKSZ);
+
+	/*
+	 * We shouldn't use chunks larger than NBuffers/num_nodes, because with
+	 * larger chunks the last NUMA node would end up with much less memory (or
+	 * no memory at all).
+	 */
+	max_items = (NBuffers / num_nodes);
+
+	/*
+	 * Did we already exceed the maximum desirable chunk size? That is, will
+	 * the last node get less than one whole chunk (or no memory at all)?
+	 */
+	if (num_items > max_items)
+		elog(WARNING, "choose_chunk_buffers: chunk items exceeds max (%ld > %ld)",
+			 num_items, max_items);
+
+	/* grow the chunk size until we hit the max limit. */
+	while (2 * num_items <= max_items)
+		num_items *= 2;
+
+	/*
+	 * XXX It's not difficult to construct cases where we end up with not
+	 * quite balanced distribution. For example, with shared_buffers=10GB and
+	 * 4 NUMA nodes, we end up with 2GB chunks, which means the first node
+	 * gets 4GB, and the three other nodes get 2GB each.
+	 *
+	 * We could be smarter, and try to get more balanced distribution. We
+	 * could simply reduce max_items e.g. to
+	 *
+	 * max_items = (NBuffers / num_nodes) / 4;
+	 *
+	 * in which cases we'd end up with 512MB chunks, and each nodes would get
+	 * the same 2.5GB chunk. It may not always work out this nicely, but it's
+	 * better than with (NBuffers / num_nodes).
+	 *
+	 * Alternatively, we could "backtrack" - try with the large max_items,
+	 * check how balanced it is, and if it's too imbalanced, try with a
+	 * smaller one.
+	 *
+	 * We however want a simple scheme.
+	 */
+
+	return num_items;
+}
+
+/*
+ * Calculate the NUMA node for a given buffer.
+ */
+int
+BufferGetNode(Buffer buffer)
+{
+	/* not NUMA interleaving */
+	if (numa_chunk_buffers == -1)
+		return -1;
+
+	return (buffer / numa_chunk_buffers) % numa_nodes;
+}
+
+/*
+ * pg_numa_interleave_memory
+ *		move memory to different NUMA nodes in larger chunks
+ *
+ * startptr - start of the region (should be aligned to page size)
+ * endptr - end of the region (doesn't need to be aligned)
+ * mem_page_size - size of the memory page size
+ * chunk_size - size of the chunk to move to a single node (should be multiple
+ *              of page size
+ * num_nodes - number of nodes to allocate memory to
+ *
+ * XXX Maybe this should use numa_tonode_memory and numa_police_memory instead?
+ * That might be more efficient than numa_move_pages, as it works on larger
+ * chunks of memory, not individual system pages, I think.
+ *
+ * XXX The "interleave" name is not quite accurate, I guess.
+ */
+static void
+pg_numa_interleave_memory(char *startptr, char *endptr,
+						  Size mem_page_size, Size chunk_size,
+						  int num_nodes)
+{
+	volatile uint64 touch pg_attribute_unused();
+	char	   *ptr = startptr;
+
+	/* chunk size has to be a multiple of memory page */
+	Assert((chunk_size % mem_page_size) == 0);
+
+	/*
+	 * Walk the memory pages in the range, and determine the node for each
+	 * one. We use numa_tonode_memory(), because then we can move a whole
+	 * memory range to the node, we don't need to worry about individual pages
+	 * like with numa_move_pages().
+	 */
+	while (ptr < endptr)
+	{
+		/* We may have an incomplete chunk at the end. */
+		Size		sz = Min(chunk_size, (endptr - ptr));
+
+		/*
+		 * What NUMA node does this range belong to? Each chunk should go to
+		 * the same NUMA node, in a round-robin manner.
+		 */
+		int			node = ((ptr - startptr) / chunk_size) % num_nodes;
+
+		/*
+		 * Nope, we have the first buffer from the next memory page, and we'll
+		 * set NUMA node for it (and all pages up to the next buffer). The
+		 * buffer should align with the memory page, thanks to the
+		 * buffer_align earlier.
+		 */
+		Assert((int64) ptr % mem_page_size == 0);
+		Assert((sz % mem_page_size) == 0);
+
+		/*
+		 * XXX no return value, to make this fail on error, has to use
+		 * numa_set_strict
+		 *
+		 * XXX Should we still touch the memory first, like with numa_move_pages,
+		 * or is that not necessary?
+		 */
+		numa_tonode_memory(ptr, sz, node);
+
+		ptr += sz;
+	}
+
+	/* should have processed all chunks */
+	Assert(ptr == endptr);
+}
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 406ce77693c..e1e1cfd379d 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -685,6 +685,7 @@ ReadRecentBuffer(RelFileLocator rlocator, ForkNumber forkNum, BlockNumber blockN
 	BufferDesc *bufHdr;
 	BufferTag	tag;
 	uint32		buf_state;
+
 	Assert(BufferIsValid(recent_buffer));
 
 	ResourceOwnerEnlarge(CurrentResourceOwner);
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..876cb64cf66 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -145,6 +145,9 @@ int			max_worker_processes = 8;
 int			max_parallel_workers = 8;
 int			MaxBackends = 0;
 
+/* NUMA stuff */
+bool		numa_buffers_interleave = false;
+
 /* GUC parameters for vacuum */
 int			VacuumBufferUsageLimit = 2048;
 
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 511dc32d519..198a57e70a5 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2116,6 +2116,16 @@ struct config_bool ConfigureNamesBool[] =
 		NULL, NULL, NULL
 	},
 
+	{
+		{"numa_buffers_interleave", PGC_POSTMASTER, DEVELOPER_OPTIONS,
+			gettext_noop("Enables NUMA interleaving of shared buffers."),
+			gettext_noop("When enabled, the buffers in shared memory are interleaved to all NUMA nodes."),
+		},
+		&numa_buffers_interleave,
+		false,
+		NULL, NULL, NULL
+	},
+
 	{
 		{"sync_replication_slots", PGC_SIGHUP, REPLICATION_STANDBY,
 			gettext_noop("Enables a physical standby to synchronize logical failover replication slots from the primary server."),
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 69b6a877dc9..c07de903f76 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -305,7 +305,7 @@ static const char *progname;
 #define	CPU_PINNING_RANDOM		1
 #define	CPU_PINNING_COLOCATED	2
 
-static int pinning_mode = CPU_PINNING_NONE;
+static int	pinning_mode = CPU_PINNING_NONE;
 
 #define WSEP '@'				/* weight separator */
 
@@ -874,20 +874,20 @@ static bool socket_has_input(socket_set *sa, int fd, int idx);
  */
 typedef struct cpu_generator_state
 {
-	int		ncpus;		/* number of CPUs available */
-	int		nitems;		/* number of items in the queue */
-	int	   *nthreads;	/* number of threads for each CPU */
-	int	   *nclients;	/* number of processes for each CPU */
-	int	   *items;		/* queue of CPUs to pick from */
-} cpu_generator_state;
+	int			ncpus;			/* number of CPUs available */
+	int			nitems;			/* number of items in the queue */
+	int		   *nthreads;		/* number of threads for each CPU */
+	int		   *nclients;		/* number of processes for each CPU */
+	int		   *items;			/* queue of CPUs to pick from */
+}			cpu_generator_state;
 
 static cpu_generator_state cpu_generator_init(int ncpus);
-static void cpu_generator_refill(cpu_generator_state *state);
-static void cpu_generator_reset(cpu_generator_state *state);
-static int cpu_generator_thread(cpu_generator_state *state);
-static int cpu_generator_client(cpu_generator_state *state, int thread_cpu);
-static void cpu_generator_print(cpu_generator_state *state);
-static bool cpu_generator_check(cpu_generator_state *state);
+static void cpu_generator_refill(cpu_generator_state * state);
+static void cpu_generator_reset(cpu_generator_state * state);
+static int	cpu_generator_thread(cpu_generator_state * state);
+static int	cpu_generator_client(cpu_generator_state * state, int thread_cpu);
+static void cpu_generator_print(cpu_generator_state * state);
+static bool cpu_generator_check(cpu_generator_state * state);
 
 static void reset_pinning(TState *threads, int nthreads);
 
@@ -7422,7 +7422,7 @@ main(int argc, char **argv)
 	/* try to assign threads/clients to CPUs */
 	if (pinning_mode != CPU_PINNING_NONE)
 	{
-		int nprocs = get_nprocs();
+		int			nprocs = get_nprocs();
 		cpu_generator_state state = cpu_generator_init(nprocs);
 
 retry:
@@ -7433,6 +7433,7 @@ retry:
 		for (i = 0; i < nthreads; i++)
 		{
 			TState	   *thread = &threads[i];
+
 			thread->cpu = cpu_generator_thread(&state);
 		}
 
@@ -7444,7 +7445,7 @@ retry:
 		while (true)
 		{
 			/* did we find any unassigned backend? */
-			bool found = false;
+			bool		found = false;
 
 			for (i = 0; i < nthreads; i++)
 			{
@@ -7678,10 +7679,10 @@ threadRun(void *arg)
 		/* determine PID of the backend, pin it to the same CPU */
 		for (int i = 0; i < nstate; i++)
 		{
-			char   *pid_str;
-			pid_t	pid;
+			char	   *pid_str;
+			pid_t		pid;
 
-			PGresult *res = PQexec(state[i].con, "select pg_backend_pid()");
+			PGresult   *res = PQexec(state[i].con, "select pg_backend_pid()");
 
 			if (PQresultStatus(res) != PGRES_TUPLES_OK)
 				pg_fatal("could not determine PID of the backend for client %d",
@@ -8184,7 +8185,7 @@ cpu_generator_init(int ncpus)
 {
 	struct timeval tv;
 
-	cpu_generator_state	state;
+	cpu_generator_state state;
 
 	state.ncpus = ncpus;
 
@@ -8207,7 +8208,7 @@ cpu_generator_init(int ncpus)
 }
 
 static void
-cpu_generator_refill(cpu_generator_state *state)
+cpu_generator_refill(cpu_generator_state * state)
 {
 	struct timeval tv;
 
@@ -8223,7 +8224,7 @@ cpu_generator_refill(cpu_generator_state *state)
 }
 
 static void
-cpu_generator_reset(cpu_generator_state *state)
+cpu_generator_reset(cpu_generator_state * state)
 {
 	state->nitems = 0;
 	cpu_generator_refill(state);
@@ -8236,15 +8237,15 @@ cpu_generator_reset(cpu_generator_state *state)
 }
 
 static int
-cpu_generator_thread(cpu_generator_state *state)
+cpu_generator_thread(cpu_generator_state * state)
 {
 	if (state->nitems == 0)
 		cpu_generator_refill(state);
 
 	while (true)
 	{
-		int idx = lrand48() % state->nitems;
-		int cpu = state->items[idx];
+		int			idx = lrand48() % state->nitems;
+		int			cpu = state->items[idx];
 
 		state->items[idx] = state->items[state->nitems - 1];
 		state->nitems--;
@@ -8256,10 +8257,10 @@ cpu_generator_thread(cpu_generator_state *state)
 }
 
 static int
-cpu_generator_client(cpu_generator_state *state, int thread_cpu)
+cpu_generator_client(cpu_generator_state * state, int thread_cpu)
 {
-	int		min_clients;
-	bool	has_valid_cpus = false;
+	int			min_clients;
+	bool		has_valid_cpus = false;
 
 	for (int i = 0; i < state->nitems; i++)
 	{
@@ -8284,8 +8285,8 @@ cpu_generator_client(cpu_generator_state *state, int thread_cpu)
 
 	while (true)
 	{
-		int idx = lrand48() % state->nitems;
-		int cpu = state->items[idx];
+		int			idx = lrand48() % state->nitems;
+		int			cpu = state->items[idx];
 
 		if (cpu == thread_cpu)
 			continue;
@@ -8303,7 +8304,7 @@ cpu_generator_client(cpu_generator_state *state, int thread_cpu)
 }
 
 static void
-cpu_generator_print(cpu_generator_state *state)
+cpu_generator_print(cpu_generator_state * state)
 {
 	for (int i = 0; i < state->ncpus; i++)
 	{
@@ -8312,10 +8313,10 @@ cpu_generator_print(cpu_generator_state *state)
 }
 
 static bool
-cpu_generator_check(cpu_generator_state *state)
+cpu_generator_check(cpu_generator_state * state)
 {
-	int	min_count = INT_MAX,
-		max_count = 0;
+	int			min_count = INT_MAX,
+				max_count = 0;
 
 	for (int i = 0; i < state->ncpus; i++)
 	{
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..014a6079af2 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -178,6 +178,8 @@ extern PGDLLIMPORT int MaxConnections;
 extern PGDLLIMPORT int max_worker_processes;
 extern PGDLLIMPORT int max_parallel_workers;
 
+extern PGDLLIMPORT bool numa_buffers_interleave;
+
 extern PGDLLIMPORT int commit_timestamp_buffers;
 extern PGDLLIMPORT int multixact_member_buffers;
 extern PGDLLIMPORT int multixact_offset_buffers;
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index 41fdc1e7693..c257c8a1c20 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -319,6 +319,7 @@ extern void EvictRelUnpinnedBuffers(Relation rel,
 /* in buf_init.c */
 extern void BufferManagerShmemInit(void);
 extern Size BufferManagerShmemSize(void);
+extern int	BufferGetNode(Buffer buffer);
 
 /* in localbuf.c */
 extern void AtProcExit_LocalBuffers(void);
-- 
2.49.0

From 6919b1c1c59a6084017ebae5a884bb6c60639364 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Thu, 22 May 2025 18:27:06 +0200
Subject: [PATCH v1 2/6] NUMA: localalloc

Set the default allocation policy to "localalloc", which means from the
local NUMA node. This is useful for process-private memory, which is not
going to be shared with other nodes, and is relatively short-lived (so
we're unlikely to have issues if the process gets moved by scheduler).

This sets default for the whole process, for all future allocations. But
that's fine, we've already populated the shared memory earlier (by
interleaving it explicitly). Otherwise we'd trigger page fault and it'd
be allocated on local node.

XXX This patch may not be necessary, as we now locate memory to nodes
using explicit numa_tonode_memory() calls, and not by interleaving. But
it's useful for experiments during development, so I'm keeping it.
---
 src/backend/utils/init/globals.c    |  1 +
 src/backend/utils/init/miscinit.c   | 16 ++++++++++++++++
 src/backend/utils/misc/guc_tables.c | 10 ++++++++++
 src/include/miscadmin.h             |  1 +
 4 files changed, 28 insertions(+)

diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 876cb64cf66..f5359db3656 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -147,6 +147,7 @@ int			MaxBackends = 0;
 
 /* NUMA stuff */
 bool		numa_buffers_interleave = false;
+bool		numa_localalloc = false;
 
 /* GUC parameters for vacuum */
 int			VacuumBufferUsageLimit = 2048;
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index 43b4dbccc3d..d11936691b2 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -28,6 +28,10 @@
 #include <arpa/inet.h>
 #include <utime.h>
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#endif
+
 #include "access/htup_details.h"
 #include "access/parallel.h"
 #include "catalog/pg_authid.h"
@@ -164,6 +168,18 @@ InitPostmasterChild(void)
 				(errcode_for_socket_access(),
 				 errmsg_internal("could not set postmaster death monitoring pipe to FD_CLOEXEC mode: %m")));
 #endif
+
+#ifdef USE_LIBNUMA
+	/*
+	 * Set the default allocation policy to local node, where the task is
+	 * executing at the time of a page fault.
+	 *
+	 * XXX I believe this is not necessary, now that we don't use automatic
+	 * interleaving (numa_set_interleave_mask).
+	 */
+	if (numa_localalloc)
+		numa_set_localalloc();
+#endif
 }
 
 /*
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 198a57e70a5..57f2df7ab74 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2126,6 +2126,16 @@ struct config_bool ConfigureNamesBool[] =
 		NULL, NULL, NULL
 	},
 
+	{
+		{"numa_localalloc", PGC_POSTMASTER, DEVELOPER_OPTIONS,
+			gettext_noop("Enables setting the default allocation policy to local node."),
+			gettext_noop("When enabled, allocate from the node where the task is executing."),
+		},
+		&numa_localalloc,
+		false,
+		NULL, NULL, NULL
+	},
+
 	{
 		{"sync_replication_slots", PGC_SIGHUP, REPLICATION_STANDBY,
 			gettext_noop("Enables a physical standby to synchronize logical failover replication slots from the primary server."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 014a6079af2..692871a401f 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -179,6 +179,7 @@ extern PGDLLIMPORT int max_worker_processes;
 extern PGDLLIMPORT int max_parallel_workers;
 
 extern PGDLLIMPORT bool numa_buffers_interleave;
+extern PGDLLIMPORT bool numa_localalloc;
 
 extern PGDLLIMPORT int commit_timestamp_buffers;
 extern PGDLLIMPORT int multixact_member_buffers;
-- 
2.49.0

From c2b2edb71d629ebe4283b636f058b8e42d1f1a35 Mon Sep 17 00:00:00 2001
From: Andres Freund <[email-masked]>
Date: Mon, 14 Oct 2024 14:10:13 -0400
Subject: [PATCH v1 3/6] freelist: Don't track tail of a freelist

The freelist tail isn't currently used, making it unnecessary overhead.
So just don't do that.
---
 src/backend/storage/buffer/freelist.c | 9 ---------
 1 file changed, 9 deletions(-)

diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index 01909be0272..e046526c149 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -40,12 +40,6 @@ typedef struct
 	pg_atomic_uint32 nextVictimBuffer;
 
 	int			firstFreeBuffer;	/* Head of list of unused buffers */
-	int			lastFreeBuffer; /* Tail of list of unused buffers */
-
-	/*
-	 * NOTE: lastFreeBuffer is undefined when firstFreeBuffer is -1 (that is,
-	 * when the list is empty)
-	 */
 
 	/*
 	 * Statistics.  These counters should be wide enough that they can't
@@ -371,8 +365,6 @@ StrategyFreeBuffer(BufferDesc *buf)
 	if (buf->freeNext == FREENEXT_NOT_IN_LIST)
 	{
 		buf->freeNext = StrategyControl->firstFreeBuffer;
-		if (buf->freeNext < 0)
-			StrategyControl->lastFreeBuffer = buf->buf_id;
 		StrategyControl->firstFreeBuffer = buf->buf_id;
 	}
 
@@ -509,7 +501,6 @@ StrategyInitialize(bool init)
 		 * assume it was previously set up by BufferManagerShmemInit().
 		 */
 		StrategyControl->firstFreeBuffer = 0;
-		StrategyControl->lastFreeBuffer = NBuffers - 1;
 
 		/* Initialize the clock sweep pointer */
 		pg_atomic_init_u32(&StrategyControl->nextVictimBuffer, 0);
-- 
2.49.0

From 6505848ac8359c8c76dfbffc7150b6601ab07601 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Thu, 22 May 2025 18:38:41 +0200
Subject: [PATCH v1 4/6] NUMA: partition buffer freelist

Instead of a single buffer freelist, partition into multiple smaller
lists, to reduce lock contention, and to spread the buffers over all
NUMA nodes more evenly.

There are four strategies, specified by GUC numa_partition_freelist

* none - single long freelist, should work just like now

* node - one freelist per NUMA node, with only buffers from that node

* cpu - one freelist per CPU

* pid - freelist determined by PID (same number of freelists as 'cpu')

When allocating a buffer, it's taken from the correct freelist (e.g.
same NUMA node).

Note: This is (probably) more important than partitioning ProcArray.
---
 src/backend/storage/buffer/buf_init.c |   4 +-
 src/backend/storage/buffer/freelist.c | 324 +++++++++++++++++++++++---
 src/backend/utils/init/globals.c      |   1 +
 src/backend/utils/misc/guc_tables.c   |  18 ++
 src/include/miscadmin.h               |   1 +
 src/include/storage/bufmgr.h          |   8 +
 6 files changed, 327 insertions(+), 29 deletions(-)

diff --git a/src/backend/storage/buffer/buf_init.c b/src/backend/storage/buffer/buf_init.c
index 2ad34624c49..920f1a32a8f 100644
--- a/src/backend/storage/buffer/buf_init.c
+++ b/src/backend/storage/buffer/buf_init.c
@@ -543,8 +543,8 @@ pg_numa_interleave_memory(char *startptr, char *endptr,
 		 * XXX no return value, to make this fail on error, has to use
 		 * numa_set_strict
 		 *
-		 * XXX Should we still touch the memory first, like with numa_move_pages,
-		 * or is that not necessary?
+		 * XXX Should we still touch the memory first, like with
+		 * numa_move_pages, or is that not necessary?
 		 */
 		numa_tonode_memory(ptr, sz, node);
 
diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index e046526c149..c93ec2841c5 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -15,14 +15,41 @@
  */
 #include "postgres.h"
 
+#include <sched.h>
+#include <sys/sysinfo.h>
+
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
 #include "pgstat.h"
 #include "port/atomics.h"
 #include "storage/buf_internals.h"
 #include "storage/bufmgr.h"
+#include "storage/ipc.h"
 #include "storage/proc.h"
 
 #define INT_ACCESS_ONCE(var)	((int)(*((volatile int *)&(var))))
 
+/*
+ * Represents one freelist partition.
+ */
+typedef struct BufferStrategyFreelist
+{
+	/* Spinlock: protects the values below */
+	slock_t		freelist_lock;
+
+	/*
+	 * XXX Not sure why this needs to be aligned like this. Need to ask
+	 * Andres.
+	 */
+	int			firstFreeBuffer __attribute__((aligned(64)));	/* Head of list of
+																 * unused buffers */
+
+	/* Number of buffers consumed from this list. */
+	uint64		consumed;
+}			BufferStrategyFreelist;
 
 /*
  * The shared freelist control information.
@@ -39,8 +66,6 @@ typedef struct
 	 */
 	pg_atomic_uint32 nextVictimBuffer;
 
-	int			firstFreeBuffer;	/* Head of list of unused buffers */
-
 	/*
 	 * Statistics.  These counters should be wide enough that they can't
 	 * overflow during a single bgwriter cycle.
@@ -51,13 +76,27 @@ typedef struct
 	/*
 	 * Bgworker process to be notified upon activity or -1 if none. See
 	 * StrategyNotifyBgWriter.
+	 *
+	 * XXX Not sure why this needs to be aligned like this. Need to ask
+	 * Andres. Also, shouldn't the alignment be specified after, like for
+	 * "consumed"?
 	 */
-	int			bgwprocno;
+	int			__attribute__((aligned(64))) bgwprocno;
+
+	BufferStrategyFreelist freelists[FLEXIBLE_ARRAY_MEMBER];
 } BufferStrategyControl;
 
 /* Pointers to shared state */
 static BufferStrategyControl *StrategyControl = NULL;
 
+/*
+ * XXX shouldn't this be in BufferStrategyControl? Probably not, we need to
+ * calculate it during sizing, and perhaps it could change before the memory
+ * gets allocated (so we need to remember the values).
+ */
+static int	strategy_nnodes;
+static int	strategy_ncpus;
+
 /*
  * Private (non-shared) state for managing a ring of shared buffers to re-use.
  * This is currently the only kind of BufferAccessStrategy object, but someday
@@ -157,6 +196,90 @@ ClockSweepTick(void)
 	return victim;
 }
 
+/*
+ * ChooseFreeList
+ *		Pick the buffer freelist to use, depending on the CPU and NUMA node.
+ *
+ * Without partitioned freelists (numa_partition_freelist=false), there's only
+ * a single freelist, so use that.
+ *
+ * With partitioned freelists, we have multiple ways how to pick the freelist
+ * for the backend:
+ *
+ * - one freelist per CPU, use the freelist for CPU the task executes on
+ *
+ * - one freelist per NUMA node, use the freelist for node task executes on
+ *
+ * - use fixed number of freelists, map processes to lists based on PID
+ *
+ * There may be some other strategies, not sure. The important thing is this
+ * needs to be refrecled during initialization, i.e. we need to create the
+ * right number of lists.
+ */
+static BufferStrategyFreelist *
+ChooseFreeList(void)
+{
+	unsigned	cpu;
+	unsigned	node;
+	int			rc;
+
+	int			freelist_idx;
+
+	/* freelist not partitioned, return the first (and only) freelist */
+	if (numa_partition_freelist == FREELIST_PARTITION_NONE)
+		return &StrategyControl->freelists[0];
+
+	/*
+	 * freelist is partitioned, so determine the CPU/NUMA node, and pick a
+	 * list based on that.
+	 */
+	rc = getcpu(&cpu, &node);
+	if (rc != 0)
+		elog(ERROR, "getcpu failed: %m");
+
+	/*
+	 * FIXME This doesn't work well if CPUs are excluded from being run or
+	 * offline. In that case we end up not using some freelists at all, but
+	 * not sure if we need to worry about that. Probably not for now. But
+	 * could that change while the system is running?
+	 *
+	 * XXX Maybe we should somehow detect changes to the list of CPUs, and
+	 * rebuild the lists if that changes? But that seems expensive.
+	 */
+	if (cpu > strategy_ncpus)
+		elog(ERROR, "cpu out of range: %d > %u", cpu, strategy_ncpus);
+	else if (node > strategy_nnodes)
+		elog(ERROR, "node out of range: %d > %u", cpu, strategy_nnodes);
+
+	/*
+	 * Pick the freelist, based on CPU, NUMA node or process PID. This matches
+	 * how we built the freelists above.
+	 *
+	 * XXX Can we rely on some of the values (especially strategy_nnodes) to
+	 * be a power-of-2? Then we could replace the modulo with a mask, which is
+	 * likely more efficient.
+	 */
+	switch (numa_partition_freelist)
+	{
+		case FREELIST_PARTITION_CPU:
+			freelist_idx = cpu % strategy_ncpus;
+			break;
+
+		case FREELIST_PARTITION_NODE:
+			freelist_idx = node % strategy_nnodes;
+			break;
+
+		case FREELIST_PARTITION_PID:
+			freelist_idx = MyProcPid % strategy_ncpus;
+			break;
+
+		default:
+			elog(ERROR, "unknown freelist partitioning value");
+	}
+
+	return &StrategyControl->freelists[freelist_idx];
+}
+
 /*
  * have_free_buffer -- a lockless check to see if there is a free buffer in
  *					   buffer pool.
@@ -168,10 +291,13 @@ ClockSweepTick(void)
 bool
 have_free_buffer(void)
 {
-	if (StrategyControl->firstFreeBuffer >= 0)
-		return true;
-	else
-		return false;
+	for (int i = 0; i < strategy_ncpus; i++)
+	{
+		if (StrategyControl->freelists[i].firstFreeBuffer >= 0)
+			return true;
+	}
+
+	return false;
 }
 
 /*
@@ -193,6 +319,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 	int			bgwprocno;
 	int			trycounter;
 	uint32		local_buf_state;	/* to avoid repeated (de-)referencing */
+	BufferStrategyFreelist *freelist;
 
 	*from_ring = false;
 
@@ -259,31 +386,35 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 	 * buffer_strategy_lock not the individual buffer spinlocks, so it's OK to
 	 * manipulate them without holding the spinlock.
 	 */
-	if (StrategyControl->firstFreeBuffer >= 0)
+	freelist = ChooseFreeList();
+	if (freelist->firstFreeBuffer >= 0)
 	{
 		while (true)
 		{
 			/* Acquire the spinlock to remove element from the freelist */
-			SpinLockAcquire(&StrategyControl->buffer_strategy_lock);
+			SpinLockAcquire(&freelist->freelist_lock);
 
-			if (StrategyControl->firstFreeBuffer < 0)
+			if (freelist->firstFreeBuffer < 0)
 			{
-				SpinLockRelease(&StrategyControl->buffer_strategy_lock);
+				SpinLockRelease(&freelist->freelist_lock);
 				break;
 			}
 
-			buf = GetBufferDescriptor(StrategyControl->firstFreeBuffer);
+			buf = GetBufferDescriptor(freelist->firstFreeBuffer);
 			Assert(buf->freeNext != FREENEXT_NOT_IN_LIST);
 
 			/* Unconditionally remove buffer from freelist */
-			StrategyControl->firstFreeBuffer = buf->freeNext;
+			freelist->firstFreeBuffer = buf->freeNext;
 			buf->freeNext = FREENEXT_NOT_IN_LIST;
 
+			/* increment number of buffers we consumed from this list */
+			freelist->consumed++;
+
 			/*
 			 * Release the lock so someone else can access the freelist while
 			 * we check out this buffer.
 			 */
-			SpinLockRelease(&StrategyControl->buffer_strategy_lock);
+			SpinLockRelease(&freelist->freelist_lock);
 
 			/*
 			 * If the buffer is pinned or has a nonzero usage_count, we cannot
@@ -305,7 +436,17 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 		}
 	}
 
-	/* Nothing on the freelist, so run the "clock sweep" algorithm */
+	/*
+	 * Nothing on the freelist, so run the "clock sweep" algorithm
+	 *
+	 * XXX Should we also make this NUMA-aware, to only access buffers from
+	 * the same NUMA node? That'd probably mean we need to make the clock
+	 * sweep NUMA-aware, perhaps by having multiple clock sweeps, each for a
+	 * subset of buffers. But that also means each process could "sweep" only
+	 * a fraction of buffers, even if the other buffers are better candidates
+	 * for eviction. Would that also mean we'd have multiple bgwriters, one
+	 * for each node, or would one bgwriter handle all of that?
+	 */
 	trycounter = NBuffers;
 	for (;;)
 	{
@@ -352,11 +493,22 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 
 /*
  * StrategyFreeBuffer: put a buffer on the freelist
+ *
+ * XXX This calls ChooseFreeList() again, and it might return the freelist to
+ * a different freelist than it was taken from (either by a different backend,
+ * or perhaps even the same backend running on a different CPU). Is that good?
+ * Maybe we should try to balance this somehow, e.g. by choosing a random list,
+ * the shortest one, or something like that? But that breaks the whole idea of
+ * having freelists with buffers from a particular NUMA node.
  */
 void
 StrategyFreeBuffer(BufferDesc *buf)
 {
-	SpinLockAcquire(&StrategyControl->buffer_strategy_lock);
+	BufferStrategyFreelist *freelist;
+
+	freelist = ChooseFreeList();
+
+	SpinLockAcquire(&freelist->freelist_lock);
 
 	/*
 	 * It is possible that we are told to put something in the freelist that
@@ -364,11 +516,11 @@ StrategyFreeBuffer(BufferDesc *buf)
 	 */
 	if (buf->freeNext == FREENEXT_NOT_IN_LIST)
 	{
-		buf->freeNext = StrategyControl->firstFreeBuffer;
-		StrategyControl->firstFreeBuffer = buf->buf_id;
+		buf->freeNext = freelist->firstFreeBuffer;
+		freelist->firstFreeBuffer = buf->buf_id;
 	}
 
-	SpinLockRelease(&StrategyControl->buffer_strategy_lock);
+	SpinLockRelease(&freelist->freelist_lock);
 }
 
 /*
@@ -432,6 +584,42 @@ StrategyNotifyBgWriter(int bgwprocno)
 	SpinLockRelease(&StrategyControl->buffer_strategy_lock);
 }
 
+/* prints some debug info / stats about freelists at shutdown */
+static void
+freelist_before_shmem_exit(int code, Datum arg)
+{
+	for (int node = 0; node < strategy_ncpus; node++)
+	{
+		BufferStrategyFreelist *freelist = &StrategyControl->freelists[node];
+		uint64		remain = 0;
+		uint64		actually_free = 0;
+		int			cur = freelist->firstFreeBuffer;
+
+		while (cur >= 0)
+		{
+			uint32		local_buf_state;
+			BufferDesc *buf;
+
+			buf = GetBufferDescriptor(cur);
+
+			remain++;
+
+			local_buf_state = LockBufHdr(buf);
+
+			if (!(local_buf_state & BM_TAG_VALID))
+				actually_free++;
+
+			UnlockBufHdr(buf, local_buf_state);
+
+			cur = buf->freeNext;
+		}
+		elog(LOG, "freelist %d, firstF: %d: consumed: %lu, remain: %lu, actually free: %lu",
+			 node,
+			 freelist->firstFreeBuffer,
+			 freelist->consumed,
+			 remain, actually_free);
+	}
+}
 
 /*
  * StrategyShmemSize
@@ -446,11 +634,33 @@ StrategyShmemSize(void)
 {
 	Size		size = 0;
 
+	/* FIXME */
+#ifdef USE_LIBNUMA
+	strategy_ncpus = numa_num_task_cpus();
+	strategy_nnodes = numa_num_task_nodes();
+#else
+	strategy_ncpus = 1;
+	strategy_nnodes = 1;
+#endif
+
+	Assert(strategy_nnodes <= strategy_ncpus);
+
 	/* size of lookup hash table ... see comment in StrategyInitialize */
 	size = add_size(size, BufTableShmemSize(NBuffers + NUM_BUFFER_PARTITIONS));
 
 	/* size of the shared replacement strategy control block */
-	size = add_size(size, MAXALIGN(sizeof(BufferStrategyControl)));
+	size = add_size(size, MAXALIGN(offsetof(BufferStrategyControl, freelists)));
+
+	/*
+	 * Allocate one frelist per CPU. We might use per-node freelists, but the
+	 * assumption is the number of CPUs is less than number of NUMA nodes.
+	 *
+	 * FIXME This assumes the we have more CPUs than NUMA nodes, which seems
+	 * like a safe assumption. But maybe we should calculate how many elements
+	 * we actually need, depending on the GUC? Not a huge amount of memory.
+	 */
+	size = add_size(size, MAXALIGN(mul_size(sizeof(BufferStrategyFreelist),
+											strategy_ncpus)));
 
 	return size;
 }
@@ -466,6 +676,7 @@ void
 StrategyInitialize(bool init)
 {
 	bool		found;
+	int			buffers_per_cpu;
 
 	/*
 	 * Initialize the shared buffer lookup hashtable.
@@ -484,23 +695,27 @@ StrategyInitialize(bool init)
 	 */
 	StrategyControl = (BufferStrategyControl *)
 		ShmemInitStruct("Buffer Strategy Status",
-						sizeof(BufferStrategyControl),
+						offsetof(BufferStrategyControl, freelists) +
+						(sizeof(BufferStrategyFreelist) * strategy_ncpus),
 						&found);
 
 	if (!found)
 	{
+		/*
+		 * XXX Calling get_nprocs() may not be quite correct, because some of
+		 * the processors may get disabled, etc.
+		 */
+		int			num_cpus = get_nprocs();
+
 		/*
 		 * Only done once, usually in postmaster
 		 */
 		Assert(init);
 
-		SpinLockInit(&StrategyControl->buffer_strategy_lock);
+		/* register callback to dump some stats on exit */
+		before_shmem_exit(freelist_before_shmem_exit, 0);
 
-		/*
-		 * Grab the whole linked list of free buffers for our strategy. We
-		 * assume it was previously set up by BufferManagerShmemInit().
-		 */
-		StrategyControl->firstFreeBuffer = 0;
+		SpinLockInit(&StrategyControl->buffer_strategy_lock);
 
 		/* Initialize the clock sweep pointer */
 		pg_atomic_init_u32(&StrategyControl->nextVictimBuffer, 0);
@@ -511,6 +726,61 @@ StrategyInitialize(bool init)
 
 		/* No pending notification */
 		StrategyControl->bgwprocno = -1;
+
+		/*
+		 * Rebuild the freelist - right now all buffers are in one huge list,
+		 * we want to rework that into multiple lists. Start by initializing
+		 * the strategy to have empty lists.
+		 */
+		for (int nfreelist = 0; nfreelist < strategy_ncpus; nfreelist++)
+		{
+			BufferStrategyFreelist *freelist;
+
+			freelist = &StrategyControl->freelists[nfreelist];
+
+			freelist->firstFreeBuffer = FREENEXT_END_OF_LIST;
+
+			SpinLockInit(&freelist->freelist_lock);
+		}
+
+		/* buffers per CPU (also used for PID partitioning) */
+		buffers_per_cpu = (NBuffers / strategy_ncpus);
+
+		elog(LOG, "NBuffers: %d, nodes %d, ncpus: %d, divide: %d, remain: %d",
+			 NBuffers, strategy_nnodes, strategy_ncpus,
+			 buffers_per_cpu, NBuffers - (strategy_ncpus * buffers_per_cpu));
+
+		/*
+		 * Walk through the buffers, add them to the correct list. Walk from
+		 * the end, because we're adding the buffers to the beginning.
+		 */
+		for (int i = NBuffers - 1; i >= 0; i--)
+		{
+			BufferDesc *buf = GetBufferDescriptor(i);
+			BufferStrategyFreelist *freelist;
+			int			belongs_to = 0; /* first freelist by default */
+
+			/*
+			 * Split the freelist into partitions, if needed (or just keep the
+			 * freelist we already built in BufferManagerShmemInit().
+			 */
+			if ((numa_partition_freelist == FREELIST_PARTITION_CPU) ||
+				(numa_partition_freelist == FREELIST_PARTITION_PID))
+			{
+				belongs_to = (i % num_cpus);
+			}
+			else if (numa_partition_freelist == FREELIST_PARTITION_NODE)
+			{
+				/* determine NUMA node for buffer */
+				belongs_to = BufferGetNode(i);
+			}
+
+			/* add to the right freelist */
+			freelist = &StrategyControl->freelists[belongs_to];
+
+			buf->freeNext = freelist->firstFreeBuffer;
+			freelist->firstFreeBuffer = i;
+		}
 	}
 	else
 		Assert(!init);
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index f5359db3656..7febf3001a3 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -148,6 +148,7 @@ int			MaxBackends = 0;
 /* NUMA stuff */
 bool		numa_buffers_interleave = false;
 bool		numa_localalloc = false;
+int			numa_partition_freelist = 0;
 
 /* GUC parameters for vacuum */
 int			VacuumBufferUsageLimit = 2048;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 57f2df7ab74..e2361c161e6 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -491,6 +491,14 @@ static const struct config_enum_entry file_copy_method_options[] = {
 	{NULL, 0, false}
 };
 
+static const struct config_enum_entry freelist_partition_options[] = {
+	{"none", FREELIST_PARTITION_NONE, false},
+	{"node", FREELIST_PARTITION_NODE, false},
+	{"cpu", FREELIST_PARTITION_CPU, false},
+	{"pid", FREELIST_PARTITION_PID, false},
+	{NULL, 0, false}
+};
+
 /*
  * Options for enum values stored in other modules
  */
@@ -5284,6 +5292,16 @@ struct config_enum ConfigureNamesEnum[] =
 		NULL, NULL, NULL
 	},
 
+	{
+		{"numa_partition_freelist", PGC_USERSET, DEVELOPER_OPTIONS,
+			gettext_noop("Enables buffer freelists to be partitioned per NUMA node."),
+			gettext_noop("When enabled, we create a separate freelist per NUMA node."),
+		},
+		&numa_partition_freelist,
+		FREELIST_PARTITION_NONE, freelist_partition_options,
+		NULL, NULL, NULL
+	},
+
 	{
 		{"wal_sync_method", PGC_SIGHUP, WAL_SETTINGS,
 			gettext_noop("Selects the method used for forcing WAL updates to disk."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 692871a401f..17528439f07 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -180,6 +180,7 @@ extern PGDLLIMPORT int max_parallel_workers;
 
 extern PGDLLIMPORT bool numa_buffers_interleave;
 extern PGDLLIMPORT bool numa_localalloc;
+extern PGDLLIMPORT int numa_partition_freelist;
 
 extern PGDLLIMPORT int commit_timestamp_buffers;
 extern PGDLLIMPORT int multixact_member_buffers;
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index c257c8a1c20..efb7e28c10f 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -93,6 +93,14 @@ typedef enum ExtendBufferedFlags
 	EB_LOCK_TARGET = (1 << 5),
 }			ExtendBufferedFlags;
 
+typedef enum FreelistPartitionMode
+{
+	FREELIST_PARTITION_NONE,
+	FREELIST_PARTITION_NODE,
+	FREELIST_PARTITION_CPU,
+	FREELIST_PARTITION_PID,
+}			FreelistPartitionMode;
+
 /*
  * Some functions identify relations either by relation or smgr +
  * relpersistence.  Used via the BMR_REL()/BMR_SMGR() macros below.  This
-- 
2.49.0

From 05c594ed8eb8a266a74038c3131d12bb03d897e3 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Thu, 22 May 2025 18:39:08 +0200
Subject: [PATCH v1 5/6] NUMA: interleave PGPROC entries

The goal is to distribute ProcArray (or rather PGPROC entries and
associated fast-path arrays) to NUMA nodes.

We can't do this by simply interleaving pages, because that wouldn't
work for both parts at the same time. We want to place the PGPROC and
it's fast-path locking structs on the same node, but the structs are
of different sizes, etc.

Another problem is that PGPROC entries are fairly small, so with huge
pages and reasonable values of max_connections everything fits onto a
single page. We don't want to make this incompatible with huge pages.

Note: If we eventually switch to allocating separate shared segments for
different parts (to allow on-line resizing), we could keep using regular
pages for procarray, and this would not be such an issue.

To make this work, we split the PGPROC array into per-node segments,
each with about (MaxBackends / numa_nodes) entries, and one segment for
auxiliary processes and prepared transations. And we do the same thing
for fast-path arrays.

The PGPROC segments are laid out like this (e.g. for 2 NUMA nodes):

 - PGPROC array / node #0
 - PGPROC array / node #1
 - PGPROC array / aux processes + 2PC transactions
 - fast-path arrays / node #0
 - fast-path arrays / node #1
 - fast-path arrays / aux processes + 2PC transaction

Each segment is aligned to (starts at) memory page, and is effectively a
multiple of multiple memory pages.

Having a single PGPROC array made certain operations easiers - e.g. it
was possible to iterate the array, and GetNumberFromPGProc() could
calculate offset by simply subtracting PGPROC pointers. With multiple
segments that's not possible, but the fallout is minimal.

Most places accessed PGPROC through PROC_HDR->allProcs, and can continue
to do so, except that now they get a pointer to the PGPROC (which most
places wanted anyway).

Note: There's an indirection, though. But the pointer does not change,
so hopefully that's not an issue. And each PGPROC entry gets an explicit
procnumber field, which is the index in allProcs, GetNumberFromPGProc
can simply return that.

Each PGPROC also gets numa_node, tracking the NUMA node, so that we
don't have to recalculate that. This is used by InitProcess() to pick
a PGPROC entry from the local NUMA node.

Note: The scheduler may migrate the process to a different CPU/node
later. Maybe we should consider pinning the process to the node?
---
 src/backend/access/transam/clog.c      |   4 +-
 src/backend/postmaster/pgarch.c        |   2 +-
 src/backend/postmaster/walsummarizer.c |   2 +-
 src/backend/storage/buffer/freelist.c  |   2 +-
 src/backend/storage/ipc/procarray.c    |  62 ++---
 src/backend/storage/lmgr/lock.c        |   6 +-
 src/backend/storage/lmgr/proc.c        | 368 +++++++++++++++++++++++--
 src/backend/utils/init/globals.c       |   1 +
 src/backend/utils/misc/guc_tables.c    |  10 +
 src/include/miscadmin.h                |   1 +
 src/include/storage/proc.h             |  11 +-
 11 files changed, 407 insertions(+), 62 deletions(-)

diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index 48f10bec91e..90ddff37bc6 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -576,7 +576,7 @@ TransactionGroupUpdateXidStatus(TransactionId xid, XidStatus status,
 	/* Walk the list and update the status of all XIDs. */
 	while (nextidx != INVALID_PROC_NUMBER)
 	{
-		PGPROC	   *nextproc = &ProcGlobal->allProcs[nextidx];
+		PGPROC	   *nextproc = ProcGlobal->allProcs[nextidx];
 		int64		thispageno = nextproc->clogGroupMemberPage;
 
 		/*
@@ -635,7 +635,7 @@ TransactionGroupUpdateXidStatus(TransactionId xid, XidStatus status,
 	 */
 	while (wakeidx != INVALID_PROC_NUMBER)
 	{
-		PGPROC	   *wakeproc = &ProcGlobal->allProcs[wakeidx];
+		PGPROC	   *wakeproc = ProcGlobal->allProcs[wakeidx];
 
 		wakeidx = pg_atomic_read_u32(&wakeproc->clogGroupNext);
 		pg_atomic_write_u32(&wakeproc->clogGroupNext, INVALID_PROC_NUMBER);
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 7e622ae4bd2..75c0e4bf53c 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -289,7 +289,7 @@ PgArchWakeup(void)
 	 * be relaunched shortly and will start archiving.
 	 */
 	if (arch_pgprocno != INVALID_PROC_NUMBER)
-		SetLatch(&ProcGlobal->allProcs[arch_pgprocno].procLatch);
+		SetLatch(&ProcGlobal->allProcs[arch_pgprocno]->procLatch);
 }

diff --git a/src/backend/postmaster/walsummarizer.c b/src/backend/postmaster/walsummarizer.c
index 0fec4f1f871..0044ef54363 100644
--- a/src/backend/postmaster/walsummarizer.c
+++ b/src/backend/postmaster/walsummarizer.c
@@ -649,7 +649,7 @@ WakeupWalSummarizer(void)
 	LWLockRelease(WALSummarizerLock);
 
 	if (pgprocno != INVALID_PROC_NUMBER)
-		SetLatch(&ProcGlobal->allProcs[pgprocno].procLatch);
+		SetLatch(&ProcGlobal->allProcs[pgprocno]->procLatch);
 }
 
 /*
diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index c93ec2841c5..4e390a77a71 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -360,7 +360,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 		 * actually fine because procLatch isn't ever freed, so we just can
 		 * potentially set the wrong process' (or no process') latch.
 		 */
-		SetLatch(&ProcGlobal->allProcs[bgwprocno].procLatch);
+		SetLatch(&ProcGlobal->allProcs[bgwprocno]->procLatch);
 	}
 
 	/*
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index e5b945a9ee3..3277480fbcf 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -268,7 +268,7 @@ typedef enum KAXCompressReason
 
 static ProcArrayStruct *procArray;
 
-static PGPROC *allProcs;
+static PGPROC **allProcs;
 
 /*
  * Cache to reduce overhead of repeated calls to TransactionIdIsInProgress()
@@ -502,7 +502,7 @@ ProcArrayAdd(PGPROC *proc)
 		int			this_procno = arrayP->pgprocnos[index];
 
 		Assert(this_procno >= 0 && this_procno < (arrayP->maxProcs + NUM_AUXILIARY_PROCS));
-		Assert(allProcs[this_procno].pgxactoff == index);
+		Assert(allProcs[this_procno]->pgxactoff == index);
 
 		/* If we have found our right position in the array, break */
 		if (this_procno > pgprocno)
@@ -538,9 +538,9 @@ ProcArrayAdd(PGPROC *proc)
 		int			procno = arrayP->pgprocnos[index];
 
 		Assert(procno >= 0 && procno < (arrayP->maxProcs + NUM_AUXILIARY_PROCS));
-		Assert(allProcs[procno].pgxactoff == index - 1);
+		Assert(allProcs[procno]->pgxactoff == index - 1);
 
-		allProcs[procno].pgxactoff = index;
+		allProcs[procno]->pgxactoff = index;
 	}
 
 	/*
@@ -581,7 +581,7 @@ ProcArrayRemove(PGPROC *proc, TransactionId latestXid)
 	myoff = proc->pgxactoff;
 
 	Assert(myoff >= 0 && myoff < arrayP->numProcs);
-	Assert(ProcGlobal->allProcs[arrayP->pgprocnos[myoff]].pgxactoff == myoff);
+	Assert(ProcGlobal->allProcs[arrayP->pgprocnos[myoff]]->pgxactoff == myoff);
 
 	if (TransactionIdIsValid(latestXid))
 	{
@@ -636,9 +636,9 @@ ProcArrayRemove(PGPROC *proc, TransactionId latestXid)
 		int			procno = arrayP->pgprocnos[index];
 
 		Assert(procno >= 0 && procno < (arrayP->maxProcs + NUM_AUXILIARY_PROCS));
-		Assert(allProcs[procno].pgxactoff - 1 == index);
+		Assert(allProcs[procno]->pgxactoff - 1 == index);
 
-		allProcs[procno].pgxactoff = index;
+		allProcs[procno]->pgxactoff = index;
 	}
 
 	/*
@@ -860,7 +860,7 @@ ProcArrayGroupClearXid(PGPROC *proc, TransactionId latestXid)
 	/* Walk the list and clear all XIDs. */
 	while (nextidx != INVALID_PROC_NUMBER)
 	{
-		PGPROC	   *nextproc = &allProcs[nextidx];
+		PGPROC	   *nextproc = allProcs[nextidx];
 
 		ProcArrayEndTransactionInternal(nextproc, nextproc->procArrayGroupMemberXid);
 
@@ -880,7 +880,7 @@ ProcArrayGroupClearXid(PGPROC *proc, TransactionId latestXid)
 	 */
 	while (wakeidx != INVALID_PROC_NUMBER)
 	{
-		PGPROC	   *nextproc = &allProcs[wakeidx];
+		PGPROC	   *nextproc = allProcs[wakeidx];
 
 		wakeidx = pg_atomic_read_u32(&nextproc->procArrayGroupNext);
 		pg_atomic_write_u32(&nextproc->procArrayGroupNext, INVALID_PROC_NUMBER);
@@ -1526,7 +1526,7 @@ TransactionIdIsInProgress(TransactionId xid)
 		pxids = other_subxidstates[pgxactoff].count;
 		pg_read_barrier();		/* pairs with barrier in GetNewTransactionId() */
 		pgprocno = arrayP->pgprocnos[pgxactoff];
-		proc = &allProcs[pgprocno];
+		proc = allProcs[pgprocno];
 		for (j = pxids - 1; j >= 0; j--)
 		{
 			/* Fetch xid just once - see GetNewTransactionId */
@@ -1650,7 +1650,7 @@ TransactionIdIsActive(TransactionId xid)
 	for (i = 0; i < arrayP->numProcs; i++)
 	{
 		int			pgprocno = arrayP->pgprocnos[i];
-		PGPROC	   *proc = &allProcs[pgprocno];
+		PGPROC	   *proc = allProcs[pgprocno];
 		TransactionId pxid;
 
 		/* Fetch xid just once - see GetNewTransactionId */
@@ -1792,7 +1792,7 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h)
 	for (int index = 0; index < arrayP->numProcs; index++)
 	{
 		int			pgprocno = arrayP->pgprocnos[index];
-		PGPROC	   *proc = &allProcs[pgprocno];
+		PGPROC	   *proc = allProcs[pgprocno];
 		int8		statusFlags = ProcGlobal->statusFlags[index];
 		TransactionId xid;
 		TransactionId xmin;
@@ -2276,7 +2276,7 @@ GetSnapshotData(Snapshot snapshot)
 			TransactionId xid = UINT32_ACCESS_ONCE(other_xids[pgxactoff]);
 			uint8		statusFlags;
 
-			Assert(allProcs[arrayP->pgprocnos[pgxactoff]].pgxactoff == pgxactoff);
+			Assert(allProcs[arrayP->pgprocnos[pgxactoff]]->pgxactoff == pgxactoff);
 
 			/*
 			 * If the transaction has no XID assigned, we can skip it; it
@@ -2350,7 +2350,7 @@ GetSnapshotData(Snapshot snapshot)
 					if (nsubxids > 0)
 					{
 						int			pgprocno = pgprocnos[pgxactoff];
-						PGPROC	   *proc = &allProcs[pgprocno];
+						PGPROC	   *proc = allProcs[pgprocno];
 
 						pg_read_barrier();	/* pairs with GetNewTransactionId */
 
@@ -2551,7 +2551,7 @@ ProcArrayInstallImportedXmin(TransactionId xmin,
 	for (index = 0; index < arrayP->numProcs; index++)
 	{
 		int			pgprocno = arrayP->pgprocnos[index];
-		PGPROC	   *proc = &allProcs[pgprocno];
+		PGPROC	   *proc = allProcs[pgprocno];
 		int			statusFlags = ProcGlobal->statusFlags[index];
 		TransactionId xid;
 
@@ -2777,7 +2777,7 @@ GetRunningTransactionData(void)
 		if (TransactionIdPrecedes(xid, oldestDatabaseRunningXid))
 		{
 			int			pgprocno = arrayP->pgprocnos[index];
-			PGPROC	   *proc = &allProcs[pgprocno];
+			PGPROC	   *proc = allProcs[pgprocno];
 
 			if (proc->databaseId == MyDatabaseId)
 				oldestDatabaseRunningXid = xid;
@@ -2808,7 +2808,7 @@ GetRunningTransactionData(void)
 		for (index = 0; index < arrayP->numProcs; index++)
 		{
 			int			pgprocno = arrayP->pgprocnos[index];
-			PGPROC	   *proc = &allProcs[pgprocno];
+			PGPROC	   *proc = allProcs[pgprocno];
 			int			nsubxids;
 
 			/*
@@ -3058,7 +3058,7 @@ GetVirtualXIDsDelayingChkpt(int *nvxids, int type)
 	for (index = 0; index < arrayP->numProcs; index++)
 	{
 		int			pgprocno = arrayP->pgprocnos[index];
-		PGPROC	   *proc = &allProcs[pgprocno];
+		PGPROC	   *proc = allProcs[pgprocno];
 
 		if ((proc->delayChkptFlags & type) != 0)
 		{
@@ -3099,7 +3099,7 @@ HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids, int type)
 	for (index = 0; index < arrayP->numProcs; index++)
 	{
 		int			pgprocno = arrayP->pgprocnos[index];
-		PGPROC	   *proc = &allProcs[pgprocno];
+		PGPROC	   *proc = allProcs[pgprocno];
 		VirtualTransactionId vxid;
 
 		GET_VXID_FROM_PGPROC(vxid, *proc);
@@ -3227,7 +3227,7 @@ BackendPidGetProcWithLock(int pid)
 
 	for (index = 0; index < arrayP->numProcs; index++)
 	{
-		PGPROC	   *proc = &allProcs[arrayP->pgprocnos[index]];
+		PGPROC	   *proc = allProcs[arrayP->pgprocnos[index]];
 
 		if (proc->pid == pid)
 		{
@@ -3270,7 +3270,7 @@ BackendXidGetPid(TransactionId xid)
 		if (other_xids[index] == xid)
 		{
 			int			pgprocno = arrayP->pgprocnos[index];
-			PGPROC	   *proc = &allProcs[pgprocno];
+			PGPROC	   *proc = allProcs[pgprocno];
 
 			result = proc->pid;
 			break;
@@ -3339,7 +3339,7 @@ GetCurrentVirtualXIDs(TransactionId limitXmin, bool excludeXmin0,
 	for (index = 0; index < arrayP->numProcs; index++)
 	{
 		int			pgprocno = arrayP->pgprocnos[index];
-		PGPROC	   *proc = &allProcs[pgprocno];
+		PGPROC	   *proc = allProcs[pgprocno];
 		uint8		statusFlags = ProcGlobal->statusFlags[index];
 
 		if (proc == MyProc)
@@ -3441,7 +3441,7 @@ GetConflictingVirtualXIDs(TransactionId limitXmin, Oid dbOid)
 	for (index = 0; index < arrayP->numProcs; index++)
 	{
 		int			pgprocno = arrayP->pgprocnos[index];
-		PGPROC	   *proc = &allProcs[pgprocno];
+		PGPROC	   *proc = allProcs[pgprocno];
 
 		/* Exclude prepared transactions */
 		if (proc->pid == 0)
@@ -3506,7 +3506,7 @@ SignalVirtualTransaction(VirtualTransactionId vxid, ProcSignalReason sigmode,
 	for (index = 0; index < arrayP->numProcs; index++)
 	{
 		int			pgprocno = arrayP->pgprocnos[index];
-		PGPROC	   *proc = &allProcs[pgprocno];
+		PGPROC	   *proc = allProcs[pgprocno];
 		VirtualTransactionId procvxid;
 
 		GET_VXID_FROM_PGPROC(procvxid, *proc);
@@ -3561,7 +3561,7 @@ MinimumActiveBackends(int min)
 	for (index = 0; index < arrayP->numProcs; index++)
 	{
 		int			pgprocno = arrayP->pgprocnos[index];
-		PGPROC	   *proc = &allProcs[pgprocno];
+		PGPROC	   *proc = allProcs[pgprocno];
 
 		/*
 		 * Since we're not holding a lock, need to be prepared to deal with
@@ -3607,7 +3607,7 @@ CountDBBackends(Oid databaseid)
 	for (index = 0; index < arrayP->numProcs; index++)
 	{
 		int			pgprocno = arrayP->pgprocnos[index];
-		PGPROC	   *proc = &allProcs[pgprocno];
+		PGPROC	   *proc = allProcs[pgprocno];
 
 		if (proc->pid == 0)
 			continue;			/* do not count prepared xacts */
@@ -3636,7 +3636,7 @@ CountDBConnections(Oid databaseid)
 	for (index = 0; index < arrayP->numProcs; index++)
 	{
 		int			pgprocno = arrayP->pgprocnos[index];
-		PGPROC	   *proc = &allProcs[pgprocno];
+		PGPROC	   *proc = allProcs[pgprocno];
 
 		if (proc->pid == 0)
 			continue;			/* do not count prepared xacts */
@@ -3667,7 +3667,7 @@ CancelDBBackends(Oid databaseid, ProcSignalReason sigmode, bool conflictPending)
 	for (index = 0; index < arrayP->numProcs; index++)
 	{
 		int			pgprocno = arrayP->pgprocnos[index];
-		PGPROC	   *proc = &allProcs[pgprocno];
+		PGPROC	   *proc = allProcs[pgprocno];
 
 		if (databaseid == InvalidOid || proc->databaseId == databaseid)
 		{
@@ -3708,7 +3708,7 @@ CountUserBackends(Oid roleid)
 	for (index = 0; index < arrayP->numProcs; index++)
 	{
 		int			pgprocno = arrayP->pgprocnos[index];
-		PGPROC	   *proc = &allProcs[pgprocno];
+		PGPROC	   *proc = allProcs[pgprocno];
 
 		if (proc->pid == 0)
 			continue;			/* do not count prepared xacts */
@@ -3771,7 +3771,7 @@ CountOtherDBBackends(Oid databaseId, int *nbackends, int *nprepared)
 		for (index = 0; index < arrayP->numProcs; index++)
 		{
 			int			pgprocno = arrayP->pgprocnos[index];
-			PGPROC	   *proc = &allProcs[pgprocno];
+			PGPROC	   *proc = allProcs[pgprocno];
 			uint8		statusFlags = ProcGlobal->statusFlags[index];
 
 			if (proc->databaseId != databaseId)
@@ -3837,7 +3837,7 @@ TerminateOtherDBBackends(Oid databaseId)
 	for (i = 0; i < procArray->numProcs; i++)
 	{
 		int			pgprocno = arrayP->pgprocnos[i];
-		PGPROC	   *proc = &allProcs[pgprocno];
+		PGPROC	   *proc = allProcs[pgprocno];
 
 		if (proc->databaseId != databaseId)
 			continue;
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 2776ceb295b..95b1da42408 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -2844,7 +2844,7 @@ FastPathTransferRelationLocks(LockMethod lockMethodTable, const LOCKTAG *locktag
 	 */
 	for (i = 0; i < ProcGlobal->allProcCount; i++)
 	{
-		PGPROC	   *proc = &ProcGlobal->allProcs[i];
+		PGPROC	   *proc = ProcGlobal->allProcs[i];
 		uint32		j;
 
 		LWLockAcquire(&proc->fpInfoLock, LW_EXCLUSIVE);
@@ -3103,7 +3103,7 @@ GetLockConflicts(const LOCKTAG *locktag, LOCKMODE lockmode, int *countp)
 		 */
 		for (i = 0; i < ProcGlobal->allProcCount; i++)
 		{
-			PGPROC	   *proc = &ProcGlobal->allProcs[i];
+			PGPROC	   *proc = ProcGlobal->allProcs[i];
 			uint32		j;
 
 			/* A backend never blocks itself */
@@ -3790,7 +3790,7 @@ GetLockStatusData(void)
 	 */
 	for (i = 0; i < ProcGlobal->allProcCount; ++i)
 	{
-		PGPROC	   *proc = &ProcGlobal->allProcs[i];
+		PGPROC	   *proc = ProcGlobal->allProcs[i];
 
 		/* Skip backends with pid=0, as they don't hold fast-path locks */
 		if (proc->pid == 0)
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index e9ef0fbfe32..9d3e94a7b3a 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -29,21 +29,29 @@
  */
 #include "postgres.h"
 
+#include <sched.h>
 #include <signal.h>
 #include <unistd.h>
 #include <sys/time.h>
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
 #include "access/transam.h"
 #include "access/twophase.h"
 #include "access/xlogutils.h"
 #include "miscadmin.h"
 #include "pgstat.h"
+#include "port/pg_numa.h"
 #include "postmaster/autovacuum.h"
 #include "replication/slotsync.h"
 #include "replication/syncrep.h"
 #include "storage/condition_variable.h"
 #include "storage/ipc.h"
 #include "storage/lmgr.h"
+#include "storage/pg_shmem.h"
 #include "storage/pmsignal.h"
 #include "storage/proc.h"
 #include "storage/procarray.h"
@@ -89,6 +97,12 @@ static void ProcKill(int code, Datum arg);
 static void AuxiliaryProcKill(int code, Datum arg);
 static void CheckDeadLock(void);
 
+/* NUMA */
+static Size get_memory_page_size(void); /* XXX duplicate */
+static void move_to_node(char *startptr, char *endptr,
+						 Size mem_page_size, int node);
+static int	numa_nodes = -1;
+
 
 /*
  * Report shared-memory space needed by PGPROC.
@@ -100,11 +114,40 @@ PGProcShmemSize(void)
 	Size		TotalProcs =
 		add_size(MaxBackends, add_size(NUM_AUXILIARY_PROCS, max_prepared_xacts));
 
+	size = add_size(size, mul_size(TotalProcs, sizeof(PGPROC *)));
 	size = add_size(size, mul_size(TotalProcs, sizeof(PGPROC)));
 	size = add_size(size, mul_size(TotalProcs, sizeof(*ProcGlobal->xids)));
 	size = add_size(size, mul_size(TotalProcs, sizeof(*ProcGlobal->subxidStates)));
 	size = add_size(size, mul_size(TotalProcs, sizeof(*ProcGlobal->statusFlags)));
 
+	/*
+	 * With NUMA, we allocate the PGPROC array in several chunks. With shared
+	 * buffers we simply manually assign parts of the buffer array to
+	 * different NUMA nodes, and that does the trick. But we can't do that for
+	 * PGPROC, as the number of PGPROC entries is much lower, especially with
+	 * huge pages. We can fit ~2k entries on a 2MB page, and NUMA does stuff
+	 * with page granularity, and the large NUMA systems are likely to use
+	 * huge pages. So with sensible max_connections we would not use more than
+	 * a single page, which means it gets to a single NUMA node.
+	 *
+	 * So we allocate PGPROC not as a single array, but one array per NUMA
+	 * node, and then one array for aux processes (without NUMA node
+	 * assigned). Each array may need up to memory-page-worth of padding,
+	 * worst case. So we just add that - it's a bit wasteful, but good enough
+	 * for PoC.
+	 *
+	 * FIXME Should be conditional, but that was causing problems in bootstrap
+	 * mode. Or maybe it was because the code that allocates stuff later does
+	 * not do that conditionally. Anyway, needs to be fixed.
+	 */
+	/* if (numa_procs_interleave) */
+	{
+		int			num_nodes = numa_num_configured_nodes();
+		Size		mem_page_size = get_memory_page_size();
+
+		size = add_size(size, mul_size((num_nodes + 1), mem_page_size));
+	}
+
 	return size;
 }
 
@@ -129,6 +172,26 @@ FastPathLockShmemSize(void)
 
 	size = add_size(size, mul_size(TotalProcs, (fpLockBitsSize + fpRelIdSize)));
 
+	/*
+	 * Same NUMA-padding logic as in PGProcShmemSize, adding a memory page per
+	 * NUMA node - but this way we add two pages per node - one for PGPROC,
+	 * one for fast-path arrays. In theory we could make this work just one
+	 * page per node, by adding fast-path arrays right after PGPROC entries on
+	 * each node. But now we allocate fast-path locks separately - good enough
+	 * for PoC.
+	 *
+	 * FIXME Should be conditional, but that was causing problems in bootstrap
+	 * mode. Or maybe it was because the code that allocates stuff later does
+	 * not do that conditionally. Anyway, needs to be fixed.
+	 */
+	/* if (numa_procs_interleave) */
+	{
+		int			num_nodes = numa_num_configured_nodes();
+		Size		mem_page_size = get_memory_page_size();
+
+		size = add_size(size, mul_size((num_nodes + 1), mem_page_size));
+	}
+
 	return size;
 }
 
@@ -191,11 +254,13 @@ ProcGlobalSemas(void)
 void
 InitProcGlobal(void)
 {
-	PGPROC	   *procs;
+	PGPROC	  **procs;
 	int			i,
 				j;
 	bool		found;
 	uint32		TotalProcs = MaxBackends + NUM_AUXILIARY_PROCS + max_prepared_xacts;
+	int			procs_total;
+	int			procs_per_node;
 
 	/* Used for setup of per-backend fast-path slots. */
 	char	   *fpPtr,
@@ -205,6 +270,8 @@ InitProcGlobal(void)
 	Size		requestSize;
 	char	   *ptr;
 
+	Size		mem_page_size = get_memory_page_size();
+
 	/* Create the ProcGlobal shared structure */
 	ProcGlobal = (PROC_HDR *)
 		ShmemInitStruct("Proc Header", sizeof(PROC_HDR), &found);
@@ -224,6 +291,9 @@ InitProcGlobal(void)
 	pg_atomic_init_u32(&ProcGlobal->procArrayGroupFirst, INVALID_PROC_NUMBER);
 	pg_atomic_init_u32(&ProcGlobal->clogGroupFirst, INVALID_PROC_NUMBER);
 
+	/* one chunk per NUMA node (without NUMA assume 1 node) */
+	numa_nodes = numa_num_configured_nodes();
+
 	/*
 	 * Create and initialize all the PGPROC structures we'll need.  There are
 	 * six separate consumers: (1) normal backends, (2) autovacuum workers and
@@ -241,19 +311,108 @@ InitProcGlobal(void)
 
 	MemSet(ptr, 0, requestSize);
 
-	procs = (PGPROC *) ptr;
-	ptr = (char *) ptr + TotalProcs * sizeof(PGPROC);
+	/* allprocs (array of pointers to PGPROC entries) */
+	procs = (PGPROC **) ptr;
+	ptr = (char *) ptr + TotalProcs * sizeof(PGPROC *);
 
 	ProcGlobal->allProcs = procs;
 	/* XXX allProcCount isn't really all of them; it excludes prepared xacts */
 	ProcGlobal->allProcCount = MaxBackends + NUM_AUXILIARY_PROCS;
 
+	/*
+	 * NUMA partitioning
+	 *
+	 * Now build the actual PGPROC arrays, one "chunk" per NUMA node (and one
+	 * extra for auxiliary processes and 2PC transactions, not associated with
+	 * any particular node).
+	 *
+	 * First determine how many "backend" procs to allocate per NUMA node. The
+	 * count may not be exactly divisible, but we mostly ignore that. The last
+	 * node may get somewhat fewer PGPROC entries, but the imbalance ought to
+	 * be pretty small (if MaxBackends >> numa_nodes).
+	 *
+	 * XXX A fairer distribution is possible, but not worth it now.
+	 */
+	procs_per_node = (MaxBackends + (numa_nodes - 1)) / numa_nodes;
+	procs_total = 0;
+
+	/* build PGPROC entries for NUMA nodes */
+	for (i = 0; i < numa_nodes; i++)
+	{
+		PGPROC	   *procs_node;
+
+		/* the last NUMA node may get fewer PGPROC entries, but meh */
+		int			count_node = Min(procs_per_node, MaxBackends - procs_total);
+
+		/* make sure to align the PGPROC array to memory page */
+		ptr = (char *) TYPEALIGN(mem_page_size, ptr);
+
+		/* allocate the PGPROC chunk for this node */
+		procs_node = (PGPROC *) ptr;
+		ptr = (char *) ptr + count_node * sizeof(PGPROC);
+
+		/* don't overflow the allocation */
+		Assert((ptr > (char *) procs) && (ptr <= (char *) procs + requestSize));
+
+		/* add pointers to the PGPROC entries to allProcs */
+		for (j = 0; j < count_node; j++)
+		{
+			procs_node[j].numa_node = i;
+			procs_node[j].procnumber = procs_total;
+
+			ProcGlobal->allProcs[procs_total++] = &procs_node[j];
+		}
+
+		move_to_node((char *) procs_node, ptr, mem_page_size, i);
+	}
+
+	/*
+	 * also build PGPROC entries for auxiliary procs / prepared xacts (we
+	 * don't assign those to any NUMA node)
+	 *
+	 * XXX Mostly duplicate of preceding block, could be reused.
+	 */
+	{
+		PGPROC	   *procs_node;
+		int			count_node = (NUM_AUXILIARY_PROCS + max_prepared_xacts);
+
+		/*
+		 * Make sure to align PGPROC array to memory page (it may not be
+		 * aligned). We won't assign this to any NUMA node, but we still don't
+		 * want it to interfere with the preceding chunk (for the last NUMA
+		 * node).
+		 */
+		ptr = (char *) TYPEALIGN(mem_page_size, ptr);
+
+		procs_node = (PGPROC *) ptr;
+		ptr = (char *) ptr + count_node * sizeof(PGPROC);
+
+		/* don't overflow the allocation */
+		Assert((ptr > (char *) procs) && (ptr <= (char *) procs + requestSize));
+
+		/* now add the PGPROC pointers to allProcs */
+		for (j = 0; j < count_node; j++)
+		{
+			procs_node[j].numa_node = -1;
+			procs_node[j].procnumber = procs_total;
+
+			ProcGlobal->allProcs[procs_total++] = &procs_node[j];
+		}
+	}
+
+	/* we should have allocated the expected number of PGPROC entries */
+	Assert(procs_total == TotalProcs);
+
 	/*
 	 * Allocate arrays mirroring PGPROC fields in a dense manner. See
 	 * PROC_HDR.
 	 *
 	 * XXX: It might make sense to increase padding for these arrays, given
 	 * how hotly they are accessed.
+	 *
+	 * XXX Would it make sense to NUMA-partition these chunks too, somehow?
+	 * But those arrays are tiny, fit into a single memory page, so would need
+	 * to be made more complex. Not sure.
 	 */
 	ProcGlobal->xids = (TransactionId *) ptr;
 	ptr = (char *) ptr + (TotalProcs * sizeof(*ProcGlobal->xids));
@@ -286,23 +445,100 @@ InitProcGlobal(void)
 	/* For asserts checking we did not overflow. */
 	fpEndPtr = fpPtr + requestSize;
 
-	for (i = 0; i < TotalProcs; i++)
+	/* reset the count */
+	procs_total = 0;
+
+	/*
+	 * Mimic the same logic as above, but for fast-path locking.
+	 */
+	for (i = 0; i < numa_nodes; i++)
 	{
-		PGPROC	   *proc = &procs[i];
+		char	   *startptr;
+		char	   *endptr;
 
-		/* Common initialization for all PGPROCs, regardless of type. */
+		/* the last NUMA node may get fewer PGPROC entries, but meh */
+		int			procs_node = Min(procs_per_node, MaxBackends - procs_total);
+
+		/* align to memory page, to make move_pages possible */
+		fpPtr = (char *) TYPEALIGN(mem_page_size, fpPtr);
+
+		startptr = fpPtr;
+		endptr = fpPtr + procs_node * (fpLockBitsSize + fpRelIdSize);
+
+		move_to_node(startptr, endptr, mem_page_size, i);
 
 		/*
-		 * Set the fast-path lock arrays, and move the pointer. We interleave
-		 * the two arrays, to (hopefully) get some locality for each backend.
+		 * Now point the PGPROC entries to the fast-path arrays, and also
+		 * advance the fpPtr.
 		 */
-		proc->fpLockBits = (uint64 *) fpPtr;
-		fpPtr += fpLockBitsSize;
+		for (j = 0; j < procs_node; j++)
+		{
+			PGPROC	   *proc = ProcGlobal->allProcs[procs_total++];
+
+			/* cross-check we got the expected NUMA node */
+			Assert(proc->numa_node == i);
+			Assert(proc->procnumber == (procs_total - 1));
+
+			/*
+			 * Set the fast-path lock arrays, and move the pointer. We
+			 * interleave the two arrays, to (hopefully) get some locality for
+			 * each backend.
+			 */
+			proc->fpLockBits = (uint64 *) fpPtr;
+			fpPtr += fpLockBitsSize;
 
-		proc->fpRelId = (Oid *) fpPtr;
-		fpPtr += fpRelIdSize;
+			proc->fpRelId = (Oid *) fpPtr;
+			fpPtr += fpRelIdSize;
 
-		Assert(fpPtr <= fpEndPtr);
+			Assert(fpPtr <= fpEndPtr);
+		}
+
+		Assert(fpPtr == endptr);
+	}
+
+	/* auxiliary processes / prepared xacts */
+	{
+		/* the last NUMA node may get fewer PGPROC entries, but meh */
+		int			procs_node = (NUM_AUXILIARY_PROCS + max_prepared_xacts);
+
+		/* align to memory page, to make move_pages possible */
+		fpPtr = (char *) TYPEALIGN(mem_page_size, fpPtr);
+
+		/* now point the PGPROC entries to the fast-path arrays */
+		for (j = 0; j < procs_node; j++)
+		{
+			PGPROC	   *proc = ProcGlobal->allProcs[procs_total++];
+
+			/* cross-check we got PGPROC with no NUMA node assigned */
+			Assert(proc->numa_node == -1);
+			Assert(proc->procnumber == (procs_total - 1));
+
+			/*
+			 * Set the fast-path lock arrays, and move the pointer. We
+			 * interleave the two arrays, to (hopefully) get some locality for
+			 * each backend.
+			 */
+			proc->fpLockBits = (uint64 *) fpPtr;
+			fpPtr += fpLockBitsSize;
+
+			proc->fpRelId = (Oid *) fpPtr;
+			fpPtr += fpRelIdSize;
+
+			Assert(fpPtr <= fpEndPtr);
+		}
+	}
+
+	/* Should have consumed exactly the expected amount of fast-path memory. */
+	Assert(fpPtr <= fpEndPtr);
+
+	/* make sure we allocated the expected number of PGPROC entries */
+	Assert(procs_total == TotalProcs);
+
+	for (i = 0; i < TotalProcs; i++)
+	{
+		PGPROC	   *proc = procs[i];
+
+		Assert(proc->procnumber == i);
 
 		/*
 		 * Set up per-PGPROC semaphore, latch, and fpInfoLock.  Prepared xact
@@ -366,15 +602,12 @@ InitProcGlobal(void)
 		pg_atomic_init_u64(&(proc->waitStart), 0);
 	}
 
-	/* Should have consumed exactly the expected amount of fast-path memory. */
-	Assert(fpPtr == fpEndPtr);
-
 	/*
 	 * Save pointers to the blocks of PGPROC structures reserved for auxiliary
 	 * processes and prepared transactions.
 	 */
-	AuxiliaryProcs = &procs[MaxBackends];
-	PreparedXactProcs = &procs[MaxBackends + NUM_AUXILIARY_PROCS];
+	AuxiliaryProcs = procs[MaxBackends];
+	PreparedXactProcs = procs[MaxBackends + NUM_AUXILIARY_PROCS];
 
 	/* Create ProcStructLock spinlock, too */
 	ProcStructLock = (slock_t *) ShmemInitStruct("ProcStructLock spinlock",
@@ -435,7 +668,45 @@ InitProcess(void)
 
 	if (!dlist_is_empty(procgloballist))
 	{
-		MyProc = dlist_container(PGPROC, links, dlist_pop_head_node(procgloballist));
+		/*
+		 * With numa interleaving of PGPROC, try to get a PROC entry from the
+		 * right NUMA node (when the process starts).
+		 *
+		 * XXX The process may move to a different NUMA node later, but
+		 * there's not much we can do about that.
+		 */
+		if (numa_procs_interleave)
+		{
+			dlist_mutable_iter iter;
+			unsigned	cpu;
+			unsigned	node;
+			int			rc;
+
+			rc = getcpu(&cpu, &node);
+			if (rc != 0)
+				elog(ERROR, "getcpu failed: %m");
+
+			MyProc = NULL;
+
+			dlist_foreach_modify(iter, procgloballist)
+			{
+				PGPROC	   *proc;
+
+				proc = dlist_container(PGPROC, links, iter.cur);
+
+				if (proc->numa_node == node)
+				{
+					MyProc = proc;
+					dlist_delete(iter.cur);
+					break;
+				}
+			}
+		}
+
+		/* didn't find PGPROC from the correct NUMA node, pick any free one */
+		if (MyProc == NULL)
+			MyProc = dlist_container(PGPROC, links, dlist_pop_head_node(procgloballist));
+
 		SpinLockRelease(ProcStructLock);
 	}
 	else
@@ -1988,7 +2259,7 @@ ProcSendSignal(ProcNumber procNumber)
 	if (procNumber < 0 || procNumber >= ProcGlobal->allProcCount)
 		elog(ERROR, "procNumber out of range");
 
-	SetLatch(&ProcGlobal->allProcs[procNumber].procLatch);
+	SetLatch(&ProcGlobal->allProcs[procNumber]->procLatch);
 }
 
 /*
@@ -2063,3 +2334,60 @@ BecomeLockGroupMember(PGPROC *leader, int pid)
 
 	return ok;
 }
+
+/* copy from buf_init.c */
+static Size
+get_memory_page_size(void)
+{
+	Size		os_page_size;
+	Size		huge_page_size;
+
+#ifdef WIN32
+	SYSTEM_INFO sysinfo;
+
+	GetSystemInfo(&sysinfo);
+	os_page_size = sysinfo.dwPageSize;
+#else
+	os_page_size = sysconf(_SC_PAGESIZE);
+#endif
+
+	/*
+	 * XXX This is a bit annoying/confusing, because we may get a different
+	 * result depending on when we call it. Before mmap() we don't know if the
+	 * huge pages get used, so we assume they will. And then if we don't get
+	 * huge pages, we'll waste memory etc.
+	 */
+
+	/* assume huge pages get used, unless HUGE_PAGES_OFF */
+	if (huge_pages_status == HUGE_PAGES_OFF)
+		huge_page_size = 0;
+	else
+		GetHugePageSize(&huge_page_size, NULL);
+
+	return Max(os_page_size, huge_page_size);
+}
+
+/*
+ * move_to_node
+ *		move all pages in the given range to the requested NUMA node
+ *
+ * XXX This is expected to only process fairly small number of pages, so no
+ * need to do batching etc. Just move pages one by one.
+ */
+static void
+move_to_node(char *startptr, char *endptr, Size mem_page_size, int node)
+{
+	while (startptr < endptr)
+	{
+		int			r,
+					status;
+
+		r = numa_move_pages(0, 1, (void **) &startptr, &node, &status, 0);
+
+		if (r != 0)
+			elog(WARNING, "failed to move page to NUMA node %d (r = %d, status = %d)",
+				 node, r, status);
+
+		startptr += mem_page_size;
+	}
+}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 7febf3001a3..bf775c76545 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -149,6 +149,7 @@ int			MaxBackends = 0;
 bool		numa_buffers_interleave = false;
 bool		numa_localalloc = false;
 int			numa_partition_freelist = 0;
+bool		numa_procs_interleave = false;
 
 /* GUC parameters for vacuum */
 int			VacuumBufferUsageLimit = 2048;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index e2361c161e6..930082588f2 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2144,6 +2144,16 @@ struct config_bool ConfigureNamesBool[] =
 		NULL, NULL, NULL
 	},
 
+	{
+		{"numa_procs_interleave", PGC_POSTMASTER, DEVELOPER_OPTIONS,
+			gettext_noop("Enables NUMA interleaving of PGPROC entries."),
+			gettext_noop("When enabled, the PGPROC entries are interleaved to all NUMA nodes."),
+		},
+		&numa_procs_interleave,
+		false,
+		NULL, NULL, NULL
+	},
+
 	{
 		{"sync_replication_slots", PGC_SIGHUP, REPLICATION_STANDBY,
 			gettext_noop("Enables a physical standby to synchronize logical failover replication slots from the primary server."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 17528439f07..f454b4e9d75 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -181,6 +181,7 @@ extern PGDLLIMPORT int max_parallel_workers;
 extern PGDLLIMPORT bool numa_buffers_interleave;
 extern PGDLLIMPORT bool numa_localalloc;
 extern PGDLLIMPORT int numa_partition_freelist;
+extern PGDLLIMPORT bool numa_procs_interleave;
 
 extern PGDLLIMPORT int commit_timestamp_buffers;
 extern PGDLLIMPORT int multixact_member_buffers;
diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h
index 9f9b3fcfbf1..5cb1632718e 100644
--- a/src/include/storage/proc.h
+++ b/src/include/storage/proc.h
@@ -194,6 +194,8 @@ struct PGPROC
 								 * vacuum must not remove tuples deleted by
 								 * xid >= xmin ! */
 
+	int			procnumber;		/* index in ProcGlobal->allProcs */
+
 	int			pid;			/* Backend's process ID; 0 if prepared xact */
 
 	int			pgxactoff;		/* offset into various ProcGlobal->arrays with
@@ -319,6 +321,9 @@ struct PGPROC
 	PGPROC	   *lockGroupLeader;	/* lock group leader, if I'm a member */
 	dlist_head	lockGroupMembers;	/* list of members, if I'm a leader */
 	dlist_node	lockGroupLink;	/* my member link, if I'm a member */
+
+	/* NUMA node */
+	int			numa_node;
 };
 
 /* NOTE: "typedef struct PGPROC PGPROC" appears in storage/lock.h. */
@@ -383,7 +388,7 @@ extern PGDLLIMPORT PGPROC *MyProc;
 typedef struct PROC_HDR
 {
 	/* Array of PGPROC structures (not including dummies for prepared txns) */
-	PGPROC	   *allProcs;
+	PGPROC	  **allProcs;
 
 	/* Array mirroring PGPROC.xid for each PGPROC currently in the procarray */
 	TransactionId *xids;
@@ -435,8 +440,8 @@ extern PGDLLIMPORT PGPROC *PreparedXactProcs;
 /*
  * Accessors for getting PGPROC given a ProcNumber and vice versa.
  */
-#define GetPGProcByNumber(n) (&ProcGlobal->allProcs[(n)])
-#define GetNumberFromPGProc(proc) ((proc) - &ProcGlobal->allProcs[0])
+#define GetPGProcByNumber(n) (ProcGlobal->allProcs[(n)])
+#define GetNumberFromPGProc(proc) ((proc)->procnumber)
 
 /*
  * We set aside some extra PGPROC structures for "special worker" processes,
-- 
2.49.0

From f76377a56f37421c61c4dd876813b57084b019df Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 27 May 2025 23:08:48 +0200
Subject: [PATCH v1 6/6] NUMA: pin backends to NUMA nodes

When initializing the backend, we pick a PGPROC entry from the right
NUMA node where the backend is running. But the process can move to a
different core / node, so to prevent that we pin it.
---
 src/backend/storage/lmgr/proc.c     | 21 +++++++++++++++++++++
 src/backend/utils/init/globals.c    |  1 +
 src/backend/utils/misc/guc_tables.c | 10 ++++++++++
 src/include/miscadmin.h             |  1 +
 4 files changed, 33 insertions(+)

diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index 9d3e94a7b3a..4c9e55608b2 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -729,6 +729,27 @@ InitProcess(void)
 	}
 	MyProcNumber = GetNumberFromPGProc(MyProc);
 
+	/*
+	 * Optionally, restrict the process to only run on CPUs from the same NUMA
+	 * as the PGPROC. We do this even if the PGPROC has a different NUMA node,
+	 * but not for PGPROC entries without a node (i.e. aux/2PC entries).
+	 *
+	 * This also means we only do this with numa_procs_interleave, because
+	 * without that we'll have numa_node=-1 for all PGPROC entries.
+	 *
+	 * FIXME add proper error-checking for libnuma functions
+	 */
+	if (numa_procs_pin && MyProc->numa_node != -1)
+	{
+		struct bitmask *cpumask = numa_allocate_cpumask();
+
+		numa_node_to_cpus(MyProc->numa_node, cpumask);
+
+		numa_sched_setaffinity(MyProcPid, cpumask);
+
+		numa_free_cpumask(cpumask);
+	}
+
 	/*
 	 * Cross-check that the PGPROC is of the type we expect; if this were not
 	 * the case, it would get returned to the wrong list.
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index bf775c76545..e584ba840ef 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -150,6 +150,7 @@ bool		numa_buffers_interleave = false;
 bool		numa_localalloc = false;
 int			numa_partition_freelist = 0;
 bool		numa_procs_interleave = false;
+bool		numa_procs_pin = false;
 
 /* GUC parameters for vacuum */
 int			VacuumBufferUsageLimit = 2048;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 930082588f2..3fc8897ae36 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2154,6 +2154,16 @@ struct config_bool ConfigureNamesBool[] =
 		NULL, NULL, NULL
 	},
 
+	{
+		{"numa_procs_pin", PGC_POSTMASTER, DEVELOPER_OPTIONS,
+			gettext_noop("Enables pinning backends to NUMA nodes (matching the PGPROC node)."),
+			gettext_noop("When enabled, sets affinity to CPUs from the same NUMA node."),
+		},
+		&numa_procs_pin,
+		false,
+		NULL, NULL, NULL
+	},
+
 	{
 		{"sync_replication_slots", PGC_SIGHUP, REPLICATION_STANDBY,
 			gettext_noop("Enables a physical standby to synchronize logical failover replication slots from the primary server."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index f454b4e9d75..d0d960caa9d 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -182,6 +182,7 @@ extern PGDLLIMPORT bool numa_buffers_interleave;
 extern PGDLLIMPORT bool numa_localalloc;
 extern PGDLLIMPORT int numa_partition_freelist;
 extern PGDLLIMPORT bool numa_procs_interleave;
+extern PGDLLIMPORT bool numa_procs_pin;
 
 extern PGDLLIMPORT int commit_timestamp_buffers;
 extern PGDLLIMPORT int multixact_member_buffers;
-- 
2.49.0

---

From: Ashutosh Bapat <[email-masked]>
Date: Wed, 2 Jul 2025 17:07:28 +0530

> freelist, without using it. So the patch removes that.

The patches for resizing buffers use the lastFreeBuffer to add new
buffers to the end of free list when expanding it. But we could as
well add it at the beginning of the free list.

This patch seems almost independent of the rest of the patches. Do you
need it in the rest of the patches? I understand that those patches
don't need to worry about maintaining lastFreeBuffer after this patch.
Is there any other effect?

If we are going to do this, let's do it earlier so that buffer
resizing patches can be adjusted.

> online resize" by Ashutosh, simply because it touches the shared memory.

I have added Dmitry to this thread since he has written most of the
shared memory handling code.

> e.g. for PGPROC. So that'd be good.

The resizing patches split the shared buffer related structures into
separate memory segments. I think that itself will help enabling huge
pages for some regions. Would that help in your case?

> need to rebuild the freelists in some way, I guess.

Yes, there's code to build the free list. I think we will need code to
remap the buffers and buffer descriptor.

-- 
Best Wishes,
Ashutosh Bapat

---

From: Tomas Vondra <[email-masked]>
Date: Wed, 2 Jul 2025 14:36:31 +0200

> resizing patches can be adjusted.

My patches don't particularly rely on this bit, it would work even with
lastFreeBuffer. I believe Andres simply noticed the current code does
not use lastFreeBuffer, it just maintains is, so he removed that as an
optimization. I don't know how significant is the improvement, but if
it's measurable we could just do that independently of our patches.

> shared memory handling code.

Thanks.

> pages for some regions. Would that help in your case?

Indirectly. My patch can work just fine with a single segment, but being
able to enable huge pages only for some of the segments seems better.

> remap the buffers and buffer descriptor.

Right. The good thing is that's just "advisory" information, it doesn't
break anything if it's temporarily out of sync. We don't need to "stop"
everything to remap the buffers to other nodes, or anything like that.
Or at least I think so.

It's one thing to "flip" the target mapping (determining which node a
buffer should be on), and actually migrating the buffers. The first part
can be done instantaneously, the second part can happen in the
background over a longer time period.

I'm not sure how you're rebuilding the freelist. Presumably it can
contain buffers that are no longer valid (after shrinking). How is that
handled to not break anything? I think the NUMA variant would do exactly
the same thing, except that there's multiple lists.

regards

-- 
Tomas Vondra

---

From: Ashutosh Bapat <[email-masked]>
Date: Thu, 3 Jul 2025 19:37:18 +0530

> the same thing, except that there's multiple lists.

Before shrinking the buffers, we walk the free list removing any
buffers that are going to be removed. When expanding, by linking the
new buffers in the order and then adding those to the already existing
free list. 0005 patch in [1] has the code for the same.

[1] https://www.postgresql.org/message-id/my4hukmejato53ef465ev7lk3sqiqvneh7436rz64wmtc7rbfj%40hmuxsf2ngov2

--
Best Wishes,
Ashutosh Bapat

---

From: Dmitry Dolgov <[email-masked]>
Date: Thu, 3 Jul 2025 16:49:34 +0200

> On Wed, Jul 02, 2025 at 05:07:28PM +0530, Ashutosh Bapat wrote:

Thanks! I like the idea behind this patch series. I haven't read it in
details yet, but I can imagine both patches (interleaving and online
resizing) could benefit from each other. In online resizing we've
introduced a possibility to use multiple shared mappings for different
types of data, maybe it would be convenient to use the same interface to
create separate mappings for different NUMA nodes as well. Using a
separate shared mapping per NUMA node would also make resizing easier,
since it would be more straightforward to fit an increased segment into
NUMA boundaries.

> pages for some regions. Would that help in your case?

Right, separate segments would allow to mix and match huge pages with
pages of regular size. It's not implemented in the latest version of
online resizing patch, purely to reduce complexity and maintain the same
invariant (everything is either using huge pages or not) -- but we could
do it other way around as well.

---

From: Jakub Wartak <[email-masked]>
Date: Fri, 4 Jul 2025 13:05:05 +0200

On Tue, Jul 1, 2025 at 9:07 PM Tomas Vondra <[email-masked]> wrote:

Hi!

> 1) v1-0001-NUMA-interleaving-buffers.patch
[..]
> descriptors are tiny - many fit on a memory page.

Oh, now I get it! OK, let's stick to this one.

> e.g. for PGPROC. So that'd be good.

You have made assumption that this is good, but small pages (4KB) are
not hugetlb, and are *swappable* (Transparent HP are swappable too,
manually allocated ones as with mmap(MMAP_HUGETLB) are not)[1]. The
most frequent problem I see these days are OOMs, and it makes me
believe that making certain critical parts of shared memory being
swappable just to make pagesize granular is possibly throwing the baby
out with the bathwater. I'm thinking about bad situations like: some
wrong settings of vm.swapiness that people keep (or distros keep?) and
general inability of PG to restrain from allocating more memory in
some cases.

> some custom PG configuration (which the patch [2] proposed to do).

0. I think that we could do better, some counter arguments to
no-configuration-at-all:

a. as Robert & Bertrand already put it there after review: let's say I
want just to run on NUMA #2 node, so here I would need to override
systemd's script ExecStart= to include that numactl (not elegant?). I
could also use `CPUAffinity=1,3,5,7..` but that's all, and it is even
less friendly. Also it probably requires root to edit/reload systemd,
while having GUC for this like in my proposal makes it more smooth (I
think?)

b. wouldn't it be better if that stayed as drop-in rather than always
on? What if there's a problem, how do you disable those internal
optimizations if they do harm in some cases?  (or let's say I want to
play with MPOL_INTERLEAVE_WEIGHTED?). So at least boolean
numa_buffers_interleave would be nice?

c. What if I want my standby (walreceiver+startup/recovery) to run
with NUMA affinity to get better performance (I'm not going to hack
around systemd script every time, but I could imagine changing
numa=X,Y,Z after restart/before promotion)

d. Now if I would be forced for some reason to do that numactl(1)
voodoo, and use the those above mentioned overrides and PG wouldn't be
having GUC (let's say I would use `numactl

> it allows experimenting with numactl etc.

.. .is not accurate anymore and we would require to have that in
(still with GUC) ?
Thoughts? I can add that mine part into Your's patches if you want.

Way too quick review and some very fast benchmark probes, I've
concentrated only on v1-0001 and v1-0005 (efficiency of buffermgmt
would be too new topic for me), but let's start:

1. normal pgbench -S (still with just s_b@4GB), done many tries,
consistent benefit for the patch with like +8..10% boost on generic
run:

   numa_buffers_interleave=off  numa_pgproc_interleave=on(due that
always on "if"), s_b just on 1 NUMA node (might happen)
        latency average = 0.373 ms
        latency stddev = 0.237 ms
        initial connection time = 45.899 ms
        tps = 160242.147877 (without initial connection time)

   numa_buffers_interleave=on   numa_pgproc_interleave=on
        latency average = 0.345 ms
        latency stddev = 0.373 ms
        initial connection time = 44.485 ms
        tps = 177564.686094 (without initial connection time)

2. Tested it the same way as I did for mine(problem#2 from Andres's
presentation): 4s32c128t, s_b=4GB (on 128GB), prewarm test (with
seqconcurrscans.pgb as earlier)
   default/numa_buffers_interleave=off
        latency average = 1375.478 ms
        latency stddev = 1141.423 ms
        initial connection time = 46.104 ms
        tps = 45.868075 (without initial connection time)

   numa_buffers_interleave=on
        latency average = 838.128 ms
        latency stddev = 498.787 ms
        initial connection time = 43.437 ms
        tps = 75.413894 (without initial connection time)

    and i've repeated the the same test (identical conditions) with my
patch, got me slightly more juice:
        latency average = 727.717 ms
        latency stddev = 410.767 ms
        initial connection time = 45.119 ms
        tps = 86.844161 (without initial connection time)

    (but mine didn't get that boost from normal pgbench as per #1
pgbench -S -- my numa='all' stays @ 160k TPS just as
numa_buffers_interleave=off), so this idea is clearly better.
    So should I close https://commitfest.postgresql.org/patch/5703/
and you'll open a new one or should I just edit the #5703 and alter it
and add this thread too?

3. Patch is not calling interleave on PQ shmem, do we want to add that
in as some next item like v1-0007? Question is whether OS interleaving
makes sense there ? I believe it does there, please see my thread
(NUMA_pq_cpu_pinning_results.txt), the issue is that PQ workers are
being spawned by postmaster and may end up on different NUMA nodes
randomly, so actually OS-interleaving that memory reduces jitter there
(AKA bandwidth-over-latency). My thinking is that one cannot expect
static/forced CPU-to-just-one-NUMA-node assignment for backend and
it's PQ workers, because it is impossible have always available CPU
power there in that NUMA node, so it might be useful to interleave
that shared mem there too (as separate patch item?)

4 In BufferManagerShmemInit() you call numa_num_configured_nodes()
(also in v1-0005). My worry is should we may put some
known-limitations docs (?) from start and mention that
if the VM is greatly resized and NUMA numa nodes appear, they might
not be used until restart?

5. In v1-0001, pg_numa_interleave_memory()

Yes, my patch has those numa_error() and numa_warn() handlers too in
pg_numa. Feel free to use it for better UX.

+                * XXX Should we still touch the memory first, like
with numa_move_pages,
+                * or is that not necessary?

It's not necessary to touch after numa_tonode_memory() (wrapper around
numa_interleave_memory()), if it is going to be used anyway it will be
correctly placed to best of my knowledge.

6. diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c

Accidental indents (also fails to apply)

7. We miss the pg_numa_* shims, but for sure that's for later and also
avoid those Linux specific #ifdef USE_LIBNUMA and so on?

8. v1-0005 2x + /* if (numa_procs_interleave) */

   Ha! it's a TRAP! I've uncommented it because I wanted to try it out
without it (just by setting GUC off) , but "MyProc->sema" is NULL :

    2025-07-04 12:31:08.103 CEST [28754] LOG:  starting PostgreSQL
19devel on x86_64-linux, compiled by gcc-12.2.0, 64-bit
    [..]
    2025-07-04 12:31:08.109 CEST [28754] LOG:  io worker (PID 28755)
was terminated by signal 11: Segmentation fault
    2025-07-04 12:31:08.109 CEST [28754] LOG:  terminating any other
active server processes
    2025-07-04 12:31:08.114 CEST [28754] LOG:  shutting down because
"restart_after_crash" is off
    2025-07-04 12:31:08.116 CEST [28754] LOG:  database system is shut down

    [New LWP 28755]
    [Thread debugging using libthread_db enabled]
    Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
    Core was generated by `postgres: io worker                     '.
    Program terminated with signal SIGSEGV, Segmentation fault.
    #0  __new_sem_wait_fast (definitive_result=1, sem=sem@entry=0x0)
at ./nptl/sem_waitcommon.c:136
    136     ./nptl/sem_waitcommon.c: No such file or directory.
    (gdb) where
    #0  __new_sem_wait_fast (definitive_result=1, sem=sem@entry=0x0)
at ./nptl/sem_waitcommon.c:136
    #1  __new_sem_trywait (sem=sem@entry=0x0) at ./nptl/sem_wait.c:81
    #2  0x00005561918e0cac in PGSemaphoreReset (sema=0x0) at
../src/backend/port/posix_sema.c:302
    #3  0x0000556191970553 in InitAuxiliaryProcess () at
../src/backend/storage/lmgr/proc.c:992
    #4  0x00005561918e51a2 in AuxiliaryProcessMainCommon () at
../src/backend/postmaster/auxprocess.c:65
    #5  0x0000556191940676 in IoWorkerMain (startup_data=<optimized
out>, startup_data_len=<optimized out>) at
../src/backend/storage/aio/method_worker.c:393
    #6  0x00005561918e8163 in postmaster_child_launch
(child_type=child_type@entry=B_IO_WORKER, child_slot=20086,
startup_data=startup_data@entry=0x0,
        startup_data_len=startup_data_len@entry=0,
client_sock=client_sock@entry=0x0) at
../src/backend/postmaster/launch_backend.c:290
    #7  0x00005561918ea09a in StartChildProcess
(type=type@entry=B_IO_WORKER) at
../src/backend/postmaster/postmaster.c:3973
    #8  0x00005561918ea308 in maybe_adjust_io_workers () at
../src/backend/postmaster/postmaster.c:4404
    [..]
    (gdb) print *MyProc->sem
    Cannot access memory at address 0x0

9. v1-0006: is this just a thought or serious candidate? I can imagine
it can easily blow-up with some backends somehow requesting CPUs only
from one NUMA node, while the second node being idle. Isn't it better
just to leave CPU scheduling, well, to the CPU scheduler? The problem
is that you have tools showing overall CPU usage, even mpstat(1) per
CPU , but no tools for per-NUMA node CPU util%, so it would be hard
for someone to realize that this is happening.

-J.

[1] - https://www.kernel.org/doc/Documentation/vm/hugetlbpage.txt

---

From: Tomas Vondra <[email-masked]>
Date: Fri, 4 Jul 2025 20:12:01 +0200

> general inability of PG to restrain from allocating more memory in

I haven't observed such issues myself, or maybe I didn't realize it's
happening. Maybe it happens, but it'd be good to see some data showing
that, or a reproducer of some sort. But let's say it's real.

I don't think we should use huge pages merely to ensure something is not
swapped out. The "not swappable" is more of a limitation of huge pages,
not an advantage. You can't just choose to make them swappable.

Wouldn't it be better to keep using 4KB pages, but lock the memory using
mlock/mlockall?

> --weighted-interleave=0,1`), then:

I'm not against doing something like this, but I don't plan to do that
in V1. I don't have a clear idea what configurability is actually
needed, so it's likely I'd do the interface wrong.

> Thoughts? I can add that mine part into Your's patches if you want.

I'm sorry, I don't understand what's the question :-(

> numa_buffers_interleave=off), so this idea is clearly better.

Good, thanks for the testing. I should have done something like this
when I posted my patches, but I forgot about that (and the email felt
too long anyway).

But this actually brings an interesting question. What exactly should we
expect / demand from these patches? In my mind it'd primarily about
predictability and stability of results.

For example, the results should not depend on how was the database
warmed up - was it done by a single backend or many backends? Was it
restarted, or what? I could probably warmup the system very carefully to
ensure it's balanced. The patches mean I don't need to be that careful.

> and you'll open a new one or should I just edit the #5703 and alter it

Good question. It's probably best to close the original entry as
"withdrawn" and I'll add a new entry. Sounds OK?

> that shared mem there too (as separate patch item?)

Excellent question. I haven't thought about this at all. I agree it
probably makes sense to interleave this memory, in some way. I don't
know what's the perfect scheme, though.

wild idea: Would it make sense to pin the workers to the same NUMA node
as the leader? And allocate all memory only from that node?

> not be used until restart?

Yes, this is one thing I need some feedback on. The patches mostly
assume there are no disabled nodes, that the set of allowed nodes does
not change, etc. I think for V1 that's a reasonable limitation.

But let's say we want to relax this a bit. How do we learn about the
change, after a node/CPU gets disabled? For some parts it's not that
difficult (e.g. we can "remap" buffers/descriptors) in the background.
But for other parts that's not practical. E.g. we can't rework how the
PGPROC gets split.

But while discussing this with Andres yesterday, he had an interesting
suggestion - to always use e.g. 8 or 16 partitions, then partition this
by NUMA node. So we'd have 16 partitions, and with 4 nodes the 0-3 would
go to node 0, 4-7 would go to node 1, etc. The advantage is that if a
node gets disabled, we can rebuild just this small "mapping" and not the
16 partitions. And the partitioning may be helpful even without NUMA.

Still have to figure out the details, but seems it might help.

> avoid those Linux specific #ifdef USE_LIBNUMA and so on?

Right, we need to add those. Or actually, we need to think about how
we'd do this for non-NUMA systems. I wonder if we even want to just
build everything the "old way" (without the partitions, etc.).

But per the earlier comment, the partitioning seems beneficial even on
non-NUMA systems, so maybe the shims are good enough OK.

>     Cannot access memory at address 0x0

Yeah, good catch. I'll look into that next week.

> for someone to realize that this is happening.

Mostly experimental, for benchmarking etc. I agree we may not want to
mess with the task scheduling too much.

Thanks for the feedback!

regards

-- 
Tomas Vondra

---

From: Cédric Villemain <[email-masked]>
Date: Sat, 5 Jul 2025 07:09:00 +0000

Hi Tomas,

I haven't yet had time to fully read all the work and proposals around 
NUMA and related features, but I hope to catch up over the summer.

However, I think it's important to share some thoughts before it's too 
late, as you might find them relevant to the NUMA management code.

> Driven by GUC "numa_procs_pin" (default: off).

In my work on more careful PostgreSQL resource management, I've come to 
the conclusion that we should avoid pushing policy too deeply into the 
PostgreSQL core itself. Therefore, I'm quite skeptical about integrating 
NUMA-specific management directly into core PostgreSQL in such a way.

We are working on a PROFILE and PROFILE MANAGER specification to provide 
PostgreSQL with only the APIs and hooks needed so that extensions can 
manage whatever they want externally.

The basic syntax (not meant to be discussed here, and even the names 
might change) is roughly as follows, just to illustrate the intent:

CREATE PROFILE MANAGER manager_name [IF NOT EXISTS]
   [ HANDLER handler_function | NO HANDLER ]
   [ VALIDATOR validator_function | NO VALIDATOR ]
   [ OPTIONS ( option 'value' [, ... ] ) ]

CREATE PROFILE profile_name
   [IF NOT EXISTS]
   USING profile_manager
     SET key = value [, key = value]...
   [USING profile_manager
     SET key = value [, key = value]...]
   [...];

CREATE PROFILE MAPPING
   [IF NOT EXISTS]
   FOR PROFILE profile_name
   [MATCH [ ALL | ANY ] (
     [ROLE role_name],
     [BACKEND TYPE backend_type],
     [DATABASE database_name],
     [APPLICATION appname]
   )];

## PROFILE RESOLUTION ORDER

1. ALTER ROLE IN DATABASE
2. ALTER ROLE
3. ALTER DATABASE
4. First matching PROFILE MAPPING (global or specific)
5. No profile (fallback)

As currently designed, this approach allows quite a lot of flexibility:

* pg_psi is used to ensure the spec is suitable for a cgroup profile 
manager (moving PIDs as needed; NUMA and cgroups could work well 
together, see e.g. this Linux kernel summary: 
https://blogs.oracle.com/linux/post/numa-balancing )

* Someone else could implement support for Windows or BSD specifics.

* Others might use it to integrate PostgreSQL's own resources (e.g., 
"areas" of shared buffers) into policies.

Hope this perspective is helpful.

Best regards,
-- 
Cédric Villemain +33 6 20 30 22 52
https://www.Data-Bene.io
PostgreSQL Support, Expertise, Training, R&D

---

From: Tomas Vondra <[email-masked]>
Date: Mon, 7 Jul 2025 14:01:53 +0200

On 7/5/25 09:09, Cédric Villemain wrote:
> together, see e.g. this Linux kernel summary: https://blogs.oracle.com/
> linux/post/numa-balancing )
> 

Can you explain how you want to manage this by an extension defined at
the SQL level, when most of this stuff has to be done when setting up
shared memory, which is waaaay before we have any access to catalogs?

regards

-- 
Tomas Vondra

---

From: Jakub Wartak <[email-masked]>
Date: Mon, 7 Jul 2025 14:31:28 +0200

Hi Tomas, some more thoughts after the weekend:

> Wouldn't it be better to keep using 4KB pages, but lock the memory using

In my book, not being swappable is a win (it's hard for me to imagine
when it could be beneficial to swap out parts of s_b).

I was trying to think about it and also got those:

Anyway mlock() probably sounds like it, but e.g. Rocky 8.10 by default
has max locked memory (ulimit -l) as low as 64kB due to systemd's
DefaultLimitMEMLOCK, but Debian/Ubuntu have those at higher values.
Wasn't expecting that - those are bizzare low values. I think we would
need something like (10000*900)/1024/1024 or more, but with each
PGPROC on a separate page that would be even way more?

Another thing with 4kB pages: there's this big assumption now made
that once we arrive in InitProcess() we won't ever change NUMA node,
so we stick to the PGPROC from where we started (based on getcpu(2)).
Let's assume CPU scheduler reassigned us to differnt node, but we have
now this 4kB patch ready for PGPROC in theory and this means we would
need to rely on the NUMA autobalancing doing it's job to migrate that
4kB page from node to node (to get better local accesses instead of
remote ones).  The questions in my head are now like that:
- but we have asked intially asked those PGPROC pages to be localized
on certain node (they have policy), so they won't autobalance? We
would need to somewhere call getcpu() again notice the difference and
unlocalize (clear the NUMA/mbind() policy) for the PGPROC page?
- mlocked() as above says stick to physical RAM page (?) , so it won't move?
- after what time kernel's autobalancing would migrate that page since
switching the active CPU<->node? I mean do we execute enough reads on
this page?

BTW: to move this into pragmatic real, what's the most
one-liner/trivial way to exercise/stress PGPROC?

> I'm sorry, I don't understand what's the question :-(

That patch reference above, it was a chain of thought from step "d".
What I had in mind was that you cannot remove the patch
`v1-0002-NUMA-localalloc.patch` from the scope if forcing people to
use numactl by not having enough configurability on the PG side. That
is: if someone will have to use systemd+numactl
--interleave/--weighted-interleave then, he will also need to have a
way to use numa_localalloc=on (to override the new/user's policy
default, otherwise local mem allocations are also going to be
interleaved, and we are back to square one). Which brings me to a
point why instead of this toggle, should include the configuration
properly inside from start (it's not that hard apparently).

> > consistent benefit for the patch with like +8..10% boost on generic
[.. removed numbers]
> ensure it's balanced. The patches mean I don't need to be that careful.

Well, pretty much the same here. I was after minimizing "stddev" (to
have better predictability of results, especially across restarts) and
increasing available bandwidth [which is pretty much related]. Without
our NUMA work, PG can just put that s_b on any random node or spill
randomly from to another (depending on size of allocation request).

> "withdrawn" and I'll add a new entry. Sounds OK?

Sure thing, marked it as `Returned with feedback`, this approach seems
to be much more advanced.

> as the leader? And allocate all memory only from that node?

I'm trying to convey exactly the opposite message or at least that it
might depend on configuration. Please see
https://www.postgresql.org/message-id/CAKZiRmxYMPbQ4WiyJWh%3DVuw_Ny%2BhLGH9_9FaacKRJvzZ-smm%2Bw%40mail.gmail.com
(btw it should read there that I don't indent spend a lot of thime on
PQ), but anyway: I think we should NOT pin the PQ workers the same
NODE as you do not know if there's CPU left there (same story as with
v1-0006 here).

I'm just proposing quick OS-based interleaving of PQ shm if using all
nodes, literally:

@@ -334,6 +336,13 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size
request_size,
     }
     *mapped_address = address;
     *mapped_size = request_size;
+
+    /* We interleave memory only at creation time. */
+    if (op == DSM_OP_CREATE && numa->setting > NUMA_OFF) {
+        elog(DEBUG1, "interleaving shm mem @ %p size=%zu",
*mapped_address, *mapped_size);
+        pg_numa_interleave_memptr(*mapped_address, *mapped_size, numa->nodes);
+    }
+

Because then if memory is interleaved you have probably less variance
for memory access. But also from that previous thread:

"So if anything:
- latency-wise: it would be best to place leader+all PQ workers close
to s_b, provided s_b fits NUMA shared/huge page memory there and you
won't need more CPU than there's on that NUMA node... (assuming e.g.
hosting 4 DBs on 4-sockets each on it's own, it would be best to pin
everything including shm, but PQ workers too)
- capacity/TPS-wise or s_b > NUMA: just interleave to maximize
bandwidth and get uniform CPU performance out of this"

So wild idea was: maybe PQ shm interleaving should on NUMA
configuration (if intereavling to all nodes, then interleave normally,
but if configuration sets to just 1 NUMA node, it automatically binds
there -- there was '@' support for that in my patch).

> not change, etc. I think for V1 that's a reasonable limitation.

Sure!

> Still have to figure out the details, but seems it might help.

Right, no idea how the shared_memory remapping patch will work
(how/when the s_b change will be executed), but we could somehow mark
that number of NUMA zones could be rechecked during SIGHUP (?) and
then just simple compare check if old_numa_num_configured_nodes ==
new_numa_num_configured_nodes is true.

Anyway, I think it's way too advanced for now, don't you think? (like
CPU ballooning [s_b itself] is rare, and NUMA ballooning seems to be
super-wild-rare).

As for the rest, forgot to include this too: getcpu() - this really
needs a portable pg_getcpu() wrapper.

-J.

---

From: Cédric Villemain <[email-masked]>
Date: Mon, 7 Jul 2025 14:51:00 +0000

> shared memory, which is waaaay before we have any access to catalogs?

I should have said module instead, I didn't follow carefully but at some 
point there were discussion about shared buffers resized "on-line".
Anyway, it was just to give some few examples, maybe this one is to be 
considered later (I'm focused on cgroup/psi, and precisely reassigning 
PIDs as needed).

---

From: Tomas Vondra <[email-masked]>
Date: Tue, 8 Jul 2025 00:35:43 +0200

> considered later (I'm focused on cgroup/psi, and precisely reassigning

I don't know. I have a hard time imagining what exactly would the
policies / profiles do exactly to respond to changes in the system
utilization. And why should that interfere with this patch ...

The main thing patch series aims to implement is partitioning different
pieces of shared memory (buffers, freelists, ...) to better work for
NUMA. I don't think there's that many ways to do this, and I doubt it
makes sense to make this easily customizable from external modules of
any kind. I can imagine providing some API allowing to isolate the
instance on selected NUMA nodes, but that's about it.

Yes, there's some relation to the online resizing of shared buffers, in
which case we need to "refresh" some of the information. But AFAICS it's
not very extensive (on top of what already needs to happen after the
resize), and it'd happen within the boundaries of the partitioning
scheme. There's not that much flexibility.

The last bit (pinning backends to a NUMA node) is experimental, and
mostly intended for easier evaluation of the earlier parts (e.g. to
limit the noise when processes get moved to a CPU from a different NUMA
node, and so on).

regards

-- 
Tomas Vondra

---

From: Andres Freund <[email-masked]>
Date: Mon, 7 Jul 2025 19:25:35 -0400

Hi,

On 2025-07-05 07:09:00 +0000, Cédric Villemain wrote:
> In my work on more careful PostgreSQL resource management, I've come to the
> conclusion that we should avoid pushing policy too deeply into the

I think it's actually the opposite - whenever we pushed stuff like this
outside of core it has hurt postgres substantially. Not having replication in
core was a huge mistake. Not having HA management in core is probably the
biggest current adoption hurdle for postgres.

To deal better with NUMA we need to improve memory placement and various
algorithms, in an interrelated way - that's pretty much impossible to do
outside of core.

Greetings,

Andres Freund

---

From: Cédric Villemain <[email-masked]>
Date: Tue, 8 Jul 2025 01:47:00 +0000

> limit the noise when processes get moved to a CPU from a different NUMA

The backend pinning can be done by replacing your patch on proc.c to 
call an external profile manager doing exactly the same thing maybe ?

Similar to:
pmroutine = GetPmRoutineForInitProcess();
if (pmroutine != NULL &&
	pmroutine->init_process != NULL)
	pmroutine->init_process(MyProc);

...

pmroutine = GetPmRoutineForInitAuxilliary();
if (pmroutine != NULL &&
	pmroutine->init_auxilliary != NULL)
	pmroutine->init_auxilliary(MyProc);

Added on some rare places should cover most if not all the requirement 
around process placement (process_shared_preload_libraries() is called 
earlier in the process creation I believe).

---

From: Cédric Villemain <[email-masked]>
Date: Tue, 8 Jul 2025 01:55:00 +0000

Hi Andres,

> algorithms, in an interrelated way - that's pretty much impossible to do

Except the backend pinning which is easy to achieve, thus my comment on 
the related patch.
I'm not claiming NUMA memory and all should be managed outside of core 
(though I didn't read other patches yet).

---

From: Cédric Villemain <[email-masked]>
Date: Tue, 8 Jul 2025 02:14:00 +0000

> earlier in the process creation I believe).

After a first read I think this works for patches 002 and 005. For this 
last one, InitProcGlobal() may setup things as you do but then expose 
the choice a bit later, basically in places where you added the if 
condition on the GUC: numa_procs_interleave).

---

From: Andres Freund <[email-masked]>
Date: Mon, 7 Jul 2025 23:04:48 -0400

Hi,

> general inability of PG to restrain from allocating more memory in

The reason it would be advantageous to put something like the procarray onto
smaller pages is that otherwise the entire procarray (unless particularly
large) ends up on a single NUMA node, increasing the latency for backends on
every other numa node and increasing memory traffic on that node.

Greetings,

Andres Freund

---

From: Tomas Vondra <[email-masked]>
Date: Tue, 8 Jul 2025 14:27:12 +0200

> every other numa node and increasing memory traffic on that node.

That's why the patch series splits the procarray into multiple pieces,
so that it can be properly distributed on multiple NUMA nodes even with
huge pages. It requires adjusting a couple places accessing the entries,
but it surprised me how limited the impact was.

If we could selectively use 4KB pages for parts of the shared memory,
maybe this wouldn't be necessary. But it's not too annoying.

The thing I'm not sure about is how much this actually helps with the
traffic between node. Sure, if we pick a PGPROC from the same node, and
the task does not get moved, it'll be local traffic. But if the task
moves, there'll be traffic. I don't have any estimates how often this
happens, e.g. for older tasks.

regards

-- 
Tomas Vondra

---

From: Tomas Vondra <[email-masked]>
Date: Tue, 8 Jul 2025 14:34:59 +0200

> (though I didn't read other patches yet).

But an "optimal backend placement" seems to very much depend on where we
placed the various pieces of shared memory. Which the external module
will have trouble following, I suspect.

I still don't have any idea what exactly would the external module do,
how would it decide where to place the backend. Can you describe some
use case with an example?

Assuming we want to actually pin tasks from within Postgres, what I
think might work is allowing modules to "advise" on where to place the
task. But the decision would still be done by core.

regards

-- 
Tomas Vondra

---

From: Andres Freund <[email-masked]>
Date: Tue, 8 Jul 2025 08:56:06 -0400

Hi,

> but it surprised me how limited the impact was.

Sure, you can do that, but it does mean that iterations over the procarray now
have an added level of indirection...

> happens, e.g. for older tasks.

I think the most important bit is to not put everything onto one numa node,
otherwise the chance of increased latency for *everyone* due to the increased
memory contention is more likely to hurt.

Greetings,

Andres Freund

---

From: Cédric Villemain <[email-masked]>
Date: Tue, 8 Jul 2025 16:06:00 +0000

> task. But the decision would still be done by core.

Possibly exactly what you're doing in proc.c when managing allocation of 
process, but not hardcoded in postgresql (patches 02, 05 and 06 are good 
candidates), I didn't get that they require information not available to 
any process executing code from a module.

Parts of your code where you assign/define policy could be in one or 
more relevant routines of a "numa profile manager", like in an 
initProcessRoutine(), and registered in pmroutine struct:

This way it's easier to manage alternative policies, and also to be able 
to adjust when hardware and linux kernel changes.

---

From: Tomas Vondra <[email-masked]>
Date: Tue, 8 Jul 2025 23:26:06 +0200

> any process executing code from a module.

Well, it needs to understand how some other stuff (especially PGPROC
entries) is distributed between nodes. I'm not sure how much of this
internal information we want to expose outside core ...

> to adjust when hardware and linux kernel changes.

I'm not against making this extensible, in some way. But I still
struggle to imagine a reasonable alternative policy, where the external
module gets the same information and ends up with a different decision.

So what would the alternate policy look like? What use case would the
module be supporting?

regards

-- 
Tomas Vondra

---

From: Cédric Villemain <[email-masked]>
Date: Wed, 9 Jul 2025 06:40:00 +0000

> So what would the alternate policy look like? What use case would the

That's the whole point: there are very distinct usages of PostgreSQL in 
the field. And maybe not all of them will require the policy defined by 
PostgreSQL core.

May I ask the reverse: what prevent external modules from taking those 
decisions ? There are already a lot of area where external code can take 
over PostgreSQL processing, like Neon is doing.

There are some very early processing for memory setup that I can see as 
a current blocker, and here I'd refer a more compliant NUMA api as 
proposed by Jakub so it's possible to arrange based on workload, 
hardware configuration or other matters. Reworking to get distinct 
segment and all as you do is great, and combo of both approach probably 
of great interest. There is also this weighted interleave discussed and 
probably much more to come in this area in Linux.

I think some points raised already about possible distinct policies, I 
am precisely claiming that it is hard to come with one good policy with 
limited setup options, thus requirement to keep that flexible enough 
(hooks, api, 100 GUc ?).

There is an EPYC story here also, given the NUMA setup can vary 
depending on BIOS setup, associated NUMA policy must probably take that 
into account (L3 can be either real cache or 4 extra "local" NUMA nodes 
- with highly distinct access cost from a RAM module).
Does that change how PostgreSQL will place memory and process? Is it 
important or of interest ?

---

From: Bertrand Drouvot <[email-masked]>
Date: Wed, 9 Jul 2025 08:09:16 +0000

Hi,

On Wed, Jul 09, 2025 at 06:40:00AM +0000, Cédric Villemain wrote:
> That's the whole point: there are very distinct usages of PostgreSQL in the
> field. And maybe not all of them will require the policy defined by
> PostgreSQL core.
> 
> There are some very early processing for memory setup that I can see as a
> current blocker, and here I'd refer a more compliant NUMA api as proposed by
> Jakub so it's possible to arrange based on workload, hardware configuration
> or other matters. Reworking to get distinct segment and all as you do is
> great, and combo of both approach probably of great interest.

I think that Tomas's approach helps to have more "predictable" performance
expectations, I mean more consistent over time, fewer "surprises".

While your approach (and Jakub's one)) could help to get performance gains
depending on a "known" context (so less generic).

So, probably having both could make sense but I think that they serve different
purposes.

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

---

From: Jakub Wartak <[email-masked]>
Date: Wed, 9 Jul 2025 12:04:00 +0200

> > > every other numa node and increasing memory traffic on that node.

Sure thing, I fully understand the motivation and underlying reason
(without claiming that I understand the exact memory access patterns
that involve procarray/PGPROC/etc and hotspots involved from PG side).
Any single-liner pgbench help for how to really easily stress the
PGPROC or procarray?

> > but it surprised me how limited the impact was.

Yes, and we are discussing if it is worth getting into smaller pages
for such usecases (e.g. 4kB ones without hugetlb with 2MB hugepages or
what more even more waste 1GB hugetlb if we dont request 2MB for some
small structs: btw, we have ability to select MAP_HUGE_2MB vs
MAP_HUGE_1GB). I'm thinking about two problems:
- 4kB are swappable and mlock() potentially (?) disarms NUMA autobalacning
- using libnuma often leads to MPOL_BIND which disarms NUMA
autobalancing, BUT apparently there are set_mempolicy(2)/mbind(2) and
since 5.12+ kernel they can take additional flag
MPOL_F_NUMA_BALANCING(!), so this looks like it has potential to move
memory anyway (if way too many tasks are relocated, so would be
memory?). It is available only in recent libnuma as
numa_set_membind_balancing(3), but sadly there's no way via libnuma to
do mbind(MPOL_F_NUMA_BALANCING) for a specific addr only? I mean it
would have be something like MPOL_F_NUMA_BALANCING | MPOL_PREFERRED?
(select one node from many for each node while still allowing
balancing?), but in [1][2] (2024) it is stated that "It's not
legitimate (yet) to use MPOL_PREFERRED + MPOL_F_NUMA_BALANCING.", but
maybe stuff has been improved since then.

Something like:
PGPROC/procarray 2MB page for node#1 - mbind(addr1,
MPOL_F_NUMA_BALANCING | MPOL_PREFERRED, [0,1]);
PGPROC/procarray 2MB page for node#2 - mbind(addr2,
MPOL_F_NUMA_BALANCING | MPOL_PREFERRED, [1,0]);

> have an added level of indirection...

So the most efficient would be the old-way (no indirections) vs
NUMA-way? Can this be done without #ifdefs at all?

> > moves, there'll be traffic.

With MPOL_F_NUMA_BALANCING, that should "auto-tune" in the worst case?

> > I don't have any estimates how often this happens, e.g. for older tasks.

We could measure, kernel 6.16+ has per PID numa_task_migrated in
/proc/{PID}/sched , but I assume we would have to throw backends >>
VCPUs at it, to simulate reality and do some "waves" between different
activity periods of certain pools (I can imagine worst case scenario:
a) pgbench "a" open $VCPU connections, all idle, with scripto to sleep
for a while
b) pgbench "b" open some $VCPU new connections to some other DB, all
active from start (tpcbb or readonly)
c) manually ping CPUs using taskset for each PID all from "b" to
specific NUMA node #2 -- just to simulate unfortunate app working on
every 2nd conn
d) pgbench "a" starts working and hits CPU imbalance -- e.g. NUMA node
#1 is idle, #2 is full, CPU scheduler starts puting "a" backends on
CPUs from #1 , and we should notice PIDs being migrated)

-J.

p.s. I hope i did write in an understandable way, because I had many
interruptions, so if anything is unclear please let me know.

[1] - https://lkml.org/lkml/2024/7/3/352
[2] - https://lkml.rescloud.iu.edu/2402.2/03227.html

---

From: Andres Freund <[email-masked]>
Date: Wed, 9 Jul 2025 12:35:13 -0400

Hi,

> > well add it at the beginning of the free list.

Yea, I don't see any point in adding buffers to the tail instead of to the
front. We probably want more recently used buffers at the front, since they
(and the associated BufferDesc) are more likely to be in a CPU cache.

> optimization.

Optimiziation / simplification.  When building multiple freelists it was
harder to maintain the tail pointer, and since it was never used...

+1 to just applying that part.

> I don't know how significant is the improvement, but if it's measurable we
> could just do that independently of our patches.

I doubt it's really an improvement in any realistic scenario, but it's also
not a regression in any way, since it's never used...

FWIW, I've started to wonder if we shouldn't just get rid of the freelist
entirely. While clocksweep is perhaps minutely slower in a single thread than
the freelist, clock sweep scales *considerably* better [1]. As it's rather
rare to be bottlenecked on clock sweep speed for a single thread (rather then
IO or memory copy overhead), I think it's worth favoring clock sweep.

Also needing to switch between getting buffers from the freelist and the sweep
makes the code more expensive.  I think just having the buffer in the sweep,
with a refcount / usagecount of zero would suffice.

That seems particularly advantageous if we invest energy in making the clock
sweep deal well with NUMA systems, because we don't need have both a NUMA
aware freelist and a NUMA aware clock sweep.

Greetings,

Andres Freund

[1]

A single pg_prewarm of a large relation shows a difference between using the
freelist and not that's around the noise level, whereas 40 parallel
pg_prewarms of seperate relations is over 5x faster when disabling the
freelist.

For the test:
- I modified pg_buffercache_evict_* to put buffers onto the freelist

- Ensured all of shared buffers is allocated by querying
  pg_shmem_allocations_numa, as otherwise the workload is dominated by the
  kernel zeroing out buffers

- used shared_buffers bigger than the data

- data for single threaded is 9.7GB, data for the parallel case is 40
  relations of 610MB each.

- in the single threaded case I pinned postgres to a single core, to make sure
  core-to-core variation doesn't play a role

- single threaded case

  c=1 && psql -Xq -c "select pg_buffercache_evict_all()" -c 'SELECT numa_node, sum(size), count(*) FROM pg_shmem_allocations_numa WHERE size != 0 GROUP BY numa_node;' && pgbench -n -P1 -c$c -j$c -f <(echo "SELECT pg_prewarm('copytest_large');") -t1

  concurrent case:

  c=40 && psql -Xq -c "select pg_buffercache_evict_all()" -c 'SELECT numa_node, sum(size), count(*) FROM pg_shmem_allocations_numa WHERE size != 0 GROUP BY numa_node;' && pgbench -n -P1 -c$c -j$c -f <(echo "SELECT pg_prewarm('copytest_:client_id');") -t1

---

From: Greg Burd <[email-masked]>
Date: Wed, 9 Jul 2025 12:55:51 -0400

> IO or memory copy overhead), I think it's worth favoring clock sweep.

Hey Andres, thanks for spending time on this.  I've worked before on
freelist implementations (last one in LMDB) and I think you're onto
something.  I think it's an innovative idea and that the speed
difference will either be lost in the noise or potentially entirely
mitigated by avoiding duplicate work.

> with a refcount / usagecount of zero would suffice.

If you're not already coding this, I'll jump in. :)
 
> aware freelist and a NUMA aware clock sweep.

100% agree here, very clever approach adapting clock sweep to a NUMA world.

best.

-greg

---

From: Andres Freund <[email-masked]>
Date: Wed, 9 Jul 2025 13:13:04 -0400

Hi,

> Any single-liner pgbench help for how to really easily stress the

Unfortunately it's probably going to be slightly more complicated workloads
that show the effect - the very simplest cases don't go iterate through the
procarray itself anymore.

> - 4kB are swappable and mlock() potentially (?) disarms NUMA autobalacning

I'm not really bought into this being a problem.  If your system has enough
pressure to swap out the PGPROC array, you're so hosed that this won't make a
difference.

> MPOL_F_NUMA_BALANCING | MPOL_PREFERRED, [1,0]);

I'm rather doubtful that it's a good idea to combine numa awareness with numa
balancing. Numa balancing adds latency and makes it much more expensive for
userspace to act in a numa aware way, since it needs to regularly update its
knowledge about where memory resides.

> NUMA-way? Can this be done without #ifdefs at all?

If we used 4k pages for the procarray we would just have ~4 procs on one page,
if that range were marked as interleaved, it'd probably suffice.

> With MPOL_F_NUMA_BALANCING, that should "auto-tune" in the worst case?

I doubt that NUMA balancing is going to help a whole lot here, there are too
many procs on one page for that to be helpful.  One thing that might be worth
doing is to *increase* the size of PGPROC by moving other pieces of data that
are keyed by ProcNumber into PGPROC.

I think the main thing to avoid is the case where all of PGPROC, buffer
mapping table, ... resides on one NUMA node (e.g. because it's the one
postmaster was scheduled on), as the increased memory traffic will lead to
queries on that node being slower than the other node.

Greetings,

Andres Freund

---

From: Andres Freund <[email-masked]>
Date: Wed, 9 Jul 2025 13:23:06 -0400

Hi,

> mitigated by avoiding duplicate work.

Agreed. FWIW, just using clock sweep actually makes things like DROP TABLE
perform better because it doesn't need to maintain the freelist anymore...

> If you're not already coding this, I'll jump in. :)

My experimental patch is literally a four character addition ;), namely adding
"0 &&" to the relevant code in StrategyGetBuffer().

Obviously a real patch would need to do some more work than that.  Feel free
to take on that project, I am not planning on tackling that in near term.

There's other things around this that could use some attention. It's not hard
to see clock sweep be a bottleneck in concurrent workloads - partially due to
the shared maintenance of the clock hand. A NUMAed clock sweep would address
that. However, we also maintain StrategyControl->numBufferAllocs, which is a
significant contention point and would not necessarily be removed by a
NUMAificiation of the clock sweep.

Greetings,

Andres Freund

---

From: Andres Freund <[email-masked]>
Date: Wed, 9 Jul 2025 13:57:36 -0400

Hi,

On 2025-07-08 16:06:00 +0000, Cédric Villemain wrote:
> candidates), I didn't get that they require information not available to any
> process executing code from a module.

> Parts of your code where you assign/define policy could be in one or more
> relevant routines of a "numa profile manager", like in an
> This way it's easier to manage alternative policies, and also to be able to
> adjust when hardware and linux kernel changes.

I am doubtful this makes sense - as you can see patch 05 needs to change a
fair bit of core code to make this work, there's no way we can delegate much
of that to an extension.

But even if it's doable, I think it's *very* premature to focus on such
extensibility at this point - we need to get the basics into a mergeable
state, if you then want to argue for adding extensibility, we can do that at
this stage.  Trying to design this for extensibility from the get go, where
that extensibility is very unlikely to be used widely, seems rather likely to
just tank this entire project without getting us anything in return.

Greetings,

Andres Freund

---

From: Andres Freund <[email-masked]>
Date: Wed, 9 Jul 2025 15:42:26 -0400

Hi,

Thanks for working on this!  I think it's an area we have long neglected...

> sure how exactly that should look. It's possible some combinations of

Wonder if some of it might be worth putting into a multi-valued GUC (like
debug_io_direct).

> BufferBlock and a related BufferDescriptor, and those might again end up

Two more disadvantages:

With OS interleaving postgres doesn't (not easily at least) know about what
maps to what, which means postgres can't do stuff like numa aware buffer
replacement.

With OS interleaving the interleaving is "too fine grained", with pages being
mapped at each page boundary, making it less likely for things like one
strategy ringbuffer to reside on a single numa node.

I wonder if we should *increase* the size of shared_buffers whenever huge
pages are in use and there's padding space due to the huge page
boundaries. Pretty pointless to waste that memory if we can instead use if for
the buffer pool.  Not that big a deal with 2MB huge pages, but with 1GB huge
pages...

> aligns with the buffer interleaving. But there are improvements needed.

I think we might eventually want something more granular than just "node" -
the freelist (and the clock sweep) can become a contention point even within
one NUMA node.  I'm imagining something like an array of freelists/clocksweep
states, where the current numa node selects a subset of the array and the cpu
is used to choose the entry within that list.

But we can do that later, that should be a fairly simple extension of what
you're doing.

> need to make it part of this branch.

I'm really curious about that patch, as I wrote elsewhere in this thread, I
think we should just get rid of the freelist alltogether.  Even if we don't do
so, in a steady state system the clock sweep is commonly much more important
than the freelist...

> rather high max_connections before we use multiple huge pages.

We should probably pad them regardless? Right now sizeof(PGPROC) happens to be
multiple of 64 (i.e. the most common cache line size), but that hasn't always
been the case, and isn't the case on systems with 128 bit cachelines like
common ARMv8 systems.  And having one cacheline hold one backends fast path
states and another backend's xmin doesn't sound like a recipe for good
performance.

Seems like we should also do some reordering of the contents within PGPROC. We
have e.g. have very frequently changing data (->waitStatus, ->lwWaiting) in
the same caceheline as almost immutable data (->pid, ->pgxactoff,
->databaseId,).

> that's simply how NUMA works - page granularity.

Theoretically we could use the "padding" memory at the end of each NUMA node's
PGPROC array to for the 2PC entries, for those we presumably don't care for
locality.  Not sure it's worth the complexity though.

For a while I thought I had a better solution: Given that we're going to waste
all the "padding" memory, why not just oversize the PGPROC array so that it
spans the required number of NUMA nodes?

The problem is that that would lead to ProcNumbers to get much larger, and we
do have other arrays that are keyed by ProcNumber. Which probably makes this
not so great an idea.

> with such examples, I'd like to see those.

I'd not be surprised if there were overhead, adding a level of indirection to
things like ProcArrayGroupClearXid(), GetVirtualXIDsDelayingChkpt(),
SignalVirtualTransaction() probably won't be free.

BUT: For at least some of these a better answer might be to add additional
"dense" arrays like we have for xids etc, so they don't need to trawl through
PGPROCs.

> freelist, I haven't split that - maybe we should?).

I guess it might be worth partitioning the freelist, iterating through a few
thousand links just to discover that there's no free proc on the current numa
node, while holding a spinlock, doesn't sound great. Even if it's likely
rarely a huge issue compared to other costs.

> some custom PG configuration (which the patch [2] proposed to do).

That seems like the right thing to me.

One thing that this patchset afaict doesn't address so far is that there is a
fair bit of other important shared memory that this patch doesn't set up
intelligently e.g. the buffer mapping table itself (but there are loads of
other cases). Because we touch a lot of that memory during startup, most it
will be allocated on whatever NUMA node postmaster was scheduled.  I suspect
that the best we can do for parts of shared memory where we don't have
explicit NUMA awareness is to default to an interleave policy.

> during benchmarking, etc.  It's less dependent on what the CPU

FWIW, I don't think zone_reclaim_mode will commonly do that? Even if enabled,
which I don't think it is anymore by default.  At least huge pages can't be
reclaimed by the kernel, but even when not using huge pages, I think the only
scenario where that would happen is if shared_buffers were swapped out.

Numa balancing might eventually "fix" such an imbalance though.

> +++ b/src/backend/storage/buffer/buf_init.c

I wonder how much of this we should try to put into port/pg_numa.c. Having
direct calls to libnuma code all over the backend will make it rather hard to
add numa awareness for hypothetical platforms not using libnuma compatible
interfaces.

> +static int64 numa_chunk_buffers = -1;

Given that NBuffers is a 32bit quantity, this probably doesn't need to be
64bit...  Anyway, I'm not going to review on that level going forward, the
patch is probably in too early a state for that.

> +	 * run outside postmaster? I don't think so.

It can run in single user mode - but that shouldn't prevent us from using
pg_get_shmem_pagesize().

> +	 * that we don't actually need). Not sure about better way, good for now.

Ugh, not seeing a great way to deal with that either.

> +	 * maybe do this only for sufficiently large areas (e.g. shared buffers,

I think that's right - there's no point in using 1GB pages for anything other
than shared_buffers, we should allocate shared_buffers separately.

> + * Up to this point there was no such dependency on huge pages.

Doing it twice sounds somewhat nasty - but perhaps we could just have the
shmem size infrastructure compute two different numbers, one for use with huge
pages and one without?

> +	while (2 * num_items <= max_items)

Something around this logic leads to a fair bit of imbalance - I started postgres with
huge_page_size=1GB, shared_buffers=4GB on a 2 node system and that results in

postgres[4188255][1]=# SELECT * FROM pg_shmem_allocations_numa WHERE name in ('Buffer Blocks', 'Buffer Descriptors');
┌────────────────────┬───────────┬────────────┐
│        name        │ numa_node │    size    │
├────────────────────┼───────────┼────────────┤
│ Buffer Blocks      │         0 │ 5368709120 │
│ Buffer Blocks      │         1 │ 1073741824 │
│ Buffer Descriptors │         0 │ 1073741824 │
│ Buffer Descriptors │         1 │ 1073741824 │
└────────────────────┴───────────┴────────────┘
(4 rows)

With shared_buffers=8GB postgres failed to start, even though 16 1GB huge
pages are available, as 18GB were requested.

After increasing the limit, the top allocations were as follows:
postgres[4189384][1]=# SELECT * FROM pg_shmem_allocations ORDER BY allocated_size DESC LIMIT 5;
┌──────────────────────┬─────────────┬────────────┬────────────────┐
│         name         │     off     │    size    │ allocated_size │
├──────────────────────┼─────────────┼────────────┼────────────────┤
│ Buffer Blocks        │  1192223104 │ 9663676416 │     9663676416 │
│ PGPROC structures    │ 10970279808 │ 3221733342 │     3221733376 │
│ Fast-Path Lock Array │ 14192013184 │ 3221396544 │     3221396608 │
│ Buffer Descriptors   │    51372416 │ 1140850688 │     1140850688 │
│ (null)               │ 17468590976 │  785020032 │      785020032 │
└──────────────────────┴─────────────┴────────────┴────────────────┘

With a fair bit of imbalance:
postgres[4189384][1]=# SELECT * FROM pg_shmem_allocations_numa WHERE name in ('Buffer Blocks', 'Buffer Descriptors');
┌────────────────────┬───────────┬────────────┐
│        name        │ numa_node │    size    │
├────────────────────┼───────────┼────────────┤
│ Buffer Blocks      │         0 │ 8589934592 │
│ Buffer Blocks      │         1 │ 2147483648 │
│ Buffer Descriptors │         0 │          0 │
│ Buffer Descriptors │         1 │ 2147483648 │
└────────────────────┴───────────┴────────────┘
(4 rows)

Note that the buffer descriptors are all on node 1.

> +	return (buffer / numa_chunk_buffers) % numa_nodes;

FWIW, this is likely rather expensive - when not a compile time constant,
divisions and modulo can take a fair number of cycles.

> +						  Size mem_page_size, Size chunk_size,

Seems like this should be in pg_numa.c?

> +++ b/src/bin/pgbench/pgbench.c

I assume those changes weren't intentionally part of this patch...

> +}			BufferStrategyFreelist;

I think this might be a leftover from measuring performance of a *non*
partitioned freelist.  I saw unnecessar contention between

  BufferStrategyControl->{nextVictimBuffer,buffer_strategy_lock,numBufferAllocs}

and was testing what effect the simplest avoidance scheme has.

I don't this should be part of this patchset.

> +	BufferStrategyFreelist freelists[FLEXIBLE_ARRAY_MEMBER];

Here the reason was that it's silly to put almost-readonly data (like
bgwprocno) onto the same cacheline as very frequently modified data like
->numBufferAllocs.  That causes unnecessary cache misses in many
StrategyGetBuffer() calls, as another backend's StrategyGetBuffer() will
always have modified ->numBufferAllocs and either ->buffer_strategy_lock or
->nextVictimBuffer.

> +		elog(ERROR, "getcpu failed: %m");

Probably should put this into somewhere abstracted away...

> +			freelist_idx = cpu % strategy_ncpus;

As mentioned earlier, modulo is rather expensive for something executed so
frequently...

> +			freelist_idx = node % strategy_nnodes;

Here we shouldn't need modulo, right?

> +	 * we actually need, depending on the GUC? Not a huge amount of memory.

FWIW, I don't think that's a safe assumption anymore. With CXL we can get a)
PCIe attached memory and b) remote memory as a separate NUMA nodes, and that
very well could end up as more NUMA nodes than cores.

Ugh, -ETOOLONG. Gotta schedule some other things...

Greetings,

Andres Freund

---

From: Jakub Wartak <[email-masked]>
Date: Thu, 10 Jul 2025 12:15:52 +0200

> pressure to swap out the PGPROC array, you're so hosed that this won't make a

OK I need to bend here, yet still part of me believes that the
situation where we have hugepages (for 'Buffer Blocks') and yet some
smaller more, but way critical structs are more likely to be swapped
out due to pressure of some backend-gone-wild random mallocs() is
unhealthy to me (especially the fact the OS might prefer swapping on
per node rather than global picture)

> knowledge about where memory resides.

Well the problem is that backends come here and go to random CPUs
often (migrated++ on very high backend counts and non-uniform
workloads in terms of backend-CPU usage), but the autobalancing
doesn't need to be on or off for everything. It could be autobalancing
for a certain memory region and it is not affecting the app in any way
(well, other than those minor page faulting, literally ).

> if that range were marked as interleaved, it'd probably suffice.

OK, this sounds like the best and simplest proposal to me, yet the
patch doesn't do OS-based interleaving for those today. Gonna try that
mlock() sooner or later... ;)

-J.

---

From: Jakub Wartak <[email-masked]>
Date: Thu, 10 Jul 2025 12:16:41 +0200

> Wonder if some of it might be worth putting into a multi-valued GUC (like

Long-term or for experimentation? Also please see below as it is related:

[..]

> very well could end up as more NUMA nodes than cores.

In my earlier apparently very way too naive approach, I've tried to
handle this CXL scenario, but I'm afraid this cannot be done without
further configuration, please see review/use cases [1] and [2]

-J.

[1] https://www.postgresql.org/message-id/attachment/178119/v4-0001-Add-capability-to-interleave-shared-memory-across.patch
- just see sgml/GUC and we have numa_parse_nodestring(3)
[2] https://www.postgresql.org/message-id/aAKPMrX1Uq6quKJy%40ip-10-97-1-34.eu-west-3.compute.internal

---

From: "Burd, Greg" <[email-masked]>
Date: Thu, 10 Jul 2025 08:13:43 -0400

> On Jul 9, 2025, at 1:23 PM, Andres Freund <[email-masked]> wrote:
> 
> Hi,
> 

I started on this last night, making good progress.  Thanks for the inspiration.  I'll create a new thread to track the work and cross-reference when I have something reasonable to show (hopefully later today).

> that.

Working on it.  Other than NUMA-fying clocksweep there is a function have_free_buffer() that might be a tad tricky to re-implement efficiently and/or make NUMA aware.  Or maybe I can remove that too?  It is used in autoprewarm.c and possibly other extensions, but no where else in core.

> However, we also maintain StrategyControl->numBufferAllocs, which is a

Yep, I noted this counter and its potential for contention too.  Fortunately, it seems like it is only used so that "bgwriter can estimate the rate of buffer consumption" which to me opens the door to a less accurate partitioned counter, perhaps something lock-free (no mutex/CAS) that is bucketed then combined when read.

A quick look at bufmgr.c indicates that recent_allocs (which is StrategyControl->numBufferAllocs) is used to track a "moving average" and other voodoo there I've yet to fully grok.  Any thoughts on this approximate count approach?

Also, what are your thoughts on updating the algorithm to CLOCK-Pro [1] while I'm there?  I guess I'd have to try it out, measure it a lot and see if there are any material benefits.  Maybe I'll keep that for a future patch, or at least layer it... back to work!

best.

-greg

[1] https://www.usenix.org/legacy/publications/library/proceedings/usenix05/tech/general/full_papers/jiang/jiang_html/html.html

---

From: Bertrand Drouvot <[email-masked]>
Date: Thu, 10 Jul 2025 14:17:21 +0000

Hi,

On Wed, Jul 09, 2025 at 03:42:26PM -0400, Andres Freund wrote:
> Hi,
> 
> Thanks for working on this!

Indeed, thanks!

> > given a buffer ID. This is helpful later, when building freelist.

I do think this is a big advantage as compare to the OS interleaving.

> the buffer pool.  Not that big a deal with 2MB huge pages, but with 1GB huge

I think that makes sense, except maybe for operations that need to scan
the whole buffer pool (i.e related to BUF_DROP_FULL_SCAN_THRESHOLD)?

> Right now sizeof(PGPROC) happens to be multiple of 64 (i.e. the most common
> cache line size)

Oh right, it's currently 832 bytes and the patch extends that to 840 bytes.

With a bit of reordering:

diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h
index 5cb1632718e..2ed2f94202a 100644
--- a/src/include/storage/proc.h
+++ b/src/include/storage/proc.h
@@ -194,8 +194,6 @@ struct PGPROC
                                                                 * vacuum must not remove tuples deleted by
                                                                 * xid >= xmin ! */

-       int                     procnumber;             /* index in ProcGlobal->allProcs */
-
        int                     pid;                    /* Backend's process ID; 0 if prepared xact */

        int                     pgxactoff;              /* offset into various ProcGlobal->arrays with
@@ -243,6 +241,7 @@ struct PGPROC

        /* Support for condition variables. */
        proclist_node cvWaitLink;       /* position in CV wait list */
+       int                     procnumber;             /* index in ProcGlobal->allProcs */

        /* Info about lock the process is currently waiting for, if any. */
        /* waitLock and waitProcLock are NULL if not currently waiting. */
@@ -268,6 +267,7 @@ struct PGPROC
         */
        XLogRecPtr      waitLSN;                /* waiting for this LSN or higher */
        int                     syncRepState;   /* wait state for sync rep */
+       int                     numa_node;
        dlist_node      syncRepLinks;   /* list link if process is in syncrep queue */

        /*
@@ -321,9 +321,6 @@ struct PGPROC
        PGPROC     *lockGroupLeader;    /* lock group leader, if I'm a member */
        dlist_head      lockGroupMembers;       /* list of members, if I'm a leader */
        dlist_node      lockGroupLink;  /* my member link, if I'm a member */
-
-       /* NUMA node */
-       int                     numa_node;
 };

That could be back to 832 (the order does not make sense logically speaking
though).

Regards,

---

From: Tomas Vondra <[email-masked]>
Date: Thu, 10 Jul 2025 17:20:50 +0200

> over PostgreSQL processing, like Neon is doing.

The complexity of making everything extensible in an arbitrary way. To
make it extensible in a useful, we need to have a reasonably clear idea
what aspects need to be extensible, and what's the goal.

> limited setup options, thus requirement to keep that flexible enough

I'm sorry, I don't want to sound too negative, but "I want arbitrary
extensibility" is not a very useful feedback. I've asked you to give
some examples of policies that'd customize some of the NUMA stuff.

> important or of interest ?

So how exactly would the policy handle this? Right now we're entirely
oblivious to L3, or on-CPU caches in general. We don't even consider the
size of L3 when sizing hash tables in a hashjoin etc.

regards

-- 
Tomas Vondra

---

From: Tomas Vondra <[email-masked]>
Date: Thu, 10 Jul 2025 17:31:45 +0200

> NUMAificiation of the clock sweep.

Wouldn't it make sense to partition the numBufferAllocs too, though? I
don't remember if my hacky experimental patch NUMA-partitioning did that
or I just thought about doing that, but why wouldn't that be enough?

Places that need the "total" count would have to sum the counters, but
it seemed to me most of the places would be fine with the "local" count
for that partition. If we also make sure to "sync" the clocksweeps so as
to not work on just a single partition, that might be enough ...

regards

-- 
Tomas Vondra

---

From: Andres Freund <[email-masked]>
Date: Fri, 11 Jul 2025 12:06:13 -0400

Hi,

> or I just thought about doing that, but why wouldn't that be enough?

It could be solved together with partitioning, yes - that's what I was trying
to reference with the emphasized bit in "would *not necessarily* be removed by
a NUMAificiation of the clock sweep".

Greetings,

Andres Freund

---

From: Andres Freund <[email-masked]>
Date: Fri, 11 Jul 2025 12:14:06 -0400

Hi,

> the whole buffer pool (i.e related to BUF_DROP_FULL_SCAN_THRESHOLD)?

I don't think the increases here are big enough for that to matter, unless
perhaps you're using 1GB huge pages. But if you're concerned about dropping
tables very fast (i.e. you're running schema change heavy regression tests),
you're not going to use 1GB huge pages.

> Oh right, it's currently 832 bytes and the patch extends that to 840 bytes.

I don't think the patch itself is the problem - it really is just happenstance
that it's a multiple of the line size right now. And it's not on common Armv8
platforms...

> That could be back to 832 (the order does not make sense logically speaking

I don't think shrinking the size in a one-off way just to keep the
"accidental" size-is-multiple-of-64 property is promising. It'll just get
broken again.  I think we should:

a) pad the size of PGPROC to a cache line (or even to a subsequent power of 2,
   to make array access cheaper, right now that involves actual
   multiplications rather than shifts or indexed `lea` instructions).

   That's probably just a pg_attribute_aligned

b) Reorder PGPROC to separate frequently modified from almost-read-only data,
   to increase cache hit ratio.

Greetings,

Andres Freund

---

From: "Burd, Greg" <[email-masked]>
Date: Fri, 11 Jul 2025 13:34:26 -0400

> On Jul 10, 2025, at 8:13 AM, Burd, Greg <[email-masked]> wrote:
> 
> 
> Working on it.

For archival sake, and to tie up loose ends I'll link from here to a new thread I just started that proposes the removal of the freelist and the buffer_strategy_lock [1].

That patch set doesn't address any NUMA-related tasks directly, but it should remove some pain when working in that direction by removing code that requires partitioning and locking and...

best.

-greg

[1] https://postgr.es/m/[email-masked]

---

From: Tomas Vondra <[email-masked]>
Date: Thu, 17 Jul 2025 23:11:16 +0200

Hi,

Here's a v2 of the patch series, with a couple changes:

* I simplified the various freelist partitioning by keeping only the
"node" partitioning (so the cpu/pid strategies are gone). Those were
meant for experimenting, but it made the code more complicated so I
ditched it.

* I changed the freelist partitioning scheme a little bit, based on the
discussion in this thread. Instead of having a single "partition" per
NUMA node, there's not a minimum number of partitions (set to 4). So
even if your system is not NUMA, you'll have 4 of them. If you have 2
nodes, you'll still have 4, and each node will get 2. With 3 nodes we
get 6 partitions (we need 2 per node, and we want to keep the number
equal to keep things simple). Once the number of nodes exceeds 4, the
heuristics switches to one partition per node.

I'm aware there's a discussion about maybe simply removing freelists
entirely. If that happens, this becomes mostly irrelevant, of course.

The code should also make sure the freelists "agree" with how the
earlier patch mapped the buffers to NUMA nodes, i.e. the freelist should
only contain buffers from the "correct" NUMA node, etc. I haven't paid
much attention to this - I believe it should work for "nice" values of
shared buffers (when it evenly divides between nodes). But I'm sure it's
possible to confuse that (won't cause crashes, but inefficiency).

* There's now a patch partitioning clocksweep, using the same scheme as
the freelists. I came to the conclusion it doesn't make much sense to
partition these things differently - I can't think of a reason why that
would be advantageous, and it makes it easier to reason about.

The clocksweep partitioning is somewhat harder, because it affects
BgBufferSync() and related code. With the partitioning we now have
multiple "clock hands" for different ranges of buffers, and the clock
sweep needs to consider that. I modified BgBufferSync to simply loop
through the ClockSweep partitions, and do a small cleanup for each.

It does work, as in "it doesn't crash". But this part definitely needs
review to make sure I got the changes to the "pacing" right.

* This new freelist/clocksweep partitioning scheme is however harder to
disable. I now realize the GUC may quite do the trick, and there even is
not a GUC for the clocksweep. I need to think about this, but I'm not
even how feasible it'd be to have two separate GUCs (because of how
these two pieces are intertwined). For now if you want to test without
the partitioning, you need to skip the patch.

I did some quick perf testing on my old xeon machine (2 NUMA nodes), and
the results are encouraging. For a read-only pgbench (2x shared buffers,
within RAM), I saw an increase from 1.1M tps to 1.3M. Not crazy, but not
bad considering the patch is more about consistency than raw throughput.

For a read-write pgbench I however saw some strange drops/increases of
throughput. I suspect this might be due to some thinko in the clocksweep
partitioning, but I'll need to take a closer look.

regards

-- 
Tomas Vondra
From ca651eb85a6656c79fee5aaabc99e4b772b1b8fe Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 27 May 2025 23:08:48 +0200
Subject: [PATCH v2 7/7] NUMA: pin backends to NUMA nodes

 
+	/*
index 6ee4684d1b8..3f88659b49f 100644
 bool		numa_partition_freelist = false;
 
 /* GUC parameters for vacuum */
 int			VacuumBufferUsageLimit = 2048;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 7b718760248..862341e137e 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2156,6 +2156,16 @@ struct config_bool ConfigureNamesBool[] =
 		NULL, NULL, NULL
 	},
 
+	{
index cdeee8dccba..a97741c6707 100644
 extern PGDLLIMPORT bool numa_partition_freelist;

From 0d79d2fb6ab9f1d5b0b3f03e500315135329b09e Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Thu, 22 May 2025 18:39:08 +0200
Subject: [PATCH v2 6/7] NUMA: interleave PGPROC entries

 - PGPROC array / node #0
 - PGPROC array / node #1

 src/backend/storage/ipc/procarray.c    |  61 ++--
 11 files changed, 406 insertions(+), 62 deletions(-)

index e80fbe109cf..928d126d0ee 100644
@@ -574,7 +574,7 @@ TransactionGroupUpdateXidStatus(TransactionId xid, XidStatus status,
 
 		/*
@@ -633,7 +633,7 @@ TransactionGroupUpdateXidStatus(TransactionId xid, XidStatus status,
 	 */
 	while (wakeidx != INVALID_PROC_NUMBER)
 	{
-		PGPROC	   *wakeproc = &ProcGlobal->allProcs[wakeidx];
+		PGPROC	   *wakeproc = ProcGlobal->allProcs[wakeidx];
 
index 78e39e5f866..e28e0f7d3bd 100644

index 777c9a8d555..087279a6a8e 100644

 /*
diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index 1827e052da7..2ce158ca9bd 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -446,7 +446,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 		 * actually fine because procLatch isn't ever freed, so we just can
 		 * potentially set the wrong process' (or no process') latch.
 		 */
-		SetLatch(&ProcGlobal->allProcs[bgwprocno].procLatch);
+		SetLatch(&ProcGlobal->allProcs[bgwprocno]->procLatch);
 	}
 
 	/*
index 2418967def6..82158eeb5d6 100644

 
 /*

 

 	/*

 

 	/*

 
@@ -1622,7 +1622,6 @@ TransactionIdIsInProgress(TransactionId xid)
 	return false;
 }
 
-
 /*
  * Determine XID horizons.
  *
@@ -1740,7 +1739,7 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h)
 	for (int index = 0; index < arrayP->numProcs; index++)
 	{
 		int			pgprocno = arrayP->pgprocnos[index];
-		PGPROC	   *proc = &allProcs[pgprocno];
+		PGPROC	   *proc = allProcs[pgprocno];
 		int8		statusFlags = ProcGlobal->statusFlags[index];
 		TransactionId xid;
 		TransactionId xmin;
@@ -2224,7 +2223,7 @@ GetSnapshotData(Snapshot snapshot)
 			TransactionId xid = UINT32_ACCESS_ONCE(other_xids[pgxactoff]);
 			uint8		statusFlags;

 			/*
@@ -2298,7 +2297,7 @@ GetSnapshotData(Snapshot snapshot)

@@ -2499,7 +2498,7 @@ ProcArrayInstallImportedXmin(TransactionId xmin,
 
@@ -2725,7 +2724,7 @@ GetRunningTransactionData(void)
 
@@ -2756,7 +2755,7 @@ GetRunningTransactionData(void)
 
 			/*
@@ -3006,7 +3005,7 @@ GetVirtualXIDsDelayingChkpt(int *nvxids, int type)
 
@@ -3047,7 +3046,7 @@ HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids, int type)
 
 		GET_VXID_FROM_PGPROC(vxid, *proc);
@@ -3175,7 +3174,7 @@ BackendPidGetProcWithLock(int pid)

 		if (proc->pid == pid)
 		{
@@ -3218,7 +3217,7 @@ BackendXidGetPid(TransactionId xid)
 
 			result = proc->pid;
 			break;
@@ -3287,7 +3286,7 @@ GetCurrentVirtualXIDs(TransactionId limitXmin, bool excludeXmin0,
 
 		if (proc == MyProc)
@@ -3389,7 +3388,7 @@ GetConflictingVirtualXIDs(TransactionId limitXmin, Oid dbOid)
 
@@ -3454,7 +3453,7 @@ SignalVirtualTransaction(VirtualTransactionId vxid, ProcSignalReason sigmode,
 
@@ -3509,7 +3508,7 @@ MinimumActiveBackends(int min)
 
 		/*
@@ -3555,7 +3554,7 @@ CountDBBackends(Oid databaseid)
 
 		if (proc->pid == 0)
@@ -3584,7 +3583,7 @@ CountDBConnections(Oid databaseid)
 
 		if (proc->pid == 0)
@@ -3615,7 +3614,7 @@ CancelDBBackends(Oid databaseid, ProcSignalReason sigmode, bool conflictPending)
 
@@ -3656,7 +3655,7 @@ CountUserBackends(Oid roleid)
 
 		if (proc->pid == 0)
@@ -3719,7 +3718,7 @@ CountOtherDBBackends(Oid databaseId, int *nbackends, int *nprepared)
 
@@ -3785,7 +3784,7 @@ TerminateOtherDBBackends(Oid databaseId)
 
index 62f3471448e..c84a2a5f1bc 100644

 
 
+#include <sched.h>
 #include <signal.h>
 #include <unistd.h>
 #include <sys/time.h>
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
 
+/* NUMA */
 
 /*

+	/*

 
+	/*

 

 

+	/*
+	 * NUMA partitioning
+	 *

 
 		/*

 
 		/*

 

 /*
 
 	return ok;
 }
+
index a11bc71a386..6ee4684d1b8 100644
 bool		numa_partition_freelist = false;
 
 /* GUC parameters for vacuum */
 int			VacuumBufferUsageLimit = 2048;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 0552ed62cc7..7b718760248 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2146,6 +2146,16 @@ struct config_bool ConfigureNamesBool[] =
 		NULL, NULL, NULL
 	},
 
+	{
index 66baf2bf33e..cdeee8dccba 100644
 extern PGDLLIMPORT bool numa_partition_freelist;

 

 
 /*

From c4d51ab87b92f9900e37d42cf74980e87b648a56 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Sun, 8 Jun 2025 18:53:12 +0200
Subject: [PATCH v2 5/7] NUMA: clockweep partitioning

---
 src/backend/storage/buffer/bufmgr.c   | 473 ++++++++++++++------------
 src/backend/storage/buffer/freelist.c | 202 ++++++++---
 src/include/storage/buf_internals.h   |   4 +-
 3 files changed, 424 insertions(+), 255 deletions(-)

diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 5922689fe5d..3d6c834d77c 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3587,6 +3587,23 @@ BufferSync(int flags)
 	TRACE_POSTGRESQL_BUFFER_SYNC_DONE(NBuffers, num_written, num_to_scan);
 }
 
+/*
+ * Information saved between calls so we can determine the strategy
+ * point's advance rate and avoid scanning already-cleaned buffers.
+ *
+ * XXX One value per partition. We don't know how many partitions are
+ * there, so allocate 32, should be enough for the PoC patch.
+ *
+ * XXX might be better to have a per-partition struct with all the info
+ */
+#define MAX_CLOCKSWEEP_PARTITIONS 32
+static bool saved_info_valid = false;
+static int	prev_strategy_buf_id[MAX_CLOCKSWEEP_PARTITIONS];
+static uint32 prev_strategy_passes[MAX_CLOCKSWEEP_PARTITIONS];
+static int	next_to_clean[MAX_CLOCKSWEEP_PARTITIONS];
+static uint32 next_passes[MAX_CLOCKSWEEP_PARTITIONS];
+
+
 /*
  * BgBufferSync -- Write out some dirty buffers in the pool.
  *
@@ -3602,55 +3619,24 @@ bool
 BgBufferSync(WritebackContext *wb_context)
 {
 	/* info obtained from freelist.c */
-	int			strategy_buf_id;
-	uint32		strategy_passes;
 	uint32		recent_alloc;
+	uint32		recent_alloc_partition;
+	int			num_partitions;
 
-	/*
-	 * Information saved between calls so we can determine the strategy
-	 * point's advance rate and avoid scanning already-cleaned buffers.
-	 */
-	static bool saved_info_valid = false;
-	static int	prev_strategy_buf_id;
-	static uint32 prev_strategy_passes;
-	static int	next_to_clean;
-	static uint32 next_passes;
-
-	/* Moving averages of allocation rate and clean-buffer density */
-	static float smoothed_alloc = 0;
-	static float smoothed_density = 10.0;
-
-	/* Potentially these could be tunables, but for now, not */
-	float		smoothing_samples = 16;
-	float		scan_whole_pool_milliseconds = 120000.0;
-
-	/* Used to compute how far we scan ahead */
-	long		strategy_delta;
-	int			bufs_to_lap;
-	int			bufs_ahead;
-	float		scans_per_alloc;
-	int			reusable_buffers_est;
-	int			upcoming_alloc_est;
-	int			min_scan_buffers;
-
-	/* Variables for the scanning loop proper */
-	int			num_to_scan;
-	int			num_written;
-	int			reusable_buffers;
+	/* assume we can hibernate, any partition can set to false */
+	bool		hibernate = true;
 
-	/* Variables for final smoothed_density update */
-	long		new_strategy_delta;
-	uint32		new_recent_alloc;
+	/* get the number of clocksweep partitions, and total alloc count */
+	StrategySyncPrepare(&num_partitions, &recent_alloc);
 
-	/*
-	 * Find out where the freelist clock sweep currently is, and how many
-	 * buffer allocations have happened since our last call.
-	 */
-	strategy_buf_id = StrategySyncStart(&strategy_passes, &recent_alloc);
+	Assert(num_partitions <= MAX_CLOCKSWEEP_PARTITIONS);
 
 	/* Report buffer alloc counts to pgstat */
 	PendingBgWriterStats.buf_alloc += recent_alloc;
 
+	/* average alloc buffers per partition */
+	recent_alloc_partition = (recent_alloc / num_partitions);
+
 	/*
 	 * If we're not running the LRU scan, just stop after doing the stats
 	 * stuff.  We mark the saved state invalid so that we can recover sanely
@@ -3663,223 +3649,282 @@ BgBufferSync(WritebackContext *wb_context)
 	}
 
 	/*
-	 * Compute strategy_delta = how many buffers have been scanned by the
-	 * clock sweep since last time.  If first time through, assume none. Then
-	 * see if we are still ahead of the clock sweep, and if so, how many
-	 * buffers we could scan before we'd catch up with it and "lap" it. Note:
-	 * weird-looking coding of xxx_passes comparisons are to avoid bogus
-	 * behavior when the passes counts wrap around.
-	 */
-	if (saved_info_valid)
-	{
-		int32		passes_delta = strategy_passes - prev_strategy_passes;
-
-		strategy_delta = strategy_buf_id - prev_strategy_buf_id;
-		strategy_delta += (long) passes_delta * NBuffers;
+	 * now process the clocksweep partitions, one by one, using the same
+	 * cleanup that we used for all buffers
+	 *
+	 * XXX Maybe we should randomize the order of partitions a bit, so that
+	 * we don't start from partition 0 all the time? Perhaps not entirely,
+	 * but at least pick a random starting point?
+	 */
+	for (int partition = 0; partition < num_partitions; partition++)
+	{
+		/* info obtained from freelist.c */
+		int			strategy_buf_id;
+		uint32		strategy_passes;
+
+		/* Moving averages of allocation rate and clean-buffer density */
+		static float smoothed_alloc = 0;
+		static float smoothed_density = 10.0;
+
+		/* Potentially these could be tunables, but for now, not */
+		float		smoothing_samples = 16;
+		float		scan_whole_pool_milliseconds = 120000.0;
+
+		/* Used to compute how far we scan ahead */
+		long		strategy_delta;
+		int			bufs_to_lap;
+		int			bufs_ahead;
+		float		scans_per_alloc;
+		int			reusable_buffers_est;
+		int			upcoming_alloc_est;
+		int			min_scan_buffers;
+
+		/* Variables for the scanning loop proper */
+		int			num_to_scan;
+		int			num_written;
+		int			reusable_buffers;
+
+		/* Variables for final smoothed_density update */
+		long		new_strategy_delta;
+		uint32		new_recent_alloc;
+
+		/* buffer range for the clocksweep partition */
+		int			first_buffer;
+		int			num_buffers;
 
-		Assert(strategy_delta >= 0);
+		/*
+		 * Find out where the freelist clock sweep currently is, and how many
+		 * buffer allocations have happened since our last call.
+		 */
+		strategy_buf_id = StrategySyncStart(partition, &strategy_passes,
+											&first_buffer, &num_buffers);
 
-		if ((int32) (next_passes - strategy_passes) > 0)
+		/*
+		 * Compute strategy_delta = how many buffers have been scanned by the
+		 * clock sweep since last time.  If first time through, assume none. Then
+		 * see if we are still ahead of the clock sweep, and if so, how many
+		 * buffers we could scan before we'd catch up with it and "lap" it. Note:
+		 * weird-looking coding of xxx_passes comparisons are to avoid bogus
+		 * behavior when the passes counts wrap around.
+		 */
+		if (saved_info_valid)
 		{
-			/* we're one pass ahead of the strategy point */
-			bufs_to_lap = strategy_buf_id - next_to_clean;
+			int32		passes_delta = strategy_passes - prev_strategy_passes[partition];
+
+			strategy_delta = strategy_buf_id - prev_strategy_buf_id[partition];
+			strategy_delta += (long) passes_delta * num_buffers;
+
+			Assert(strategy_delta >= 0);
+
+			if ((int32) (next_passes[partition] - strategy_passes) > 0)
+			{
+				/* we're one pass ahead of the strategy point */
+				bufs_to_lap = strategy_buf_id - next_to_clean[partition];
 #ifdef BGW_DEBUG
-			elog(DEBUG2, "bgwriter ahead: bgw %u-%u strategy %u-%u delta=%ld lap=%d",
-				 next_passes, next_to_clean,
-				 strategy_passes, strategy_buf_id,
-				 strategy_delta, bufs_to_lap);
+				elog(DEBUG2, "bgwriter ahead: bgw %u-%u strategy %u-%u delta=%ld lap=%d",
+					 next_passes, next_to_clean,
+					 strategy_passes, strategy_buf_id,
+					 strategy_delta, bufs_to_lap);
 #endif
-		}
-		else if (next_passes == strategy_passes &&
-				 next_to_clean >= strategy_buf_id)
-		{
-			/* on same pass, but ahead or at least not behind */
-			bufs_to_lap = NBuffers - (next_to_clean - strategy_buf_id);
+			}
+			else if (next_passes[partition] == strategy_passes &&
+					 next_to_clean[partition] >= strategy_buf_id)
+			{
+				/* on same pass, but ahead or at least not behind */
+				bufs_to_lap = num_buffers - (next_to_clean[partition] - strategy_buf_id);
+#ifdef BGW_DEBUG
+				elog(DEBUG2, "bgwriter ahead: bgw %u-%u strategy %u-%u delta=%ld lap=%d",
+					 next_passes, next_to_clean,
+					 strategy_passes, strategy_buf_id,
+					 strategy_delta, bufs_to_lap);
+#endif
+			}
+			else
+			{
+				/*
+				 * We're behind, so skip forward to the strategy point and start
+				 * cleaning from there.
+				 */
 #ifdef BGW_DEBUG
-			elog(DEBUG2, "bgwriter ahead: bgw %u-%u strategy %u-%u delta=%ld lap=%d",
-				 next_passes, next_to_clean,
-				 strategy_passes, strategy_buf_id,
-				 strategy_delta, bufs_to_lap);
+				elog(DEBUG2, "bgwriter behind: bgw %u-%u strategy %u-%u delta=%ld",
+					 next_passes, next_to_clean,
+					 strategy_passes, strategy_buf_id,
+					 strategy_delta);
 #endif
+				next_to_clean[partition] = strategy_buf_id;
+				next_passes[partition] = strategy_passes;
+				bufs_to_lap = num_buffers;
+			}
 		}
 		else
 		{
 			/*
-			 * We're behind, so skip forward to the strategy point and start
-			 * cleaning from there.
+			 * Initializing at startup or after LRU scanning had been off. Always
+			 * start at the strategy point.
 			 */
 #ifdef BGW_DEBUG
-			elog(DEBUG2, "bgwriter behind: bgw %u-%u strategy %u-%u delta=%ld",
-				 next_passes, next_to_clean,
-				 strategy_passes, strategy_buf_id,
-				 strategy_delta);
+			elog(DEBUG2, "bgwriter initializing: strategy %u-%u",
+				 strategy_passes, strategy_buf_id);
 #endif
-			next_to_clean = strategy_buf_id;
-			next_passes = strategy_passes;
-			bufs_to_lap = NBuffers;
+			strategy_delta = 0;
+			next_to_clean[partition] = strategy_buf_id;
+			next_passes[partition] = strategy_passes;
+			bufs_to_lap = num_buffers;
 		}
-	}
-	else
-	{
-		/*
-		 * Initializing at startup or after LRU scanning had been off. Always
-		 * start at the strategy point.
-		 */
-#ifdef BGW_DEBUG
-		elog(DEBUG2, "bgwriter initializing: strategy %u-%u",
-			 strategy_passes, strategy_buf_id);
-#endif
-		strategy_delta = 0;
-		next_to_clean = strategy_buf_id;
-		next_passes = strategy_passes;
-		bufs_to_lap = NBuffers;
-	}
 
-	/* Update saved info for next time */
-	prev_strategy_buf_id = strategy_buf_id;
-	prev_strategy_passes = strategy_passes;
-	saved_info_valid = true;
+		/* Update saved info for next time */
+		prev_strategy_buf_id[partition] = strategy_buf_id;
+		prev_strategy_passes[partition] = strategy_passes;
+		// FIXME has to happen after all partitions
+		// saved_info_valid = true;
 
-	/*
-	 * Compute how many buffers had to be scanned for each new allocation, ie,
-	 * 1/density of reusable buffers, and track a moving average of that.
-	 *
-	 * If the strategy point didn't move, we don't update the density estimate
-	 */
-	if (strategy_delta > 0 && recent_alloc > 0)
-	{
-		scans_per_alloc = (float) strategy_delta / (float) recent_alloc;
-		smoothed_density += (scans_per_alloc - smoothed_density) /
-			smoothing_samples;
-	}
+		/*
+		 * Compute how many buffers had to be scanned for each new allocation, ie,
+		 * 1/density of reusable buffers, and track a moving average of that.
+		 *
+		 * If the strategy point didn't move, we don't update the density estimate
+		 */
+		if (strategy_delta > 0 && recent_alloc_partition > 0)
+		{
+			scans_per_alloc = (float) strategy_delta / (float) recent_alloc_partition;
+			smoothed_density += (scans_per_alloc - smoothed_density) /
+				smoothing_samples;
+		}
 
-	/*
-	 * Estimate how many reusable buffers there are between the current
-	 * strategy point and where we've scanned ahead to, based on the smoothed
-	 * density estimate.
-	 */
-	bufs_ahead = NBuffers - bufs_to_lap;
-	reusable_buffers_est = (float) bufs_ahead / smoothed_density;
+		/*
+		 * Estimate how many reusable buffers there are between the current
+		 * strategy point and where we've scanned ahead to, based on the smoothed
+		 * density estimate.
+		 */
+		bufs_ahead = num_buffers - bufs_to_lap;
+		reusable_buffers_est = (float) bufs_ahead / smoothed_density;
 
-	/*
-	 * Track a moving average of recent buffer allocations.  Here, rather than
-	 * a true average we want a fast-attack, slow-decline behavior: we
-	 * immediately follow any increase.
-	 */
-	if (smoothed_alloc <= (float) recent_alloc)
-		smoothed_alloc = recent_alloc;
-	else
-		smoothed_alloc += ((float) recent_alloc - smoothed_alloc) /
-			smoothing_samples;
+		/*
+		 * Track a moving average of recent buffer allocations.  Here, rather than
+		 * a true average we want a fast-attack, slow-decline behavior: we
+		 * immediately follow any increase.
+		 */
+		if (smoothed_alloc <= (float) recent_alloc_partition)
+			smoothed_alloc = recent_alloc_partition;
+		else
+			smoothed_alloc += ((float) recent_alloc_partition - smoothed_alloc) /
+				smoothing_samples;
 
-	/* Scale the estimate by a GUC to allow more aggressive tuning. */
-	upcoming_alloc_est = (int) (smoothed_alloc * bgwriter_lru_multiplier);
+		/* Scale the estimate by a GUC to allow more aggressive tuning. */
+		upcoming_alloc_est = (int) (smoothed_alloc * bgwriter_lru_multiplier);
 
-	/*
-	 * If recent_alloc remains at zero for many cycles, smoothed_alloc will
-	 * eventually underflow to zero, and the underflows produce annoying
-	 * kernel warnings on some platforms.  Once upcoming_alloc_est has gone to
-	 * zero, there's no point in tracking smaller and smaller values of
-	 * smoothed_alloc, so just reset it to exactly zero to avoid this
-	 * syndrome.  It will pop back up as soon as recent_alloc increases.
-	 */
-	if (upcoming_alloc_est == 0)
-		smoothed_alloc = 0;
+		/*
+		 * If recent_alloc remains at zero for many cycles, smoothed_alloc will
+		 * eventually underflow to zero, and the underflows produce annoying
+		 * kernel warnings on some platforms.  Once upcoming_alloc_est has gone to
+		 * zero, there's no point in tracking smaller and smaller values of
+		 * smoothed_alloc, so just reset it to exactly zero to avoid this
+		 * syndrome.  It will pop back up as soon as recent_alloc increases.
+		 */
+		if (upcoming_alloc_est == 0)
+			smoothed_alloc = 0;
 
-	/*
-	 * Even in cases where there's been little or no buffer allocation
-	 * activity, we want to make a small amount of progress through the buffer
-	 * cache so that as many reusable buffers as possible are clean after an
-	 * idle period.
-	 *
-	 * (scan_whole_pool_milliseconds / BgWriterDelay) computes how many times
-	 * the BGW will be called during the scan_whole_pool time; slice the
-	 * buffer pool into that many sections.
-	 */
-	min_scan_buffers = (int) (NBuffers / (scan_whole_pool_milliseconds / BgWriterDelay));
+		/*
+		 * Even in cases where there's been little or no buffer allocation
+		 * activity, we want to make a small amount of progress through the buffer
+		 * cache so that as many reusable buffers as possible are clean after an
+		 * idle period.
+		 *
+		 * (scan_whole_pool_milliseconds / BgWriterDelay) computes how many times
+		 * the BGW will be called during the scan_whole_pool time; slice the
+		 * buffer pool into that many sections.
+		 */
+		min_scan_buffers = (int) (num_buffers / (scan_whole_pool_milliseconds / BgWriterDelay));
 
-	if (upcoming_alloc_est < (min_scan_buffers + reusable_buffers_est))
-	{
+		if (upcoming_alloc_est < (min_scan_buffers + reusable_buffers_est))
+		{
 #ifdef BGW_DEBUG
-		elog(DEBUG2, "bgwriter: alloc_est=%d too small, using min=%d + reusable_est=%d",
-			 upcoming_alloc_est, min_scan_buffers, reusable_buffers_est);
+			elog(DEBUG2, "bgwriter: alloc_est=%d too small, using min=%d + reusable_est=%d",
+				 upcoming_alloc_est, min_scan_buffers, reusable_buffers_est);
 #endif
-		upcoming_alloc_est = min_scan_buffers + reusable_buffers_est;
-	}
-
-	/*
-	 * Now write out dirty reusable buffers, working forward from the
-	 * next_to_clean point, until we have lapped the strategy scan, or cleaned
-	 * enough buffers to match our estimate of the next cycle's allocation
-	 * requirements, or hit the bgwriter_lru_maxpages limit.
-	 */
+			upcoming_alloc_est = min_scan_buffers + reusable_buffers_est;
+		}
 
-	num_to_scan = bufs_to_lap;
-	num_written = 0;
-	reusable_buffers = reusable_buffers_est;
+		/*
+		 * Now write out dirty reusable buffers, working forward from the
+		 * next_to_clean point, until we have lapped the strategy scan, or cleaned
+		 * enough buffers to match our estimate of the next cycle's allocation
+		 * requirements, or hit the bgwriter_lru_maxpages limit.
+		 */
 
-	/* Execute the LRU scan */
-	while (num_to_scan > 0 && reusable_buffers < upcoming_alloc_est)
-	{
-		int			sync_state = SyncOneBuffer(next_to_clean, true,
-											   wb_context);
+		num_to_scan = bufs_to_lap;
+		num_written = 0;
+		reusable_buffers = reusable_buffers_est;
 
-		if (++next_to_clean >= NBuffers)
+		/* Execute the LRU scan */
+		while (num_to_scan > 0 && reusable_buffers < upcoming_alloc_est)
 		{
-			next_to_clean = 0;
-			next_passes++;
-		}
-		num_to_scan--;
+			int			sync_state = SyncOneBuffer(next_to_clean[partition], true,
+												   wb_context);
 
-		if (sync_state & BUF_WRITTEN)
-		{
-			reusable_buffers++;
-			if (++num_written >= bgwriter_lru_maxpages)
+			if (++next_to_clean[partition] >= (first_buffer + num_buffers))
 			{
-				PendingBgWriterStats.maxwritten_clean++;
-				break;
+				next_to_clean[partition] = first_buffer;
+				next_passes[partition]++;
+			}
+			num_to_scan--;
+
+			if (sync_state & BUF_WRITTEN)
+			{
+				reusable_buffers++;
+				if (++num_written >= (bgwriter_lru_maxpages / num_partitions))
+				{
+					PendingBgWriterStats.maxwritten_clean++;
+					break;
+				}
 			}
+			else if (sync_state & BUF_REUSABLE)
+				reusable_buffers++;
 		}
-		else if (sync_state & BUF_REUSABLE)
-			reusable_buffers++;
-	}
 
-	PendingBgWriterStats.buf_written_clean += num_written;
+		PendingBgWriterStats.buf_written_clean += num_written;
 
 #ifdef BGW_DEBUG
-	elog(DEBUG1, "bgwriter: recent_alloc=%u smoothed=%.2f delta=%ld ahead=%d density=%.2f reusable_est=%d upcoming_est=%d scanned=%d wrote=%d reusable=%d",
-		 recent_alloc, smoothed_alloc, strategy_delta, bufs_ahead,
-		 smoothed_density, reusable_buffers_est, upcoming_alloc_est,
-		 bufs_to_lap - num_to_scan,
-		 num_written,
-		 reusable_buffers - reusable_buffers_est);
+		elog(DEBUG1, "bgwriter: recent_alloc=%u smoothed=%.2f delta=%ld ahead=%d density=%.2f reusable_est=%d upcoming_est=%d scanned=%d wrote=%d reusable=%d",
+			 recent_alloc_partition, smoothed_alloc, strategy_delta, bufs_ahead,
+			 smoothed_density, reusable_buffers_est, upcoming_alloc_est,
+			 bufs_to_lap - num_to_scan,
+			 num_written,
+			 reusable_buffers - reusable_buffers_est);
 #endif
 
-	/*
-	 * Consider the above scan as being like a new allocation scan.
-	 * Characterize its density and update the smoothed one based on it. This
-	 * effectively halves the moving average period in cases where both the
-	 * strategy and the background writer are doing some useful scanning,
-	 * which is helpful because a long memory isn't as desirable on the
-	 * density estimates.
-	 */
-	new_strategy_delta = bufs_to_lap - num_to_scan;
-	new_recent_alloc = reusable_buffers - reusable_buffers_est;
-	if (new_strategy_delta > 0 && new_recent_alloc > 0)
-	{
-		scans_per_alloc = (float) new_strategy_delta / (float) new_recent_alloc;
-		smoothed_density += (scans_per_alloc - smoothed_density) /
-			smoothing_samples;
+		/*
+		 * Consider the above scan as being like a new allocation scan.
+		 * Characterize its density and update the smoothed one based on it. This
+		 * effectively halves the moving average period in cases where both the
+		 * strategy and the background writer are doing some useful scanning,
+		 * which is helpful because a long memory isn't as desirable on the
+		 * density estimates.
+		 */
+		new_strategy_delta = bufs_to_lap - num_to_scan;
+		new_recent_alloc = reusable_buffers - reusable_buffers_est;
+		if (new_strategy_delta > 0 && new_recent_alloc > 0)
+		{
+			scans_per_alloc = (float) new_strategy_delta / (float) new_recent_alloc;
+			smoothed_density += (scans_per_alloc - smoothed_density) /
+				smoothing_samples;
 
 #ifdef BGW_DEBUG
-		elog(DEBUG2, "bgwriter: cleaner density alloc=%u scan=%ld density=%.2f new smoothed=%.2f",
-			 new_recent_alloc, new_strategy_delta,
-			 scans_per_alloc, smoothed_density);
+			elog(DEBUG2, "bgwriter: cleaner density alloc=%u scan=%ld density=%.2f new smoothed=%.2f",
+				 new_recent_alloc, new_strategy_delta,
+				 scans_per_alloc, smoothed_density);
 #endif
+		}
+
+		/* hibernate if all partitions can hibernate */
+		hibernate &= (bufs_to_lap == 0 && recent_alloc_partition == 0);
 	}
 
+	/* now that we've scanned all partitions, mark the cached info as valid */
+	saved_info_valid = true;
+
 	/* Return true if OK to hibernate */
-	return (bufs_to_lap == 0 && recent_alloc == 0);
+	return hibernate;
 }
 
 /*
diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index e38e5c7ec3d..1827e052da7 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -63,17 +63,27 @@ typedef struct BufferStrategyFreelist
 #define MIN_FREELIST_PARTITIONS		4
 
 /*
- * The shared freelist control information.
+ * Information about one partition of the ClockSweep (on a subset of buffers).
+ *
+ * XXX Should be careful to align this to cachelines, etc.
  */
 typedef struct
 {
 	/* Spinlock: protects the values below */
-	slock_t		buffer_strategy_lock;
+	slock_t		clock_sweep_lock;
+
+	/* range for this clock weep partition */
+	int32		firstBuffer;
+	int32		numBuffers;
 
 	/*
 	 * Clock sweep hand: index of next buffer to consider grabbing. Note that
 	 * this isn't a concrete buffer - we only ever increase the value. So, to
 	 * get an actual buffer, it needs to be used modulo NBuffers.
+	 *
+	 * XXX This is relative to firstBuffer, so needs to be offset properly.
+	 *
+	 * XXX firstBuffer + (nextVictimBuffer % numBuffers)
 	 */
 	pg_atomic_uint32 nextVictimBuffer;
 
@@ -83,6 +93,15 @@ typedef struct
 	 */
 	uint32		completePasses; /* Complete cycles of the clock sweep */
 	pg_atomic_uint32 numBufferAllocs;	/* Buffers allocated since last reset */
+} ClockSweep;
+
+/*
+ * The shared freelist control information.
+ */
+typedef struct
+{
+	/* Spinlock: protects the values below */
+	slock_t		buffer_strategy_lock;
 
 	/*
 	 * Bgworker process to be notified upon activity or -1 if none. See
@@ -99,6 +118,9 @@ typedef struct
 	int			num_partitions_groups;	/* effectively num of NUMA nodes */
 	int			num_partitions_per_group;
 
+	/* clocksweep partitions */
+	ClockSweep *sweeps;
+
 	BufferStrategyFreelist freelists[FLEXIBLE_ARRAY_MEMBER];
 } BufferStrategyControl;
 
@@ -152,6 +174,7 @@ static BufferDesc *GetBufferFromRing(BufferAccessStrategy strategy,
 									 uint32 *buf_state);
 static void AddBufferToRing(BufferAccessStrategy strategy,
 							BufferDesc *buf);
+static ClockSweep *ChooseClockSweep(void);
 
 /*
  * ClockSweepTick - Helper routine for StrategyGetBuffer()
@@ -163,6 +186,7 @@ static inline uint32
 ClockSweepTick(void)
 {
 	uint32		victim;
+	ClockSweep *sweep = ChooseClockSweep();
 
 	/*
 	 * Atomically move hand ahead one buffer - if there's several processes
@@ -170,14 +194,14 @@ ClockSweepTick(void)
 	 * apparent order.
 	 */
 	victim =
-		pg_atomic_fetch_add_u32(&StrategyControl->nextVictimBuffer, 1);
+		pg_atomic_fetch_add_u32(&sweep->nextVictimBuffer, 1);
 
-	if (victim >= NBuffers)
+	if (victim >= sweep->numBuffers)
 	{
 		uint32		originalVictim = victim;
 
 		/* always wrap what we look up in BufferDescriptors */
-		victim = victim % NBuffers;
+		victim = victim % sweep->numBuffers;
 
 		/*
 		 * If we're the one that just caused a wraparound, force
@@ -203,19 +227,23 @@ ClockSweepTick(void)
 				 * could lead to an overflow of nextVictimBuffers, but that's
 				 * highly unlikely and wouldn't be particularly harmful.
 				 */
-				SpinLockAcquire(&StrategyControl->buffer_strategy_lock);
+				SpinLockAcquire(&sweep->clock_sweep_lock);
 
-				wrapped = expected % NBuffers;
+				wrapped = expected % sweep->numBuffers;
 
-				success = pg_atomic_compare_exchange_u32(&StrategyControl->nextVictimBuffer,
+				success = pg_atomic_compare_exchange_u32(&sweep->nextVictimBuffer,
 														 &expected, wrapped);
 				if (success)
-					StrategyControl->completePasses++;
-				SpinLockRelease(&StrategyControl->buffer_strategy_lock);
+					sweep->completePasses++;
+				SpinLockRelease(&sweep->clock_sweep_lock);
 			}
 		}
 	}
-	return victim;
+
+	/* XXX buffer IDs are 1-based, we're calculating 0-based indexes */
+	Assert(BufferIsValid(1 + sweep->firstBuffer + (victim % sweep->numBuffers)));
+
+	return sweep->firstBuffer + victim;
 }
 
 /*
@@ -289,6 +317,28 @@ calculate_partition_index()
 	return index;
 }
 
+/*
+ * ChooseClockSweep
+ *		pick a clocksweep partition based on NUMA node and CPU
+ *
+ * The number of clocksweep partitions may not match the number of NUMA
+ * nodes, but it should not be lower. Each partition should be mapped to
+ * a single NUMA node, but a node may have multiple partitions. If there
+ * are multiple partitions per node (all nodes have the same number of
+ * partitions), we pick the partition using CPU.
+ *
+ * XXX Maybe we should do both the total and "per group" counts a power of
+ * two? That'd allow using shifts instead of divisions in the calculation,
+ * and that's cheaper. But how would that deal with odd number of nodes?
+ */
+static ClockSweep *
+ChooseClockSweep(void)
+{
+	int index = calculate_partition_index();
+
+	return &StrategyControl->sweeps[index];
+}
+
 /*
  * ChooseFreeList
  *		Pick the buffer freelist to use, depending on the CPU and NUMA node.
@@ -404,7 +454,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 	 * the rate of buffer consumption.  Note that buffers recycled by a
 	 * strategy object are intentionally not counted here.
 	 */
-	pg_atomic_fetch_add_u32(&StrategyControl->numBufferAllocs, 1);
+	pg_atomic_fetch_add_u32(&ChooseClockSweep()->numBufferAllocs, 1);
 
 	/*
 	 * First check, without acquiring the lock, whether there's buffers in the
@@ -475,13 +525,17 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 	/*
 	 * Nothing on the freelist, so run the "clock sweep" algorithm
 	 *
-	 * XXX Should we also make this NUMA-aware, to only access buffers from
-	 * the same NUMA node? That'd probably mean we need to make the clock
-	 * sweep NUMA-aware, perhaps by having multiple clock sweeps, each for a
-	 * subset of buffers. But that also means each process could "sweep" only
-	 * a fraction of buffers, even if the other buffers are better candidates
-	 * for eviction. Would that also mean we'd have multiple bgwriters, one
-	 * for each node, or would one bgwriter handle all of that?
+	 * XXX Note that ClockSweepTick() is NUMA-aware, i.e. it only looks at
+	 * buffers from a single partition, aligned with the NUMA node. That
+	 * means it only accesses buffers from the same NUMA node.
+	 *
+	 * XXX That also means each process "sweeps" only a fraction of buffers,
+	 * even if the other buffers are better candidates for eviction. Maybe
+	 * there should be some logic to "steal" buffers from other freelists
+	 * or other nodes?
+	 *
+	 * XXX Would that also mean we'd have multiple bgwriters, one for each
+	 * node, or would one bgwriter handle all of that?
 	 */
 	trycounter = NBuffers;
 	for (;;)
@@ -563,6 +617,41 @@ StrategyFreeBuffer(BufferDesc *buf)
 	SpinLockRelease(&freelist->freelist_lock);
 }
 
+/*
+ * StrategySyncStart -- prepare for sync of all partitions
+ *
+ * Determine the number of clocksweep partitions, and calculate the recent
+ * buffers allocs (as a sum of all the partitions). This allows BgBufferSync
+ * to calculate average number of allocations per partition for the next
+ * sync cycle.
+ *
+ * In addition it returns the count of recent buffer allocs, which is a total
+ * summed from all partitions. The alloc counts are reset after being read,
+ * as the partitions are walked.
+ */
+void
+StrategySyncPrepare(int *num_parts, uint32 *num_buf_alloc)
+{
+	*num_buf_alloc = 0;
+	*num_parts = StrategyControl->num_partitions;
+
+	/*
+	 * We lock the partitions one by one, so not exacly in sync, but that
+	 * should be fine. We're only looking for heuristics anyway.
+	 */
+	for (int i = 0; i < StrategyControl->num_partitions; i++)
+	{
+		ClockSweep *sweep = &StrategyControl->sweeps[i];
+
+		SpinLockAcquire(&sweep->clock_sweep_lock);
+		if (num_buf_alloc)
+		{
+			*num_buf_alloc += pg_atomic_exchange_u32(&sweep->numBufferAllocs, 0);
+		}
+		SpinLockRelease(&sweep->clock_sweep_lock);
+	}
+}
+
 /*
  * StrategySyncStart -- tell BgBufferSync where to start syncing
  *
@@ -570,37 +659,44 @@ StrategyFreeBuffer(BufferDesc *buf)
  * BgBufferSync() will proceed circularly around the buffer array from there.
  *
  * In addition, we return the completed-pass count (which is effectively
- * the higher-order bits of nextVictimBuffer) and the count of recent buffer
- * allocs if non-NULL pointers are passed.  The alloc count is reset after
- * being read.
+ * the higher-order bits of nextVictimBuffer).
+ *
+ * This only considers a single clocksweep partition, as BgBufferSync looks
+ * at them one by one.
  */
 int
-StrategySyncStart(uint32 *complete_passes, uint32 *num_buf_alloc)
+StrategySyncStart(int partition, uint32 *complete_passes,
+				  int *first_buffer, int *num_buffers)
 {
 	uint32		nextVictimBuffer;
 	int			result;
+	ClockSweep *sweep = &StrategyControl->sweeps[partition];
 
-	SpinLockAcquire(&StrategyControl->buffer_strategy_lock);
-	nextVictimBuffer = pg_atomic_read_u32(&StrategyControl->nextVictimBuffer);
-	result = nextVictimBuffer % NBuffers;
+	Assert((partition >= 0) && (partition < StrategyControl->num_partitions));
+
+	SpinLockAcquire(&sweep->clock_sweep_lock);
+	nextVictimBuffer = pg_atomic_read_u32(&sweep->nextVictimBuffer);
+	result = nextVictimBuffer % sweep->numBuffers;
+
+	*first_buffer = sweep->firstBuffer;
+	*num_buffers = sweep->numBuffers;
 
 	if (complete_passes)
 	{
-		*complete_passes = StrategyControl->completePasses;
+		*complete_passes = sweep->completePasses;
 
 		/*
 		 * Additionally add the number of wraparounds that happened before
 		 * completePasses could be incremented. C.f. ClockSweepTick().
 		 */
-		*complete_passes += nextVictimBuffer / NBuffers;
+		*complete_passes += nextVictimBuffer / sweep->numBuffers;
 	}
+	SpinLockRelease(&sweep->clock_sweep_lock);
 
-	if (num_buf_alloc)
-	{
-		*num_buf_alloc = pg_atomic_exchange_u32(&StrategyControl->numBufferAllocs, 0);
-	}
-	SpinLockRelease(&StrategyControl->buffer_strategy_lock);
-	return result;
+	/* XXX buffer IDs start at 1, we're calculating 0-based indexes */
+	Assert(BufferIsValid(1 + sweep->firstBuffer + result));
+
+	return sweep->firstBuffer + result;
 }
 
 /*
@@ -696,6 +792,10 @@ StrategyShmemSize(void)
 	size = add_size(size, MAXALIGN(mul_size(sizeof(BufferStrategyFreelist),
 											num_partitions)));
 
+	/* size of clocksweep partitions (at least one per NUMA node) */
+	size = add_size(size, MAXALIGN(mul_size(sizeof(ClockSweep),
+											num_partitions)));
+
 	return size;
 }
 
@@ -714,6 +814,7 @@ StrategyInitialize(bool init)
 
 	int			num_partitions;
 	int			num_partitions_per_group;
+	char	   *ptr;
 
 	/* */
 	num_partitions = calculate_partition_count(strategy_nnodes);
@@ -736,7 +837,8 @@ StrategyInitialize(bool init)
 	StrategyControl = (BufferStrategyControl *)
 		ShmemInitStruct("Buffer Strategy Status",
 						MAXALIGN(offsetof(BufferStrategyControl, freelists)) +
-						MAXALIGN(sizeof(BufferStrategyFreelist) * num_partitions),
+						MAXALIGN(sizeof(BufferStrategyFreelist) * num_partitions) +
+						MAXALIGN(sizeof(ClockSweep) * num_partitions),
 						&found);
 
 	if (!found)
@@ -758,12 +860,32 @@ StrategyInitialize(bool init)
 
 		SpinLockInit(&StrategyControl->buffer_strategy_lock);
 
-		/* Initialize the clock sweep pointer */
-		pg_atomic_init_u32(&StrategyControl->nextVictimBuffer, 0);
+		/* have to point the sweeps array to right after the freelists */
+		ptr = (char *) StrategyControl +
+				MAXALIGN(offsetof(BufferStrategyControl, freelists)) +
+				MAXALIGN(sizeof(BufferStrategyFreelist) * num_partitions);
+		StrategyControl->sweeps = (ClockSweep *) ptr;
 
-		/* Clear statistics */
-		StrategyControl->completePasses = 0;
-		pg_atomic_init_u32(&StrategyControl->numBufferAllocs, 0);
+		/* Initialize the clock sweep pointers (for all partitions) */
+		for (int i = 0; i < num_partitions; i++)
+		{
+			SpinLockInit(&StrategyControl->sweeps[i].clock_sweep_lock);
+
+			pg_atomic_init_u32(&StrategyControl->sweeps[i].nextVictimBuffer, 0);
+
+			/*
+			 * FIXME This may not quite right, because if NBuffers is not
+			 * a perfect multiple of numBuffers, the last partition will have
+			 * numBuffers set too high. buf_init handles this by tracking the
+			 * remaining number of buffers, and not overflowing.
+			 */
+			StrategyControl->sweeps[i].numBuffers = numBuffers;
+			StrategyControl->sweeps[i].firstBuffer = (numBuffers * i);
+
+			/* Clear statistics */
+			StrategyControl->sweeps[i].completePasses = 0;
+			pg_atomic_init_u32(&StrategyControl->sweeps[i].numBufferAllocs, 0);
+		}
 
 		/* No pending notification */
 		StrategyControl->bgwprocno = -1;
diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h
index 52a71b138f7..b50f9458156 100644
--- a/src/include/storage/buf_internals.h
+++ b/src/include/storage/buf_internals.h
@@ -448,7 +448,9 @@ extern void StrategyFreeBuffer(BufferDesc *buf);
 extern bool StrategyRejectBuffer(BufferAccessStrategy strategy,
 								 BufferDesc *buf, bool from_ring);
 
-extern int	StrategySyncStart(uint32 *complete_passes, uint32 *num_buf_alloc);
+extern void StrategySyncPrepare(int *num_parts, uint32 *num_buf_alloc);
+extern int	StrategySyncStart(int partition, uint32 *complete_passes,
+							  int *first_buffer, int *num_buffers);
 extern void StrategyNotifyBgWriter(int bgwprocno);
 
 extern Size StrategyShmemSize(void);
-- 
2.49.0

From d67278a64983b5f2eb5e408a51e9516aa8fd2264 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Thu, 22 May 2025 18:38:41 +0200
Subject: [PATCH v2 4/7] NUMA: partition buffer freelist

Note: This is (probably) more important than partitioning ProcArray.
---
 src/backend/storage/buffer/buf_init.c |   4 +-
 src/backend/storage/buffer/freelist.c | 372 ++++++++++++++++++++++++--
 src/backend/utils/init/globals.c      |   1 +
 src/backend/utils/misc/guc_tables.c   |  10 +
 src/include/miscadmin.h               |   1 +
 src/include/storage/bufmgr.h          |   8 +
 6 files changed, 367 insertions(+), 29 deletions(-)

diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index e046526c149..e38e5c7ec3d 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -15,14 +15,52 @@
  */
 #include "postgres.h"
 
+#include <sched.h>

+/*
+ * Represents one freelist partition.
+ */
+typedef struct BufferStrategyFreelist
+{
+	/* Spinlock: protects the values below */
+	slock_t		freelist_lock;
+
+	/*
+	 * XXX Not sure why this needs to be aligned like this. Need to ask
+	 * Andres.
+	 */
+	int			firstFreeBuffer __attribute__((aligned(64)));	/* Head of list of
+																 * unused buffers */
+
+	/* Number of buffers consumed from this list. */
+	uint64		consumed;
+}			BufferStrategyFreelist;
+
+/*
+ * The minimum number of partitions we want to have. We want at least this
+ * number of partitions, even on non-NUMA system, as it helps with contention
+ * for buffers. But with multiple NUMA nodes, we want a separate partition per
+ * node. But we may get multiple partitions per node, for low node count.
+ *
+ * With multiple partitions per NUMA node, we pick the partition based on CPU
+ * (or some other parameter).
+ */
+#define MIN_FREELIST_PARTITIONS		4
 
 /*
  * The shared freelist control information.
@@ -39,8 +77,6 @@ typedef struct
 	 */
 	pg_atomic_uint32 nextVictimBuffer;
 
-	int			firstFreeBuffer;	/* Head of list of unused buffers */
-
 	/*
 	 * Statistics.  These counters should be wide enough that they can't
 	 * overflow during a single bgwriter cycle.
@@ -51,13 +87,38 @@ typedef struct
 	/*
 	 * Bgworker process to be notified upon activity or -1 if none. See
 	 * StrategyNotifyBgWriter.
+	 *
+	 * XXX Not sure why this needs to be aligned like this. Need to ask
+	 * Andres. Also, shouldn't the alignment be specified after, like for
+	 * "consumed"?
 	 */
-	int			bgwprocno;
+	int			__attribute__((aligned(64))) bgwprocno;
+
+	/* info about freelist partitioning */
+	int			num_partitions;
+	int			num_partitions_groups;	/* effectively num of NUMA nodes */
+	int			num_partitions_per_group;
+
+	BufferStrategyFreelist freelists[FLEXIBLE_ARRAY_MEMBER];
 } BufferStrategyControl;

+/*
+ * XXX shouldn't this be in BufferStrategyControl? Probably not, we need to
+ * calculate it during sizing, and perhaps it could change before the memory
+ * gets allocated (so we need to remember the values).
+ *
+ * XXX We should probably have a fixed number of partitions, and map the
+ * NUMA nodes to them, somehow (i.e. each node would get some subset of
+ * partitions). Similar to NUM_LOCK_PARTITIONS.
+ *
+ * XXX We don't use the ncpus, really.
+ */
+static int	strategy_ncpus;
+static int	strategy_nnodes;
+
 /*
  * Private (non-shared) state for managing a ring of shared buffers to re-use.
  * This is currently the only kind of BufferAccessStrategy object, but someday
@@ -157,6 +218,104 @@ ClockSweepTick(void)
 	return victim;
 }
 
+/*
+ * Size the clocksweep partitions. At least one partition per NUMA node,
+ * but at least MIN_FREELIST_PARTITIONS partitions in total.
+*/
+static int
+calculate_partition_count(int num_nodes)
+{
+	int		num_per_node = 1;
+
+	while (num_per_node * num_nodes < MIN_FREELIST_PARTITIONS)
+		num_per_node++;
+
+	return (num_nodes * num_per_node);
+}
+
+static int
+calculate_partition_index()
+{
+	int			rc;
+	unsigned	cpu;
+	unsigned	node;
+	int			index;
+
+	Assert(StrategyControl->num_partitions_groups == strategy_nnodes);
+
+	Assert(StrategyControl->num_partitions ==
+		   (strategy_nnodes * StrategyControl->num_partitions_per_group));
+
+	/*
+	 * freelist is partitioned, so determine the CPU/NUMA node, and pick a
+	 * list based on that.
+	 */
+	rc = getcpu(&cpu, &node);
+	if (rc != 0)
+		elog(ERROR, "getcpu failed: %m");
+
+	/*
+	 * XXX We should't get nodes that we haven't considered while building
+	 * the partitions. Maybe if we allow this (e.g. due to support adjusting
+	 * the NUMA stuff at runtime), we should just do our best to minimize
+	 * the conflicts somehow. But it'll make the mapping harder, so for now
+	 * we ignore it.
+	 */
+	if (node > strategy_nnodes)
+		elog(ERROR, "node out of range: %d > %u", cpu, strategy_nnodes);
+
+	/*
+	 * Find the partition. If we have a single partition per node, we can
+	 * calculate the index directly from node. Otherwise we need to do two
+	 * steps, using node and then cpu.
+	 */
+	if (StrategyControl->num_partitions_per_group == 1)
+	{
+		index = (node % StrategyControl->num_partitions);
+	}
+	else
+	{
+		int		index_group,
+				index_part;
+
+		/* two steps - calculate group from node, partition from cpu */
+		index_group = (node % StrategyControl->num_partitions_groups);
+		index_part = (cpu % StrategyControl->num_partitions_per_group);
+
+		index = (index_group * StrategyControl->num_partitions_per_group)
+				+ index_part;
+	}
+
+	return index;
+}
+
+/*
+ * ChooseFreeList
+ *		Pick the buffer freelist to use, depending on the CPU and NUMA node.
+ *
+ * Without partitioned freelists (numa_partition_freelist=false), there's only
+ * a single freelist, so use that.
+ *
+ * With partitioned freelists, we have multiple ways how to pick the freelist
+ * for the backend:
+ *
+ * - one freelist per CPU, use the freelist for CPU the task executes on
+ *
+ * - one freelist per NUMA node, use the freelist for node task executes on
+ *
+ * - use fixed number of freelists, map processes to lists based on PID
+ *
+ * There may be some other strategies, not sure. The important thing is this
+ * needs to be refrecled during initialization, i.e. we need to create the
+ * right number of lists.
+ */
+static BufferStrategyFreelist *
+ChooseFreeList(void)
+{
+	int index = calculate_partition_index();
+	return &StrategyControl->freelists[index];
+}
+
 /*
  * have_free_buffer -- a lockless check to see if there is a free buffer in
  *					   buffer pool.
@@ -168,10 +327,13 @@ ClockSweepTick(void)
 bool
 have_free_buffer(void)
 {
-	if (StrategyControl->firstFreeBuffer >= 0)
-		return true;
-	else
-		return false;
+	for (int i = 0; i < strategy_nnodes; i++)
+	{
+		if (StrategyControl->freelists[i].firstFreeBuffer >= 0)
+			return true;
+	}
+
+	return false;
 }
 
 /*
@@ -193,6 +355,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 	int			bgwprocno;
 	int			trycounter;
 	uint32		local_buf_state;	/* to avoid repeated (de-)referencing */
+	BufferStrategyFreelist *freelist;
 
 	*from_ring = false;
 
@@ -259,31 +422,35 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r

 

 			/*
 			 * If the buffer is pinned or has a nonzero usage_count, we cannot
@@ -305,7 +472,17 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 		}
 	}
 
-	/* Nothing on the freelist, so run the "clock sweep" algorithm */
+	/*
+	 * Nothing on the freelist, so run the "clock sweep" algorithm
+	 *
+	 * XXX Should we also make this NUMA-aware, to only access buffers from
+	 * the same NUMA node? That'd probably mean we need to make the clock
+	 * sweep NUMA-aware, perhaps by having multiple clock sweeps, each for a
+	 * subset of buffers. But that also means each process could "sweep" only
+	 * a fraction of buffers, even if the other buffers are better candidates
+	 * for eviction. Would that also mean we'd have multiple bgwriters, one
+	 * for each node, or would one bgwriter handle all of that?
+	 */
 	trycounter = NBuffers;
 	for (;;)
 	{
@@ -356,7 +533,22 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 void
 StrategyFreeBuffer(BufferDesc *buf)
 {
-	SpinLockAcquire(&StrategyControl->buffer_strategy_lock);
+	BufferStrategyFreelist *freelist;
+
+	/*
+	 * We don't want to call ChooseFreeList() again, because we might get a
+	 * completely different freelist - either a different partition in the
+	 * same group, or even a different group if the NUMA node changed. But
+	 * we can calculate the proper freelist from the buffer id.
+	 */
+	int index = (BufferGetNode(buf->buf_id) * StrategyControl->num_partitions_per_group)
+				 + (buf->buf_id % StrategyControl->num_partitions_per_group);
+
+	Assert((index >= 0) && (index < StrategyControl->num_partitions));
+
+	freelist = &StrategyControl->freelists[index];
+
+	SpinLockAcquire(&freelist->freelist_lock);
 
 	/*
@@ -364,11 +556,11 @@ StrategyFreeBuffer(BufferDesc *buf)

 /*
@@ -432,6 +624,42 @@ StrategyNotifyBgWriter(int bgwprocno)
 	SpinLockRelease(&StrategyControl->buffer_strategy_lock);
 }
 
+	for (int node = 0; node < strategy_nnodes; node++)
 
 /*
  * StrategyShmemSize
@@ -445,12 +673,28 @@ Size
 StrategyShmemSize(void)
 {
 	Size		size = 0;
+	int			num_partitions;
+
+	/* FIXME */
+#ifdef USE_LIBNUMA
+	strategy_ncpus = numa_num_task_cpus();
+	strategy_nnodes = numa_num_task_nodes();
+#else
+	strategy_ncpus = 1;
+	strategy_nnodes = 1;
+#endif
+
+	num_partitions = calculate_partition_count(strategy_nnodes);

+	/* size of freelist partitions (at least one per NUMA node) */
+											num_partitions)));
 
 	return size;
 }
@@ -466,6 +710,13 @@ void
 StrategyInitialize(bool init)
 {
 	bool		found;
+	int			buffers_per_partition;
+
+	int			num_partitions;
+	int			num_partitions_per_group;
+
+	/* */
+	num_partitions = calculate_partition_count(strategy_nnodes);
 
 	/*
 	 * Initialize the shared buffer lookup hashtable.
@@ -484,23 +735,28 @@ StrategyInitialize(bool init)
 	 */
 	StrategyControl = (BufferStrategyControl *)
 		ShmemInitStruct("Buffer Strategy Status",
-						sizeof(BufferStrategyControl),
+						MAXALIGN(offsetof(BufferStrategyControl, freelists)) +
+						MAXALIGN(sizeof(BufferStrategyFreelist) * num_partitions),
 						&found);
 
 	if (!found)
 	{
+		int32	numBuffers = NBuffers / num_partitions;
+
+		while (numBuffers * num_partitions < NBuffers)
+			numBuffers++;
+
+		Assert(numBuffers * num_partitions == NBuffers);
+
 		/*
 		 * Only done once, usually in postmaster
 		 */
 		Assert(init);

-		/*
 
 		/* Initialize the clock sweep pointer */
 		pg_atomic_init_u32(&StrategyControl->nextVictimBuffer, 0);
@@ -511,6 +767,68 @@ StrategyInitialize(bool init)
 
 		/* No pending notification */
 		StrategyControl->bgwprocno = -1;
+
+		/* always a multiple of NUMA nodes */
+		Assert(num_partitions % strategy_nnodes == 0);
+
+		num_partitions_per_group = (num_partitions / strategy_nnodes);
+
+		/* initialize the partitioned clocksweep */
+		StrategyControl->num_partitions = num_partitions;
+		StrategyControl->num_partitions_groups = strategy_nnodes;
+		StrategyControl->num_partitions_per_group = num_partitions_per_group;
+
+		/*
+		 * Rebuild the freelist - right now all buffers are in one huge list,
+		 * we want to rework that into multiple lists. Start by initializing
+		 * the strategy to have empty lists.
+		 */
+		for (int nfreelist = 0; nfreelist < num_partitions; nfreelist++)
+		{
+			BufferStrategyFreelist *freelist;
+
+			freelist = &StrategyControl->freelists[nfreelist];
+
+			freelist->firstFreeBuffer = FREENEXT_END_OF_LIST;
+
+			SpinLockInit(&freelist->freelist_lock);
+		}
+
+		/* buffers per partition */
+		buffers_per_partition = (NBuffers / num_partitions);
+
+		elog(LOG, "NBuffers: %d, nodes %d, ncpus: %d, divide: %d, remain: %d",
+			 NBuffers, strategy_nnodes, strategy_ncpus,
+			 buffers_per_partition, NBuffers - (num_partitions * buffers_per_partition));
+
+		/*
+		 * Walk through the buffers, add them to the correct list. Walk from
+		 * the end, because we're adding the buffers to the beginning.
+		 */
+		for (int i = NBuffers - 1; i >= 0; i--)
+		{
+			BufferDesc *buf = GetBufferDescriptor(i);
+			BufferStrategyFreelist *freelist;
+			int			node;
+			int			index;
+
+			/*
+			 * Split the freelist into partitions, if needed (or just keep the
+			 * freelist we already built in BufferManagerShmemInit().
+			 */
+
+			/* determine NUMA node for buffer, this determines the group */
+			node = BufferGetNode(i);
+
+			/* now calculate the actual freelist index */
+			index = node * num_partitions_per_group + (i % num_partitions_per_group);
+
+			/* add to the right freelist */
+			freelist = &StrategyControl->freelists[index];
+
+			buf->freeNext = freelist->firstFreeBuffer;
+			freelist->firstFreeBuffer = i;
+		}
 	}
 	else
 		Assert(!init);
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index f5359db3656..a11bc71a386 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -148,6 +148,7 @@ int			MaxBackends = 0;
 /* NUMA stuff */
 bool		numa_buffers_interleave = false;
 bool		numa_localalloc = false;
+bool		numa_partition_freelist = false;
 
 /* GUC parameters for vacuum */
 int			VacuumBufferUsageLimit = 2048;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index a21f20800fb..0552ed62cc7 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2136,6 +2136,16 @@ struct config_bool ConfigureNamesBool[] =
 		NULL, NULL, NULL
 	},
 
+	{
+		{"numa_partition_freelist", PGC_POSTMASTER, DEVELOPER_OPTIONS,
index 692871a401f..66baf2bf33e 100644
 
 extern PGDLLIMPORT bool numa_buffers_interleave;
 extern PGDLLIMPORT bool numa_localalloc;
+extern PGDLLIMPORT bool numa_partition_freelist;

From 2faefc2d10dcd9e31e96be5565e82d1904bd7280 Mon Sep 17 00:00:00 2001
From: Andres Freund <[email-masked]>
Date: Mon, 14 Oct 2024 14:10:13 -0400
Subject: [PATCH v2 3/7] freelist: Don't track tail of a freelist

 
 
 	/*

From c0acd3385fa961e56eb435b85bb021e7ce9e2cb8 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Thu, 22 May 2025 18:27:06 +0200
Subject: [PATCH v2 2/7] NUMA: localalloc

 
 /* NUMA stuff */

+#ifdef USE_LIBNUMA
+#include <numa.h>
+#endif
+
 
 /*
index 9570087aa60..a21f20800fb 100644
 
+	{

From 1eab6285dab1fdc78d80f6054ec3278624a662f1 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 6 May 2025 21:12:21 +0200
Subject: [PATCH v2 1/7] NUMA: interleaving buffers

The effect is similar to

     numactl --interleave=all

Notes:

 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+

 

 
+		/*
 
+	/*

+/*

 
 
 	return size;
 }
+
+/*
+ * choose_chunk_buffers
index 94db3e7c976..5922689fe5d 100644

+/* NUMA stuff */
 
index d14b1678e7f..9570087aa60 100644
 
+	{

 

 
 
 retry:

 

 
 	state.ncpus = ncpus;

 static void

 static void
 
 static int
 
 	while (true)
 	{

 static int

 	while (true)
 	{
 
 		if (cpu == thread_cpu)
 			continue;
 
 static void
 
 static bool

 
 /* in localbuf.c */

---

From: Tomas Vondra <[email-masked]>
Date: Thu, 17 Jul 2025 23:14:52 +0200

> Yeah, good catch. I'll look into that next week.

I've been unable to reproduce this issue, but I'm not sure what settings
you actually used for this instance. Can you give me more details how to
reproduce this?

regards

-- 
Tomas Vondra

---

From: Andres Freund <[email-masked]>
Date: Fri, 18 Jul 2025 12:46:44 -0400

Hi,

> Here's a v2 of the patch series, with a couple changes:

Not a deep look at the code, just a quick reply.

> NUMA node, there's not a minimum number of partitions (set to 4). So

I assume s/not/now/?

> the freelists.

Nice!

> I came to the conclusion it doesn't make much sense to partition these
> things differently - I can't think of a reason why that would be
> advantageous, and it makes it easier to reason about.

Agreed.

> through the ClockSweep partitions, and do a small cleanup for each.

That probably makes sense for now. It might need a bit of a larger adjustment
at some point, but ...

> the partitioning, you need to skip the patch.

I think it's totally fair to enable/disable them at the same time. They're so
closely related, that I don't think it really makes sense to measure them
separately.

> bad considering the patch is more about consistency than raw throughput.

Personally I think an 1.18x improvement on a relatively small NUMA machine is
really rather awesome.

> partitioning, but I'll need to take a closer look.

Was that with pinning etc enabled or not?

> +	 * there should be some logic to "steal" buffers from other freelists

I think we *definitely* need "stealing" from other clock sweeps, whenever
there's a meaningful imbalance between the different sweeps.

I don't think we need to be overly precise about it, a small imbalance won't
have that much of an effect. But clearly it doesn't make sense to say that one
backend can only fill buffers in the current partition, that'd lead to massive
performance issues in a lot of workloads.

The hardest thing probably is to make the logic for when to check foreign
clock sweeps cheap enough.

One way would be to do it whenever a sweep wraps around, that'd probably
amortize the cost sufficiently, and I don't think it'd be too imprecise, as
we'd have processed that set of buffers in a row without partitioning as
well. But it'd probably be too coarse when determining for how long to use a
foreign sweep instance. But we probably could address that by rechecking the
balanace more frequently when using a foreign partition.

Another way would be to have bgwriter manage this. Whenever it detects that
one ring is too far ahead, it could set a "avoid this partition" bit, which
would trigger backends that natively use that partition to switch to foreign
partitions that don't currently have that bit set.  I suspect there's a
problem with that approach though, I worry that the amount of time that
bgwriter spends in BgBufferSync() may sometimes be too long, leading to too
much imbalance.

Greetings,

Andres Freund

---

From: Tomas Vondra <[email-masked]>
Date: Fri, 18 Jul 2025 22:48:00 +0200

>> NUMA node, there's not a minimum number of partitions (set to 4). So

Yes.

> That probably makes sense for now. It might need a bit of a larger
adjustment at some point, but ...

I couldn't think of something fundamentally better and not too complex.
I suspect we might want to use multiple bgwriters in the future, and
this scheme seems to be reasonably well suited for that too.

I'm also thinking about having some sort of "unified" partitioning
scheme for all the places partitioning shared buffers. Right now each of
the places does it on it's own, i.e. buff_init, freelist and clocksweep
all have their code splitting NBuffers into partitions. And it should
align. Because what would be the benefit if it didn't? But I guess
having three variants of the same code seems a bit pointless.

I think buff_init should build a common definition of buffer partitions,
and the remaining parts should use that as the source of truth ...

> closely related, that I don't think it really makes sense to measure them

Yeah, that's a fair point.

> Personally I think an 1.18x improvement on a relatively small NUMA machine is

True, but I want to stress out it's just one quick (& simple test). Much
more testing is needed before I can make reliable claims.

> Was that with pinning etc enabled or not?

IIRC it was with everything enabled, except for numa_procs_pin (which
pins backend to NUMA node). I found that to actually harm performance in
some of the tests (even just read-only ones), resulting in uneven usage
of cores and lower throughput.

Agreed.

> balanace more frequently when using a foreign partition.

What you mean by "it"? What would happen after a sweep wraps around?

> bgwriter spends in BgBufferSync() may sometimes be too long, leading to too

I'm afraid having hard "avoid" flags would lead to sudden and unexpected
changes in performance as we enable/disable partitions. I think a good
solution should "smooth it out" somehow, e.g. by not having a true/false
flag, but having some sort of "preference" factor with values between
(0.0, 1.0) which says how much we should use that partition.

I was imagining something like this:

Say we know the number of buffers allocated for each partition (in the
last round), and we (or rather the BgBufferSync) calculate:

    coefficient = 1.0 - (nallocated_partition / nallocated)

and then use that to "correct" which partition to allocate buffers from.
Or maybe just watch how far from the "fair share" we were in the last
interval, and gradually increase/decrease the "partition preference"
which would say how often we need to "steal" from other partitions.

E.g. we find nallocated_partition is 2x the fair share, i.e.

   nallocated_partition / (nallocated / nparts) = 2.0

Then we say 25% of the time look at some other partition, to "cut" the
imbalance in half. And then repeat that in the next cycle, etc.

So a process would look at it's "home partition" by default, but it's
"roll a dice" first and if above the calculated probability it'd pick
some other partition instead (this would need to be done so that it gets
balanced overall).

If the bgwriter interval is too long, maybe the recalculation could be
triggered regularly after any of the clocksweeps wraps around, or after
some number of allocations, or something like that.

regards

-- 
Tomas Vondra

---

From: Andres Freund <[email-masked]>
Date: Fri, 18 Jul 2025 17:03:59 -0400

Hi,

> of cores and lower throughput.

FWIW, I really doubt that something like numa_procs_pin is viable outside of
very narrow niches until we have a *lot* more infrastructure in place. Like PG
would need to be threaded, we'd need a separation between thread and
connection and an executor that'd allow us to switch from working on one query
to working on another query.

> What you mean by "it"?

it := Considering switching back from using a "foreign" clock sweep instance
whenever the sweep wraps around.

> What would happen after a sweep wraps around?

The scenario I'm worried about is this:

1) a bunch of backends read buffers on numa node A, using the local clock
   sweep instance

2) due to all of that activity, the clock sweep advances much faster than the
   clock sweep for numa node B

3) the clock sweep on A wraps around, we discover the imbalance, and all the
   backend switch to scanning on numa node B, moving that clock sweep ahead
   much more aggressively

4) clock sweep on B wraps around, there's imbalance the other way round now,
   so they all switch back to A

> (0.0, 1.0) which says how much we should use that partition.

Yea, I think that's a fair worry.

> some other partition instead (this would need to be done so that it gets

That does sound reasonable.

> some number of allocations, or something like that.

I'm pretty sure the bgwriter might not be often enough and not predictably
frequently running for that.

Greetings,

Andres Freund

---

From: Jakub Wartak <[email-masked]>
Date: Fri, 25 Jul 2025 12:27:11 +0200

> you actually used for this instance. Can you give me more details how to

Better late than never, well feel free to partially ignore me, i've
missed that it is known issue as per FIXME there, but I would just rip
out that commented out `if(numa_proc_interleave)` from
FastPathLockShmemSize() and PGProcShmemSize() unless you want to save
those memory pages of course (in case of no-NUMA). If you do want to
save those pages I think we have problem:

For complete picture, steps:

1. patch -p1 < v2-0001-NUMA-interleaving-buffers.patch
2. patch -p1 < v2-0006-NUMA-interleave-PGPROC-entries.patch

BTW the pgbench accidentinal ident is still there (part of v2-0001 patch))
14 out of 14 hunks FAILED -- saving rejects to file
src/bin/pgbench/pgbench.c.rej

3. As I'm just applying 0001 and 0006, I've got two simple rejects,
but fixed it (due to not applying missing numa_ freelist patches).
That's intentional on my part, because I wanted to play just with
those two.

4. Then I uncomment those two "if (numa_procs_interleave)" related for
optional memory shm initialization - add_size() and so on (that have
XXX comment above that it is causing bootstrap issues)

5. initdb with numa_procs_interleave=on, huge_pages = on (!), start, it is ok

6. restart with numa_procs_interleave=off, which gets me to every bg
worker crashing e.g.:

(gdb) where
#0  __new_sem_wait_fast (definitive_result=1, sem=sem@entry=0x0) at
./nptl/sem_waitcommon.c:136
#1  __new_sem_trywait (sem=sem@entry=0x0) at ./nptl/sem_wait.c:81
#2  0x0000563e2d6e4d5c in PGSemaphoreReset (sema=0x0) at
../src/backend/port/posix_sema.c:302
#3  0x0000563e2d774d93 in InitAuxiliaryProcess () at
../src/backend/storage/lmgr/proc.c:995
#4  0x0000563e2d6e9252 in AuxiliaryProcessMainCommon () at
../src/backend/postmaster/auxprocess.c:65
#5  0x0000563e2d6eb683 in CheckpointerMain (startup_data=<optimized
out>, startup_data_len=<optimized out>) at
../src/backend/postmaster/checkpointer.c:190
#6  0x0000563e2d6ec363 in postmaster_child_launch
(child_type=child_type@entry=B_CHECKPOINTER, child_slot=249,
startup_data=startup_data@entry=0x0,
    startup_data_len=startup_data_len@entry=0,
client_sock=client_sock@entry=0x0) at
../src/backend/postmaster/launch_backend.c:290
#7  0x0000563e2d6ee29a in StartChildProcess
(type=type@entry=B_CHECKPOINTER) at
../src/backend/postmaster/postmaster.c:3973
#8  0x0000563e2d6f17a6 in PostmasterMain (argc=argc@entry=3,
argv=argv@entry=0x563e377cc0e0) at
../src/backend/postmaster/postmaster.c:1386
#9  0x0000563e2d4948fc in main (argc=3, argv=0x563e377cc0e0) at
../src/backend/main/main.c:231

notice sema=0x0, because:
#3  0x000056050928cd93 in InitAuxiliaryProcess () at
../src/backend/storage/lmgr/proc.c:995
995             PGSemaphoreReset(MyProc->sem);
(gdb) print MyProc
$1 = (PGPROC *) 0x7f09a0c013b0
(gdb) print MyProc->sem
$2 = (PGSemaphore) 0x0

or with printfs:

2025-07-25 11:17:23.683 CEST [21772] LOG:  in InitProcGlobal
PGPROC=0x7f9de827b880 requestSize=148770
// after proc && ptr manipulation:
2025-07-25 11:17:23.683 CEST [21772] LOG:  in InitProcGlobal
PGPROC=0x7f9de827bdf0 requestSize=148770 procs=0x7f9de827b880
ptr=0x7f9de827bdf0
[..initialization of aux PGPROCs i=0.., still fromInitProcGlobal(),
each gets proper sem allocated as one would expect:]
[..for i loop:]
2025-07-25 11:17:23.689 CEST [21772] LOG:  i=136 ,
proc=0x7f9de8600000, proc->sem=0x7f9da4e04438
2025-07-25 11:17:23.689 CEST [21772] LOG:  i=137 ,
proc=0x7f9de8600348, proc->sem=0x7f9da4e044b8
2025-07-25 11:17:23.689 CEST [21772] LOG:  i=138 ,
proc=0x7f9de8600690, proc->sem=0x7f9da4e04538
[..but then in the children codepaths, out of the blue in
InitAuxilaryProcess the whole MyProc looks like it would memsetted to
zeros:]
2025-07-25 11:17:23.693 CEST [21784] LOG:  auxiliary process using
MyProc=0x7f9de8600000 auxproc=0x7f9de8600000 proctype=0
MyProcPid=21784 MyProc->sem=(nil)

above got pgproc slot i=136 with addr 0x7f9de8600000 and later that
auxiliary is launched but somehow something NULLified ->sem there
(according to gdb , everything is zero there)

7. Original patch v2-0006 (with commented out 2x if
numa_procs_interleave), behaves OK, so in my case here with 1x NUMA
node that gives add_size(.., 1+1 * 2MB)=4MB

2025-07-25 11:38:54.131 CEST [23939] LOG:  in InitProcGlobal
PGPROC=0x7f25cbe7b880 requestSize=4343074
2025-07-25 11:38:54.132 CEST [23939] LOG:  in InitProcGlobal
PGPROC=0x7f25cbe7bdf0 requestSize=4343074 procs=0x7f25cbe7b880
ptr=0x7f25cbe7bdf0

so something is zeroing out all those MyProc structures apparently on
startup (probably due to some wrong alignment maybe somewhere ?) I was
thinking about trapping via mprotect() this single i=136
0x7f9de8600000 PGPROC to see what is resetting it, but oh well,
mprotect() works only on whole pages...

-J.

---

From: Tomas Vondra <[email-masked]>
Date: Fri, 25 Jul 2025 12:51:41 +0200

> XXX comment above that it is causing bootstrap issues)

Ah, I didn't realize you uncommented these "if" conditions. In that case
the crash is not very surprising, because the actual initialization in
InitProcGlobal ignores the GUCs and just assumes it's enabled. But
without the extra padding that likely messes up something. Or something
allocated later "overwrites" the some of the memory.

I need to clean this up, to actually consider the GUC properly.

FWIW I do have a new patch version that I plan to share in a day or two,
once I get some numbers. It didn't change this particular part, though,
it's more about the buffers/freelists/clocksweep. I'll work on PGPROC
next, I think.

regards

-- 
Tomas Vondra

---

From: Tomas Vondra <[email-masked]>
Date: Mon, 28 Jul 2025 16:19:07 +0200

Hi,

Here's a somewhat cleaned up v3 of this patch series, with various
improvements and a lot of cleanup. Still WIP, but I hope it resolves the
various crashes reported for v2, but it still requires --with-libnuma
(it won't build without it).

I'm aware there's an ongoing discussion about removing the freelists,
and changing the clocksweep in some way. If that happens, the relevant
parts of this series will need some adjustment, of course. I haven't
looked into that yet, I plan to review those patches soon.

main changes in v3
------------------

1) I've introduced "registry" of the buffer partitions (imagine a small
array of structs), serving as a source of truth for places that need
info about the partitions (range of buffers, ...).

With v2 there was no "shared definition" - the shared buffers, freelist
and clocksweep did their own thing. But per the discussion it doesn't
really make much sense for to partition buffers in different ways.

So in v3 the 0001 patch defines the partitions, records them in shared
memory (in a small array), and the later parts just reuse this.

I also added a pg_buffercache_partitions() listing the partitions, with
first/last buffer, etc. The freelist/clocksweep patches add additional
information.

2) The PGPROC part introduces a similar registry, even though there are
no other patches building on this. But it seemed useful to have a clear
place recording this info.

There's also a view pg_buffercache_pgproc. The pg_buffercache location
is a bit bogus - it has nothing to do with buffers, but it was good
enough for now.

3) The PGPROC partitioning is reworked and should fix the crash with the
GUC set to "off".

4) This still doesn't do anything about "balancing" the clocksweep. I
have some ideas how to do that, I'll work on that next.

simple benchmark
----------------

I did a simple benchmark, measuring pgbench throughput with scale still
fitting into RAM, but much larger (~2x) than shared buffers. See the
attached test script, testing builds with more and more of the patches.

I'm attaching results from two different machines (the "usual" 2P xeon
and also a much larger cloud instance with EPYC/Genoa) - both the raw
CSV files, with average tps and percentiles, and PDFs. The PDFs also
have a comparison either to the "preceding" build (right side), or to
master (below the table).

There's results for the three "pgbench pinning" strategies, and that can
have pretty significant impact (colocated generally performs much better
than either "none" or "random").

For the "bigger" machine (wiuth 176 cores) the incremental results look
like this (for pinning=none, i.e. regular pgbench):

      mode   s_b buffers localal no-tail freelist sweep pgproc pinning
  ====================================================================
  prepared  16GB     99%    101%    100%     103%  111%    99%    102%
            32GB     98%    102%     99%     103%  107%   101%    112%
             8GB     97%    102%    100%     102%  101%   101%    106%
  --------------------------------------------------------------------
    simple  16GB    100%    100%     99%     105%  108%    99%    108%
            32GB     98%    101%    100%     103%  100%   101%     97%
             8GB    100%    100%    101%      99%  100%   104%    104%

The way I read this is that the first three patches have about no impact
on throughput. Then freelist partitioning and (especially) clocksweep
partitioning can help quite a bit. pgproc is again close to ~0%, and
PGPROC pinning can help again (but this part is merely experimental).

For the xeon the differences (in either direction) are much smaller, so
I'm not going to post it here. It's in the PDF, though.

I think this looks reasonable. The way I see this patch series is not
about improving peak throughput, but more about reducing imbalance and
making the behavior more consistent.

The results are more a confirmation there's not some sort of massive
overhead, somewhere. But I'll get to this in a minute.

To quantify this kind of improvement, I think we'll need tests that
intentionally cause (or try to) imbalance. If you have ideas for such
tests, let me know.

overhead of partitioning calculation
------------------------------------

Regarding the "overhead", while the results look mostly OK, I think
we'll need to rethink the partitioning scheme - particularly how the
partition size is calculated. The current scheme has to use %, which can
be somewhat expensive.

The 0001 patch calculates a "chunk size", which is the smallest number
of buffers it can "assign" to a NUMA node. This depends on how many
buffer descriptors fit onto a single memory page, and it's either 512KB
(with 4KB pages), or 256MB (with 2MB huge pages). And then each NUMA
node gets multiple chunks, to cover shared_buffers/num_nodes. But this
can be an arbitrary number - it minimizes the imbalance, but it also
forces the use of % and / in the formulas.

AFAIK if we required the partitions to be 2^k multiples of the chunk
size, we could switch to using shifts and masking. Which is supposed to
be much faster. But I haven't measured this, and the cost is that some
of the nodes could get much less memory. Maybe that's fine.

reserving number of huge pages
------------------------------

The other thing I realized is that partitioning buffers with huge pages
is quite tricky, and can easily lead to SIGBUS when accessing the memory
later. The crashes I saw happen like this:

1) figure # of pages needed (using shared_memory_size_in_huge_pages)

   This can be 16828 for shared_buffers=32GB.

2) make sure there's enough huge pages

   echo 16828 > /proc/sys/vm/nr_hugepages

3) start postgres - everything seems to works just fine

4) query pg_buffercache_numa - triggers SIGBUS accessing memory for a
valid buffer (usually ~2GB from the end)

It took me ages to realize what's happening, but it's very simple. The
nr_hugepages is a global limit, but it's also translated into limits for
each NUMA node. So when you write 16828 to it, in a 4-node system each
node gets 1/4 of that. See

  $ numastat -cm

Then we do the mmap(), and everything looks great, because there really
is enough huge pages and the system can allocate memory from any NUMA
node it needs.

And then we come around, and do the numa_tonode_memory(). And that's
where the issues start, because AFAIK this does not check the per-node
limit of huge pages in any way. It just appears to work. And then later,
when we finally touch the buffer, it tries to actually allocate the
memory on the node, and realizes there's not enough huge pages. And
triggers the SIGBUS.

You may ask why the per-node limit is too low. We still need just
shared_memory_size_in_huge_pages, right? And if we were partitioning the
whole memory segment, that'd be true. But we only to that for shared
buffers, and there's a lot of other shared memory - could be 1-2GB or
so, depending on the configuration.

And this gets placed on one of the nodes, and it counts against the
limit on that particular node. And so it doesn't have enough huge pages
to back the partition of shared buffers.

The only way around this I found is by inflating the number of huge
pages, significantly above the shared_memory_size_in_huge_pages value.
Just to make sure the nodes get enough huge pages.

I don't know what to do about this. It's quite annoying. If we only used
huge pages for the partitioned parts, this wouldn't be a problem.

I also realize this can be used to make sure the memory is balanced on
NUMA systems. Because if you set nr_hugepages, the kernel will ensure
the shared memory is distributed on all the nodes.

It won't have the benefits of "coordinating" the buffers and buffer
descriptors, and so on. But it will be balanced.

regards

-- 
Tomas Vondra
From 3b3a929b007205c46b3c193cf38e3edf71084af7 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 27 May 2025 23:08:48 +0200
Subject: [PATCH v3 7/7] NUMA: pin backends to NUMA nodes

diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index 11259151a7d..dbb4cbb1bfa 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -766,6 +766,27 @@ InitProcess(void)
 	}
 	MyProcNumber = GetNumberFromPGProc(MyProc);
 
+	/*

+	{
 
2.50.1

From 5addb5973ce571debebf07b17adc07eb828a48ee Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Thu, 22 May 2025 18:39:08 +0200
Subject: [PATCH v3 6/7] NUMA: interleave PGPROC entries

 - PGPROC array / node #0
 - PGPROC array / node #1

With the feature disabled, there's only a single "partition" for all
PGPROC entries.

Similarly to the buffer partitioning, this introduces a small "registry"
of partitions, as a source of truth. And then also a new "system" view
"pg_buffercache_pgproc" showing basic infromation abouut the partitions.

Note: The scheduler may migrate the process to a different CPU/node
later. Maybe we should consider pinning the process to the node?
---
 .../pg_buffercache--1.6--1.7.sql              |  19 +
 contrib/pg_buffercache/pg_buffercache_pages.c |  94 +++
 src/backend/access/transam/clog.c             |   4 +-
 src/backend/postmaster/pgarch.c               |   2 +-
 src/backend/postmaster/walsummarizer.c        |   2 +-
 src/backend/storage/buffer/buf_init.c         |   2 -
 src/backend/storage/buffer/freelist.c         |   2 +-
 src/backend/storage/ipc/procarray.c           |  63 +-
 src/backend/storage/lmgr/lock.c               |   6 +-
 src/backend/storage/lmgr/proc.c               | 565 +++++++++++++++++-
 src/backend/utils/init/globals.c              |   1 +
 src/backend/utils/misc/guc_tables.c           |  10 +
 src/include/miscadmin.h                       |   1 +
 src/include/storage/proc.h                    |  14 +-
 src/tools/pgindent/typedefs.list              |   1 +
 15 files changed, 722 insertions(+), 64 deletions(-)

diff --git a/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql b/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
index b7d8ea45ed7..c48950a9d3b 100644
--- a/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
+++ b/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
@@ -23,3 +23,22 @@ REVOKE ALL ON pg_buffercache_partitions FROM PUBLIC;
 
 GRANT EXECUTE ON FUNCTION pg_buffercache_partitions() TO pg_monitor;
 GRANT SELECT ON pg_buffercache_partitions TO pg_monitor;
+
+-- Register the new functions.
+CREATE OR REPLACE FUNCTION pg_buffercache_pgproc()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_pgproc'
+LANGUAGE C PARALLEL SAFE;
+
+-- Create a view for convenient access.
+CREATE VIEW pg_buffercache_pgproc AS
+	SELECT P.* FROM pg_buffercache_pgproc() AS P
+	(partition integer,
+	 numa_node integer, num_procs integer, pgproc_ptr bigint, fastpath_ptr bigint);
+
+-- Don't want these to be available to public.
+REVOKE ALL ON FUNCTION pg_buffercache_pgproc() FROM PUBLIC;
+REVOKE ALL ON pg_buffercache_pgproc FROM PUBLIC;
+
+GRANT EXECUTE ON FUNCTION pg_buffercache_pgproc() TO pg_monitor;
+GRANT SELECT ON pg_buffercache_pgproc TO pg_monitor;
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 5169655ae78..22396f36c09 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -15,6 +15,7 @@
 #include "port/pg_numa.h"
 #include "storage/buf_internals.h"
 #include "storage/bufmgr.h"
+#include "storage/proc.h"
 #include "utils/rel.h"

@@ -28,6 +29,7 @@
 
 #define NUM_BUFFERCACHE_NUMA_ELEM	3
 #define NUM_BUFFERCACHE_PARTITIONS_ELEM	11
+#define NUM_BUFFERCACHE_PGPROC_ELEM 5
 
 PG_MODULE_MAGIC_EXT(
 					.name = "pg_buffercache",
@@ -102,6 +104,7 @@ PG_FUNCTION_INFO_V1(pg_buffercache_evict);
 PG_FUNCTION_INFO_V1(pg_buffercache_evict_relation);
 PG_FUNCTION_INFO_V1(pg_buffercache_evict_all);
 PG_FUNCTION_INFO_V1(pg_buffercache_partitions);
+PG_FUNCTION_INFO_V1(pg_buffercache_pgproc);

 /* Only need to touch memory once per backend process lifetime */
@@ -905,3 +908,94 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 	else
 		SRF_RETURN_DONE(funcctx);
 }
+
+/*
+ * Inquire about partitioning of PGPROC array.
+ */
+Datum
+pg_buffercache_pgproc(PG_FUNCTION_ARGS)
+{
+	FuncCallContext *funcctx;
+	MemoryContext oldcontext;
+	TupleDesc	tupledesc;
+	TupleDesc	expected_tupledesc;
+	HeapTuple	tuple;
+	Datum		result;
+
+	if (SRF_IS_FIRSTCALL())
+	{
+		funcctx = SRF_FIRSTCALL_INIT();
+
+		/* Switch context when allocating stuff to be used in later calls */
+		oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
+
+		if (get_call_result_type(fcinfo, NULL, &expected_tupledesc) != TYPEFUNC_COMPOSITE)
+			elog(ERROR, "return type must be a row type");
+
+		if (expected_tupledesc->natts != NUM_BUFFERCACHE_PGPROC_ELEM)
+			elog(ERROR, "incorrect number of output arguments");
+
+		/* Construct a tuple descriptor for the result rows. */
+		tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 1, "partition",
+						   INT4OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 2, "numa_node",
+						   INT4OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 3, "num_procs",
+						   INT4OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 4, "pgproc_ptr",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 5, "fastpath_ptr",
+						   INT8OID, -1, 0);
+
+		funcctx->user_fctx = BlessTupleDesc(tupledesc);
+
+		/* Return to original context when allocating transient memory */
+		MemoryContextSwitchTo(oldcontext);
+
+		/* Set max calls and remember the user function context. */
+		funcctx->max_calls = ProcPartitionCount();
+	}
+
+	funcctx = SRF_PERCALL_SETUP();
+
+	if (funcctx->call_cntr < funcctx->max_calls)
+	{
+		uint32		i = funcctx->call_cntr;
+
+		int			numa_node,
+					num_procs;
+
+		void	   *pgproc_ptr,
+				   *fastpath_ptr;
+
+		Datum		values[NUM_BUFFERCACHE_PGPROC_ELEM];
+		bool		nulls[NUM_BUFFERCACHE_PGPROC_ELEM];
+
+		ProcPartitionGet(i, &numa_node, &num_procs,
+						 &pgproc_ptr, &fastpath_ptr);
+
+		values[0] = Int32GetDatum(i);
+		nulls[0] = false;
+
+		values[1] = Int32GetDatum(numa_node);
+		nulls[1] = false;
+
+		values[2] = Int32GetDatum(num_procs);
+		nulls[2] = false;
+
+		values[3] = PointerGetDatum(pgproc_ptr);
+		nulls[3] = false;
+
+		values[4] = PointerGetDatum(fastpath_ptr);
+		nulls[4] = false;
+
+		/* Build and return the tuple. */
+		tuple = heap_form_tuple((TupleDesc) funcctx->user_fctx, values, nulls);
+		result = HeapTupleGetDatum(tuple);
+
+		SRF_RETURN_NEXT(funcctx, result);
+	}
+	else
+		SRF_RETURN_DONE(funcctx);
+}
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index e80fbe109cf..928d126d0ee 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -574,7 +574,7 @@ TransactionGroupUpdateXidStatus(TransactionId xid, XidStatus status,
 	/* Walk the list and update the status of all XIDs. */
 	while (nextidx != INVALID_PROC_NUMBER)
 	{
-		PGPROC	   *nextproc = &ProcGlobal->allProcs[nextidx];
+		PGPROC	   *nextproc = ProcGlobal->allProcs[nextidx];
 		int64		thispageno = nextproc->clogGroupMemberPage;
 
 		/*

 
 
 /*
diff --git a/src/backend/storage/buffer/buf_init.c b/src/backend/storage/buffer/buf_init.c
index 5b65a855b29..fb52039e1a6 100644
--- a/src/backend/storage/buffer/buf_init.c
+++ b/src/backend/storage/buffer/buf_init.c
@@ -500,8 +500,6 @@ buffer_partitions_prepare(void)
 	if (numa_nodes < 1)
 		numa_nodes = 1;
 
-	elog(WARNING, "IsUnderPostmaster %d", IsUnderPostmaster);
-
 	/*
 	 * XXX A bit weird. Do we need to worry about postmaster? Could this even
 	 * run outside postmaster? I don't think so.
diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index ff02dc8e00b..d8d602f0a4e 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -416,7 +416,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 		 * actually fine because procLatch isn't ever freed, so we just can
 		 * potentially set the wrong process' (or no process') latch.
 		 */
-		SetLatch(&ProcGlobal->allProcs[bgwprocno].procLatch);
+		SetLatch(&ProcGlobal->allProcs[bgwprocno]->procLatch);
 	}
 
 	/*
index bf987aed8d3..3e86e4ca2ae 100644

 
 /*

 

 	/*

 

 	/*

 
 
-
 /*

 			/*

 

 			/*
@@ -2858,7 +2857,7 @@ GetOldestActiveTransactionId(bool inCommitOnly, bool allDbs)
 	{
 		TransactionId xid;
 		int			pgprocno = arrayP->pgprocnos[index];
-		PGPROC	   *proc = &allProcs[pgprocno];
+		PGPROC	   *proc = allProcs[pgprocno];
 
 		/* Fetch xid just once - see GetNewTransactionId */
 		xid = UINT32_ACCESS_ONCE(other_xids[index]);
@@ -3020,7 +3019,7 @@ GetVirtualXIDsDelayingChkpt(int *nvxids, int type)
 	for (index = 0; index < arrayP->numProcs; index++)
 	{
 		int			pgprocno = arrayP->pgprocnos[index];
-		PGPROC	   *proc = &allProcs[pgprocno];
+		PGPROC	   *proc = allProcs[pgprocno];
 
@@ -3061,7 +3060,7 @@ HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids, int type)
 
 		GET_VXID_FROM_PGPROC(vxid, *proc);
@@ -3189,7 +3188,7 @@ BackendPidGetProcWithLock(int pid)

 		if (proc->pid == pid)
 		{
@@ -3232,7 +3231,7 @@ BackendXidGetPid(TransactionId xid)
 
 			result = proc->pid;
 			break;
@@ -3301,7 +3300,7 @@ GetCurrentVirtualXIDs(TransactionId limitXmin, bool excludeXmin0,
 
 		if (proc == MyProc)
@@ -3403,7 +3402,7 @@ GetConflictingVirtualXIDs(TransactionId limitXmin, Oid dbOid)
 
@@ -3468,7 +3467,7 @@ SignalVirtualTransaction(VirtualTransactionId vxid, ProcSignalReason sigmode,
 
@@ -3523,7 +3522,7 @@ MinimumActiveBackends(int min)
 
 		/*
@@ -3569,7 +3568,7 @@ CountDBBackends(Oid databaseid)
 
 		if (proc->pid == 0)
@@ -3598,7 +3597,7 @@ CountDBConnections(Oid databaseid)
 
 		if (proc->pid == 0)
@@ -3629,7 +3628,7 @@ CancelDBBackends(Oid databaseid, ProcSignalReason sigmode, bool conflictPending)
 
@@ -3670,7 +3669,7 @@ CountUserBackends(Oid roleid)
 
 		if (proc->pid == 0)
@@ -3733,7 +3732,7 @@ CountOtherDBBackends(Oid databaseId, int *nbackends, int *nprepared)
 
@@ -3799,7 +3798,7 @@ TerminateOtherDBBackends(Oid databaseId)

 
 
index e9ef0fbfe32..11259151a7d 100644
 
+#include <sched.h>
 #include <signal.h>
 #include <unistd.h>
 #include <sys/time.h>
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
@@ -90,6 +98,31 @@ static void AuxiliaryProcKill(int code, Datum arg);

+/* number of NUMA nodes (as returned by numa_num_configured_nodes) */
+static int	numa_nodes = -1;	/* number of nodes when sizing */
+static Size numa_page_size = 0; /* page used to size partitions */
+static bool numa_can_partition = false; /* can map to NUMA nodes? */
+static int	numa_procs_per_node = -1;	/* pgprocs per node */
+
+static Size get_memory_page_size(void); /* XXX duplicate with bufi_init.c */
+
+static void pgproc_partitions_prepare(void);
+static char *pgproc_partition_init(char *ptr, int num_procs,
+								   int allprocs_index, int node);
+static char *fastpath_partition_init(char *ptr, int num_procs,
+									 int allprocs_index, int node,
+									 Size fpLockBitsSize, Size fpRelIdSize);
+
+typedef struct PGProcPartition
+{
+	int			num_procs;
+	int			numa_node;
+	void	   *pgproc_ptr;
+	void	   *fastpath_ptr;
+} PGProcPartition;
+
+static PGProcPartition *partitions = NULL;
+
 /*
  * Report shared-memory space needed by PGPROC.
  */
@@ -100,11 +133,63 @@ PGProcShmemSize(void)
 	Size		TotalProcs =
 		add_size(MaxBackends, add_size(NUM_AUXILIARY_PROCS, max_prepared_xacts));

+	/*
+	 * To support NUMA partitioning, the PGPROC array will be divided into
+	 * multiple chunks - one per NUMA node, and one extra for auxiliary/2PC
+	 * entries (which are not assigned to any NUMA node).
+	 *
+	 * We can't simply map pages of a single continuous array, because the
+	 * PGPROC entries are very small and too many of them would fit on a
+	 * single page (at least with huge pages). Far more than reasonable values
+	 * of max_connections. So instead we cut the array into separate pieces
+	 * for each node.
+	 *
+	 * Each piece may need up to one memory page of padding, to make it
+	 * aligned with memory page (for NUMA), So we just add a page - it's a bit
+	 * wasteful, but should not matter much - NUMA is meant for large boxes,
+	 * so a couple pages is negligible.
+	 *
+	 * We only do this with NUMA partitioning. With the GUC disabled, or when
+	 * we find we can't do that for some reason, we just allocate the PGPROC
+	 * array as a single chunk. This is determined by the earlier call to
+	 * pgproc_partitions_prepare().
+	 *
+	 * XXX It might be more painful with very large huge pages (e.g. 1GB).
+	 */
+
+	/*
+	 * If PGPROC partitioning is enabled, and we decided it's possible, we
+	 * need to add one memory page per NUMA node (and one for auxiliary/2PC
+	 * processes) to allow proper alignment.
+	 *
+	 * XXX This is a a bit wasteful, because it might actually add pages even
+	 * when not strictly needed (if it's already aligned). And we always
+	 * assume we'll add a whole page, even if the alignment needs only less
+	 * memory.
+	 */
+	if (numa_procs_interleave && numa_can_partition)
+	{
+		Assert(numa_nodes > 0);
+		size = add_size(size, mul_size((numa_nodes + 1), numa_page_size));
+
+		/*
+		 * Also account for a small registry of partitions, a simple array of
+		 * partitions at the beginning.
+		 */
+		size = add_size(size, mul_size((numa_nodes + 1), sizeof(PGProcPartition)));
+	}
+	else
+	{
+		/* otherwise add only a tiny registry, with a single partition */
+		size = add_size(size, sizeof(PGProcPartition));
+	}
+
 	return size;
 }
 
@@ -129,6 +214,25 @@ FastPathLockShmemSize(void)

+	/*
+	 * When applying NUMA to the fast-path locks, we follow the same logic as
+	 * for PGPROC entries. See the comments in PGProcShmemSize().
+	 *
+	 * If PGPROC partitioning is enabled, and we decided it's possible, we
+	 * need to add one memory page per NUMA node (and one for auxiliary/2PC
+	 * processes) to allow proper alignment.
+	 *
+	 * XXX This is a a bit wasteful, because it might actually add pages even
+	 * when not strictly needed (if it's already aligned). And we always
+	 * assume we'll add a whole page, even if the alignment needs only less
+	 * memory.
+	 */
+	if (numa_procs_interleave && numa_can_partition)
+	{
+		Assert(numa_nodes > 0);
+		size = add_size(size, mul_size((numa_nodes + 1), numa_page_size));
+	}
+
 	return size;
 }
 
@@ -140,6 +244,9 @@ ProcGlobalShmemSize(void)
 {
 	Size		size = 0;
 
+	/* calculate partition info for pgproc entries etc */
+	pgproc_partitions_prepare();
+
 	/* ProcGlobal */
 	size = add_size(size, sizeof(PROC_HDR));
 	size = add_size(size, sizeof(slock_t));
@@ -191,7 +298,7 @@ ProcGlobalSemas(void)
 void
 InitProcGlobal(void)
 {
-	PGPROC	   *procs;
+	PGPROC	  **procs;
 	int			i,
 				j;
 	bool		found;
@@ -205,6 +312,8 @@ InitProcGlobal(void)
 	Size		requestSize;
 	char	   *ptr;
 
@@ -241,19 +350,115 @@ InitProcGlobal(void)

 
 
+	/*
+	 * If NUMA partitioning is enabled, and we decided we actually can do the
+	 * partitioning, allocate the chunks.
+	 *
+	 * Otherwise we'll allocate a single array for everything. It's not quite
+	 * what we did without NUMA, because there's an extra level of
+	 * indirection, but it's the best we can do.
+	 */
+	if (numa_procs_interleave && numa_can_partition)
+	{
+		int			node_procs;
+		int			total_procs = 0;
+
+		Assert(numa_procs_per_node > 0);
+		Assert(numa_nodes > 0);
+
+		/*
+		 * Now initialize the PGPROC partition registry with one partitoion
+		 * per NUMA node.
+		 */
+		partitions = (PGProcPartition *) ptr;
+		ptr += (numa_nodes * sizeof(PGProcPartition));
+
+		/* build PGPROC entries for NUMA nodes */
+		for (i = 0; i < numa_nodes; i++)
+		{
+			/* the last NUMA node may get fewer PGPROC entries, but meh */
+			node_procs = Min(numa_procs_per_node, MaxBackends - total_procs);
+
+			/* make sure to align the PGPROC array to memory page */
+			ptr = (char *) TYPEALIGN(numa_page_size, ptr);
+
+			/* fill in the partition info */
+			partitions[i].num_procs = node_procs;
+			partitions[i].numa_node = i;
+			partitions[i].pgproc_ptr = ptr;
+
+			ptr = pgproc_partition_init(ptr, node_procs, total_procs, i);
+
+			total_procs += node_procs;
+
+			/* don't underflow/overflow the allocation */
+			Assert((ptr > (char *) procs) && (ptr <= (char *) procs + requestSize));
+		}
+
+		Assert(total_procs == MaxBackends);
+
+		/*
+		 * Also build PGPROC entries for auxiliary procs / prepared xacts (we
+		 * however don't assign those to any NUMA node).
+		 */
+		node_procs = (NUM_AUXILIARY_PROCS + max_prepared_xacts);
+
+		/* make sure to align the PGPROC array to memory page */
+		ptr = (char *) TYPEALIGN(numa_page_size, ptr);
+
+		/* fill in the partition info */
+		partitions[numa_nodes].num_procs = node_procs;
+		partitions[numa_nodes].numa_node = -1;
+		partitions[numa_nodes].pgproc_ptr = ptr;
+
+		ptr = pgproc_partition_init(ptr, node_procs, total_procs, -1);
+
+		total_procs += node_procs;
+
+		/* don't overflow the allocation */
+		Assert((ptr > (char *) procs) && (ptr <= (char *) procs + requestSize));
+
+		Assert(total_procs = TotalProcs);
+	}
+	else
+	{
+		/*
+		 * Now initialize the PGPROC partition registry with a single
+		 * partition for all the procs.
+		 */
+		partitions = (PGProcPartition *) ptr;
+		ptr += sizeof(PGProcPartition);
+
+		/* just treat everything as a single array, with no alignment */
+		ptr = pgproc_partition_init(ptr, TotalProcs, 0, -1);
+
+		/* fill in the partition info */
+		partitions[0].num_procs = TotalProcs;
+		partitions[0].numa_node = -1;
+		partitions[0].pgproc_ptr = ptr;
+
+		/* don't overflow the allocation */
+		Assert((ptr > (char *) procs) && (ptr <= (char *) procs + requestSize));
+	}
+
 	/*
 	 * Allocate arrays mirroring PGPROC fields in a dense manner. See
 	 * PROC_HDR.
 	 *
 	 * XXX: It might make sense to increase padding for these arrays, given
 	 * how hotly they are accessed.
+	 *
+	 * XXX Would it make sense to NUMA-partition these chunks too, somehow?
+	 * But those arrays are tiny, fit into a single memory page, so would need
+	 * to be made more complex. Not sure.
 	 */
 	ProcGlobal->xids = (TransactionId *) ptr;
 	ptr = (char *) ptr + (TotalProcs * sizeof(*ProcGlobal->xids));
@@ -286,24 +491,92 @@ InitProcGlobal(void)
 	/* For asserts checking we did not overflow. */
 	fpEndPtr = fpPtr + requestSize;
 
-	for (i = 0; i < TotalProcs; i++)
+	/*
+	 * Mimic the logic we used to partition PGPROC entries.
+	 */
+
+	/*
+	 * If NUMA partitioning is enabled, and we decided we actually can do the
+	 * partitioning, allocate the chunks.
+	 *
+	 * Otherwise we'll allocate a single array for everything. It's not quite
+	 * what we did without NUMA, because there's an extra level of
+	 * indirection, but it's the best we can do.
+	 */
+	if (numa_procs_interleave && numa_can_partition)
 	{
-		PGPROC	   *proc = &procs[i];
+		int			node_procs;
+		int			total_procs = 0;
+
+		Assert(numa_procs_per_node > 0);
+
+		/* build PGPROC entries for NUMA nodes */
+		for (i = 0; i < numa_nodes; i++)
+		{
+			/* the last NUMA node may get fewer PGPROC entries, but meh */
+			node_procs = Min(numa_procs_per_node, MaxBackends - total_procs);
+
+			/* make sure to align the PGPROC array to memory page */
+			fpPtr = (char *) TYPEALIGN(mem_page_size, fpPtr);
 
-		/* Common initialization for all PGPROCs, regardless of type. */
+			/* remember this pointer too */
+			partitions[i].fastpath_ptr = fpPtr;
+			Assert(node_procs == partitions[i].num_procs);
+
+			fpPtr = fastpath_partition_init(fpPtr, node_procs, total_procs, i,
+											fpLockBitsSize, fpRelIdSize);
+
+			total_procs += node_procs;
+
+			/* don't overflow the allocation */
+			Assert(fpPtr <= fpEndPtr);
+		}
+
+		Assert(total_procs == MaxBackends);
 
 		/*
-		 * Set the fast-path lock arrays, and move the pointer. We interleave
-		 * the two arrays, to (hopefully) get some locality for each backend.
+		 * Also build PGPROC entries for auxiliary procs / prepared xacts (we
+		 * however don't assign those to any NUMA node).
 		 */
-		proc->fpLockBits = (uint64 *) fpPtr;
-		fpPtr += fpLockBitsSize;
+		node_procs = (NUM_AUXILIARY_PROCS + max_prepared_xacts);
 
-		proc->fpRelId = (Oid *) fpPtr;
-		fpPtr += fpRelIdSize;
+		/* make sure to align the PGPROC array to memory page */
+		fpPtr = (char *) TYPEALIGN(numa_page_size, fpPtr);
 
+		/* remember this pointer too */
+		partitions[numa_nodes].fastpath_ptr = fpPtr;
+		Assert(node_procs == partitions[numa_nodes].num_procs);
+
+		fpPtr = fastpath_partition_init(fpPtr, node_procs, total_procs, -1,
+										fpLockBitsSize, fpRelIdSize);
+
+		total_procs += node_procs;
+
+		/* don't overflow the allocation */
 		Assert(fpPtr <= fpEndPtr);
 
+		Assert(total_procs = TotalProcs);
+	}
+	else
+	{
+		/* remember this pointer too */
+		partitions[0].fastpath_ptr = fpPtr;
+		Assert(TotalProcs == partitions[0].num_procs);
+
+		/* just treat everything as a single array, with no alignment */
+		fpPtr = fastpath_partition_init(fpPtr, TotalProcs, 0, -1,
+										fpLockBitsSize, fpRelIdSize);
+
+		/* don't overflow the allocation */
+		Assert(fpPtr <= fpEndPtr);
+	}
+
+	for (i = 0; i < TotalProcs; i++)
+	{
+		PGPROC	   *proc = procs[i];
+
+		Assert(proc->procnumber == i);
+
 		/*
 		 * Set up per-PGPROC semaphore, latch, and fpInfoLock.  Prepared xact
 		 * dummy PGPROCs don't need these though - they're never associated
@@ -366,15 +639,12 @@ InitProcGlobal(void)
 		pg_atomic_init_u64(&(proc->waitStart), 0);
 	}

 	/* Create ProcStructLock spinlock, too */
 	ProcStructLock = (slock_t *) ShmemInitStruct("ProcStructLock spinlock",
@@ -435,7 +705,45 @@ InitProcess(void)
 
@@ -1988,7 +2296,7 @@ ProcSendSignal(ProcNumber procNumber)

 /*
@@ -2063,3 +2371,222 @@ BecomeLockGroupMember(PGPROC *leader, int pid)
 
 	return ok;
 }
+
+/* copy from buf_init.c */
+static Size
+get_memory_page_size(void)
+{
+	Size		os_page_size;
+	Size		huge_page_size;
+
+#ifdef WIN32
+	SYSTEM_INFO sysinfo;
+
+	GetSystemInfo(&sysinfo);
+	os_page_size = sysinfo.dwPageSize;
+#else
+	os_page_size = sysconf(_SC_PAGESIZE);
+#endif
+
+	/*
+	 * XXX This is a bit annoying/confusing, because we may get a different
+	 * result depending on when we call it. Before mmap() we don't know if the
+	 * huge pages get used, so we assume they will. And then if we don't get
+	 * huge pages, we'll waste memory etc.
+	 */
+
+	/* assume huge pages get used, unless HUGE_PAGES_OFF */
+	if (huge_pages_status == HUGE_PAGES_OFF)
+		huge_page_size = 0;
+	else
+		GetHugePageSize(&huge_page_size, NULL);
+
+	return Max(os_page_size, huge_page_size);
+}
+
+/*
+ * pgproc_partitions_prepare
+ *		Calculate parameters for partitioning buffers.
+ *
+ * NUMA partitioning
+ *
+ * Now build the actual PGPROC arrays, one "chunk" per NUMA node (and one
+ * extra for auxiliary processes and 2PC transactions, not associated with
+ * any particular node).
+ *
+ * First determine how many "backend" procs to allocate per NUMA node. The
+ * count may not be exactly divisible, but we mostly ignore that. The last
+ * node may get somewhat fewer PGPROC entries, but the imbalance ought to
+ * be pretty small (if MaxBackends >> numa_nodes).
+ *
+ * XXX A fairer distribution is possible, but not worth it for now.
+ */
+static void
+pgproc_partitions_prepare(void)
+{
+	/* bail out if already initialized (calculate only once) */
+	if (numa_nodes != -1)
+		return;
+
+	/* XXX only gives us the number, the nodes may not be 0, 1, 2, ... */
+	numa_nodes = numa_num_configured_nodes();
+
+	/* XXX can this happen? */
+	if (numa_nodes < 1)
+		numa_nodes = 1;
+
+	/*
+	 * XXX A bit weird. Do we need to worry about postmaster? Could this even
+	 * run outside postmaster? I don't think so.
+	 *
+	 * XXX Another issue is we may get different values than when sizing the
+	 * the memory, because at that point we didn't know if we get huge pages,
+	 * so we assumed we will. Shouldn't cause crashes, but we might allocate
+	 * shared memory and then not use some of it (because of the alignment
+	 * that we don't actually need). Not sure about better way, good for now.
+	 */
+	if (IsUnderPostmaster)
+		numa_page_size = pg_get_shmem_pagesize();
+	else
+		numa_page_size = get_memory_page_size();
+
+	numa_procs_per_node = (MaxBackends + (numa_nodes - 1)) / numa_nodes;
+
+	elog(LOG, "NUMA: pgproc backends %d num_nodes %d per_node %d",
+		 MaxBackends, numa_nodes, numa_procs_per_node);
+
+	Assert(numa_nodes * numa_procs_per_node >= MaxBackends);
+
+	/* success */
+	numa_can_partition = true;
+}
+
+static void
+pg_numa_move_to_node(char *startptr, char *endptr, int node)
+{
+	Size		mem_page_size;
+	Size		sz;
+
+	/*
+	 * Get the "actual" memory page size, not the one we used for sizing. We
+	 * might have used huge page for sizing, but only get regular pages when
+	 * allocating, so we must use the smaller pages here.
+	 *
+	 * XXX A bit weird. Do we need to worry about postmaster? Could this even
+	 * run outside postmaster? I don't think so.
+	 */
+	if (IsUnderPostmaster)
+		mem_page_size = pg_get_shmem_pagesize();
+	else
+		mem_page_size = get_memory_page_size();
+
+	Assert((int64) startptr % mem_page_size == 0);
+
+	sz = (endptr - startptr);
+	numa_tonode_memory(startptr, sz, node);
+}
+
+/*
+ * doesn't do alignment
+ */
+static char *
+pgproc_partition_init(char *ptr, int num_procs, int allprocs_index, int node)
+{
+	PGPROC	   *procs_node;
+
+	/* allocate the PGPROC chunk for this node */
+	procs_node = (PGPROC *) ptr;
+
+	/* pointer right after this array */
+	ptr = (char *) ptr + num_procs * sizeof(PGPROC);
+
+	elog(LOG, "NUMA: pgproc_init_partition procs %p endptr %p num_procs %d node %d",
+		 procs_node, ptr, num_procs, node);
+
+	/*
+	 * if node specified, move to node - do this before we start touching the
+	 * memory, to make sure it's not mapped to any node yet
+	 */
+	if (node != -1)
+		pg_numa_move_to_node((char *) procs_node, ptr, node);
+
+	/* add pointers to the PGPROC entries to allProcs */
+	for (int i = 0; i < num_procs; i++)
+	{
+		procs_node[i].numa_node = node;
+		procs_node[i].procnumber = allprocs_index;
+
+		ProcGlobal->allProcs[allprocs_index] = &procs_node[i];
+
+		allprocs_index++;
+	}
+
+	return ptr;
+}
+
+static char *
+fastpath_partition_init(char *ptr, int num_procs, int allprocs_index, int node,
+						Size fpLockBitsSize, Size fpRelIdSize)
+{
+	char	   *endptr = ptr + num_procs * (fpLockBitsSize + fpRelIdSize);
+
+	/*
+	 * if node specified, move to node - do this before we start touching the
+	 * memory, to make sure it's not mapped to any node yet
+	 */
+	if (node != -1)
+		pg_numa_move_to_node(ptr, endptr, node);
+
+	/*
+	 * Now point the PGPROC entries to the fast-path arrays, and also advance
+	 * the fpPtr.
+	 */
+	for (int i = 0; i < num_procs; i++)
+	{
+		PGPROC	   *proc = ProcGlobal->allProcs[allprocs_index];
+
+		/* cross-check we got the expected NUMA node */
+		Assert(proc->numa_node == node);
+		Assert(proc->procnumber == allprocs_index);
+
+		/*
+		 * Set the fast-path lock arrays, and move the pointer. We interleave
+		 * the two arrays, to (hopefully) get some locality for each backend.
+		 */
+		proc->fpLockBits = (uint64 *) ptr;
+		ptr += fpLockBitsSize;
+
+		proc->fpRelId = (Oid *) ptr;
+		ptr += fpRelIdSize;
+
+		Assert(ptr <= endptr);
+
+		allprocs_index++;
+	}
+
+	Assert(ptr == endptr);
+
+	return endptr;
+}
+
+int
+ProcPartitionCount(void)
+{
+	if (numa_procs_interleave && numa_can_partition)
+		return (numa_nodes + 1);
+
+	return 1;
+}
+
+void
+ProcPartitionGet(int idx, int *node, int *nprocs, void **procsptr, void **fpptr)
+{
+	PGProcPartition *part = &partitions[idx];
+
+	Assert((idx >= 0) && (idx < ProcPartitionCount()));
+
+	*nprocs = part->num_procs;
+	*procsptr = part->pgproc_ptr;
+	*fpptr = part->fastpath_ptr;
+	*node = part->numa_node;
+}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index a11bc71a386..6ee4684d1b8 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -149,6 +149,7 @@ int			MaxBackends = 0;
 bool		numa_buffers_interleave = false;
 bool		numa_localalloc = false;
 bool		numa_partition_freelist = false;
+bool		numa_procs_interleave = false;

+	{
 
 extern PGDLLIMPORT int commit_timestamp_buffers;
 extern PGDLLIMPORT int multixact_member_buffers;
diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h
index c6f5ebceefd..d2d269941fc 100644
--- a/src/include/storage/proc.h
+++ b/src/include/storage/proc.h
@@ -202,6 +202,8 @@ struct PGPROC
 								 * vacuum must not remove tuples deleted by
 								 * xid >= xmin ! */

@@ -327,6 +329,9 @@ struct PGPROC
 
 /* NOTE: "typedef struct PGPROC PGPROC" appears in storage/lock.h. */
@@ -391,7 +396,7 @@ extern PGDLLIMPORT PGPROC *MyProc;
 typedef struct PROC_HDR
 {
 	/* Array of PGPROC structures (not including dummies for prepared txns) */
-	PGPROC	   *allProcs;
+	PGPROC	  **allProcs;
 
@@ -443,8 +448,8 @@ extern PGDLLIMPORT PGPROC *PreparedXactProcs;
 
 /*
  * We set aside some extra PGPROC structures for "special worker" processes,
@@ -520,4 +525,7 @@ extern PGPROC *AuxiliaryPidGetProc(int pid);
 extern void BecomeLockGroupLeader(void);
 extern bool BecomeLockGroupMember(PGPROC *leader, int pid);
 
+extern int	ProcPartitionCount(void);
+extern void ProcPartitionGet(int idx, int *node, int *nprocs, void **procsptr, void **fpptr);
+
 #endif							/* _PROC_H_ */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index a38dd8d6242..5595cd48eee 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1877,6 +1877,7 @@ PGP_MPI
 PGP_PubKey
 PGP_S2K
 PGPing
+PGProcPartition
 PGQueryClass
 PGRUsage
 PGSemaphore
-- 
2.50.1

From 2bfd4a824b12e9a865c5ef0a8ed33e215fb1b698 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Sun, 8 Jun 2025 18:53:12 +0200
Subject: [PATCH v3 5/7] NUMA: clockweep partitioning

Similar to the frelist patch - partition the "clocksweep" algorithm to
work on the sequence of smaller partitions, one by one.

It extends the "pg_buffercache_partitions" view to include information
about the clocksweep activity.

Note: This needs some sort of "balancing" when one of the partitions is
much busier than the rest (e.g. because there's a single backend consuming
a lot of buffers from it).
---
 .../pg_buffercache--1.6--1.7.sql              |   5 +-
 contrib/pg_buffercache/pg_buffercache_pages.c |  24 +-
 src/backend/storage/buffer/bufmgr.c           | 476 ++++++++++--------
 src/backend/storage/buffer/freelist.c         | 224 +++++++--
 src/include/storage/buf_internals.h           |   4 +-
 src/include/storage/bufmgr.h                  |   5 +-
 src/tools/pgindent/typedefs.list              |   1 +
 7 files changed, 478 insertions(+), 261 deletions(-)

diff --git a/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql b/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
index 3871c261528..b7d8ea45ed7 100644
--- a/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
+++ b/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
@@ -12,7 +12,10 @@ LANGUAGE C PARALLEL SAFE;
 -- Create a view for convenient access.
 CREATE VIEW pg_buffercache_partitions AS
 	SELECT P.* FROM pg_buffercache_partitions() AS P
-	(partition integer, numa_node integer, num_buffers integer, first_buffer integer, last_buffer integer, buffers_consumed bigint, buffers_remain bigint, buffers_free bigint);
+	(partition integer,
+	 numa_node integer, num_buffers integer, first_buffer integer, last_buffer integer,
+	 buffers_consumed bigint, buffers_remain bigint, buffers_free bigint,
+	 complete_passes bigint, buffer_allocs bigint, next_victim_buffer integer);
 
 -- Don't want these to be available to public.
 REVOKE ALL ON FUNCTION pg_buffercache_partitions() FROM PUBLIC;
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 668ada8c47b..5169655ae78 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -27,7 +27,7 @@
 #define NUM_BUFFERCACHE_EVICT_ALL_ELEM 3
 
 #define NUM_BUFFERCACHE_NUMA_ELEM	3
-#define NUM_BUFFERCACHE_PARTITIONS_ELEM	8
+#define NUM_BUFFERCACHE_PARTITIONS_ELEM	11
 
 PG_MODULE_MAGIC_EXT(
 					.name = "pg_buffercache",
@@ -818,6 +818,12 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 						   INT8OID, -1, 0);
 		TupleDescInitEntry(tupledesc, (AttrNumber) 8, "buffers_free",
 						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 9, "complete_passes",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 10, "buffer_allocs",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 11, "next_victim_buffer",
+						   INT4OID, -1, 0);
 
 		funcctx->user_fctx = BlessTupleDesc(tupledesc);
 
@@ -843,6 +849,10 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 					buffers_remain,
 					buffers_free;
 
+		uint32		complete_passes,
+					buffer_allocs,
+					next_victim_buffer;
+
 		Datum		values[NUM_BUFFERCACHE_PARTITIONS_ELEM];
 		bool		nulls[NUM_BUFFERCACHE_PARTITIONS_ELEM];
 
@@ -850,7 +860,8 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 						   &first_buffer, &last_buffer);
 
 		FreelistPartitionGetInfo(i, &buffers_consumed, &buffers_remain,
-								 &buffers_free);
+								 &buffers_free, &complete_passes,
+								 &buffer_allocs, &next_victim_buffer);
 
 		values[0] = Int32GetDatum(i);
 		nulls[0] = false;
@@ -876,6 +887,15 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		values[7] = Int64GetDatum(buffers_free);
 		nulls[7] = false;
 
+		values[8] = Int32GetDatum(complete_passes);
+		nulls[8] = false;
+
+		values[9] = Int32GetDatum(buffer_allocs);
+		nulls[9] = false;
+
+		values[10] = Int32GetDatum(next_victim_buffer);
+		nulls[10] = false;
+
 		/* Build and return the tuple. */
 		tuple = heap_form_tuple((TupleDesc) funcctx->user_fctx, values, nulls);
 		result = HeapTupleGetDatum(tuple);
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 5922689fe5d..bd007c1c621 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3587,6 +3587,23 @@ BufferSync(int flags)
 	TRACE_POSTGRESQL_BUFFER_SYNC_DONE(NBuffers, num_written, num_to_scan);
 }
 
+/*
 
-	/*

-	/*

@@ -3663,223 +3649,285 @@ BgBufferSync(WritebackContext *wb_context)
 	}
 
 	/*
+	 * XXX Maybe we should randomize the order of partitions a bit, so that we
+	 * don't start from partition 0 all the time? Perhaps not entirely, but at
+	 * least pick a random starting point?

+		 * clock sweep since last time.  If first time through, assume none.
+		 * Then see if we are still ahead of the clock sweep, and if so, how
+		 * many buffers we could scan before we'd catch up with it and "lap"
+		 * it. Note: weird-looking coding of xxx_passes comparisons are to
+		 * avoid bogus behavior when the passes counts wrap around.
+				 * We're behind, so skip forward to the strategy point and
+				 * start cleaning from there.
+			 * Initializing at startup or after LRU scanning had been off.
+			 * Always start at the strategy point.
 
-	/* Update saved info for next time */
-	prev_strategy_buf_id = strategy_buf_id;
-	prev_strategy_passes = strategy_passes;
-	saved_info_valid = true;
+		/* Update saved info for next time */
+		prev_strategy_buf_id[partition] = strategy_buf_id;
+		prev_strategy_passes[partition] = strategy_passes;
+		/* FIXME has to happen after all partitions */
+		/* saved_info_valid = true; */
 
-	/*
-	 * Compute how many buffers had to be scanned for each new allocation, ie,
-	 * 1/density of reusable buffers, and track a moving average of that.
-	 *
-	 * If the strategy point didn't move, we don't update the density estimate
-	 */
-	if (strategy_delta > 0 && recent_alloc > 0)
-	{
-		scans_per_alloc = (float) strategy_delta / (float) recent_alloc;
-		smoothed_density += (scans_per_alloc - smoothed_density) /
-			smoothing_samples;
-	}
+		/*
+		 * Compute how many buffers had to be scanned for each new allocation,
+		 * ie, 1/density of reusable buffers, and track a moving average of
+		 * that.
+		 *
+		 * If the strategy point didn't move, we don't update the density
+		 * estimate
+		 */
+		if (strategy_delta > 0 && recent_alloc_partition > 0)
+		{
+			scans_per_alloc = (float) strategy_delta / (float) recent_alloc_partition;
+			smoothed_density += (scans_per_alloc - smoothed_density) /
+				smoothing_samples;
+		}
 
-	/*
-	 * Estimate how many reusable buffers there are between the current
-	 * strategy point and where we've scanned ahead to, based on the smoothed
-	 * density estimate.
-	 */
-	bufs_ahead = NBuffers - bufs_to_lap;
-	reusable_buffers_est = (float) bufs_ahead / smoothed_density;
+		/*
+		 * Estimate how many reusable buffers there are between the current
+		 * strategy point and where we've scanned ahead to, based on the
+		 * smoothed density estimate.
+		 */
+		bufs_ahead = num_buffers - bufs_to_lap;
+		reusable_buffers_est = (float) bufs_ahead / smoothed_density;
 
-	/*
-	 * Track a moving average of recent buffer allocations.  Here, rather than
-	 * a true average we want a fast-attack, slow-decline behavior: we
-	 * immediately follow any increase.
-	 */
-	if (smoothed_alloc <= (float) recent_alloc)
-		smoothed_alloc = recent_alloc;
-	else
-		smoothed_alloc += ((float) recent_alloc - smoothed_alloc) /
-			smoothing_samples;
+		/*
+		 * Track a moving average of recent buffer allocations.  Here, rather
+		 * than a true average we want a fast-attack, slow-decline behavior:
+		 * we immediately follow any increase.
+		 */
+		if (smoothed_alloc <= (float) recent_alloc_partition)
+			smoothed_alloc = recent_alloc_partition;
+		else
+			smoothed_alloc += ((float) recent_alloc_partition - smoothed_alloc) /
+				smoothing_samples;

-	/*
-	 * If recent_alloc remains at zero for many cycles, smoothed_alloc will
-	 * eventually underflow to zero, and the underflows produce annoying
-	 * kernel warnings on some platforms.  Once upcoming_alloc_est has gone to
-	 * zero, there's no point in tracking smaller and smaller values of
-	 * smoothed_alloc, so just reset it to exactly zero to avoid this
-	 * syndrome.  It will pop back up as soon as recent_alloc increases.
-	 */
-	if (upcoming_alloc_est == 0)
-		smoothed_alloc = 0;
+		/*
+		 * If recent_alloc remains at zero for many cycles, smoothed_alloc
+		 * will eventually underflow to zero, and the underflows produce
+		 * annoying kernel warnings on some platforms.  Once
+		 * upcoming_alloc_est has gone to zero, there's no point in tracking
+		 * smaller and smaller values of smoothed_alloc, so just reset it to
+		 * exactly zero to avoid this syndrome.  It will pop back up as soon
+		 * as recent_alloc increases.
+		 */
+		if (upcoming_alloc_est == 0)
+			smoothed_alloc = 0;
 
-	/*
-	 * Even in cases where there's been little or no buffer allocation
-	 * activity, we want to make a small amount of progress through the buffer
-	 * cache so that as many reusable buffers as possible are clean after an
-	 * idle period.
-	 *
-	 * (scan_whole_pool_milliseconds / BgWriterDelay) computes how many times
-	 * the BGW will be called during the scan_whole_pool time; slice the
-	 * buffer pool into that many sections.
-	 */
-	min_scan_buffers = (int) (NBuffers / (scan_whole_pool_milliseconds / BgWriterDelay));
+		/*
+		 * Even in cases where there's been little or no buffer allocation
+		 * activity, we want to make a small amount of progress through the
+		 * buffer cache so that as many reusable buffers as possible are clean
+		 * after an idle period.
+		 *
+		 * (scan_whole_pool_milliseconds / BgWriterDelay) computes how many
+		 * times the BGW will be called during the scan_whole_pool time; slice
+		 * the buffer pool into that many sections.
+		 */
+		min_scan_buffers = (int) (num_buffers / (scan_whole_pool_milliseconds / BgWriterDelay));

-	num_to_scan = bufs_to_lap;
-	num_written = 0;
-	reusable_buffers = reusable_buffers_est;
+		/*
+		 * Now write out dirty reusable buffers, working forward from the
+		 * next_to_clean point, until we have lapped the strategy scan, or
+		 * cleaned enough buffers to match our estimate of the next cycle's
+		 * allocation requirements, or hit the bgwriter_lru_maxpages limit.
+		 */

 

 #ifdef BGW_DEBUG
 
-	/*
-	 * Consider the above scan as being like a new allocation scan.
-	 * Characterize its density and update the smoothed one based on it. This
-	 * effectively halves the moving average period in cases where both the
-	 * strategy and the background writer are doing some useful scanning,
-	 * which is helpful because a long memory isn't as desirable on the
-	 * density estimates.
-	 */
-	new_strategy_delta = bufs_to_lap - num_to_scan;
-	new_recent_alloc = reusable_buffers - reusable_buffers_est;
-	if (new_strategy_delta > 0 && new_recent_alloc > 0)
-	{
-		scans_per_alloc = (float) new_strategy_delta / (float) new_recent_alloc;
-		smoothed_density += (scans_per_alloc - smoothed_density) /
-			smoothing_samples;
+		/*
+		 * Consider the above scan as being like a new allocation scan.
+		 * Characterize its density and update the smoothed one based on it.
+		 * This effectively halves the moving average period in cases where
+		 * both the strategy and the background writer are doing some useful
+		 * scanning, which is helpful because a long memory isn't as desirable
+		 * on the density estimates.
+		 */
+		new_strategy_delta = bufs_to_lap - num_to_scan;
+		new_recent_alloc = reusable_buffers - reusable_buffers_est;
+		if (new_strategy_delta > 0 && new_recent_alloc > 0)
+		{
+			scans_per_alloc = (float) new_strategy_delta / (float) new_recent_alloc;
+			smoothed_density += (scans_per_alloc - smoothed_density) /
+				smoothing_samples;
 
 #ifdef BGW_DEBUG

 /*
index c3fbd651dd5..ff02dc8e00b 100644
 
 /*
 
 	/*

 	/*
 	 * Bgworker process to be notified upon activity or -1 if none. See
@@ -99,6 +118,9 @@ typedef struct
 	int			num_partitions;
 	int			num_partitions_per_node;

@@ -138,6 +160,7 @@ static BufferDesc *GetBufferFromRing(BufferAccessStrategy strategy,
 									 uint32 *buf_state);
 static void AddBufferToRing(BufferAccessStrategy strategy,
 							BufferDesc *buf);
+static ClockSweep *ChooseClockSweep(void);
 
 /*
  * ClockSweepTick - Helper routine for StrategyGetBuffer()
@@ -149,6 +172,7 @@ static inline uint32
 ClockSweepTick(void)
 {
 	uint32		victim;
+	ClockSweep *sweep = ChooseClockSweep();
 
 	/*
 	 * Atomically move hand ahead one buffer - if there's several processes
@@ -156,14 +180,14 @@ ClockSweepTick(void)
 	 * apparent order.
 	 */
 	victim =
-		pg_atomic_fetch_add_u32(&StrategyControl->nextVictimBuffer, 1);
+		pg_atomic_fetch_add_u32(&sweep->nextVictimBuffer, 1);

 
 		/*
@@ -189,19 +213,23 @@ ClockSweepTick(void)

 
 static int
@@ -258,6 +286,28 @@ calculate_partition_index()
 	return index;
 }
 
+/*
+ * ChooseClockSweep
@@ -374,7 +424,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 
 	/*
 	 * First check, without acquiring the lock, whether there's buffers in the
@@ -445,13 +495,17 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 	/*
 	 * Nothing on the freelist, so run the "clock sweep" algorithm
 	 *
-	 * XXX Should we also make this NUMA-aware, to only access buffers from
-	 * the same NUMA node? That'd probably mean we need to make the clock
-	 * sweep NUMA-aware, perhaps by having multiple clock sweeps, each for a
-	 * subset of buffers. But that also means each process could "sweep" only
-	 * a fraction of buffers, even if the other buffers are better candidates
-	 * for eviction. Would that also mean we'd have multiple bgwriters, one
-	 * for each node, or would one bgwriter handle all of that?
+	 * XXX Note that ClockSweepTick() is NUMA-aware, i.e. it only looks at
+	 * buffers from a single partition, aligned with the NUMA node. That means
+	 * it only accesses buffers from the same NUMA node.
+	 *
+	 * XXX That also means each process "sweeps" only a fraction of buffers,
+	 * even if the other buffers are better candidates for eviction. Maybe
+	 * there should be some logic to "steal" buffers from other freelists or
+	 * other nodes?
+	 *
+	 * XXX Would that also mean we'd have multiple bgwriters, one for each
+	 * node, or would one bgwriter handle all of that?
 	 */
 	trycounter = NBuffers;
 	for (;;)
@@ -533,6 +587,41 @@ StrategyFreeBuffer(BufferDesc *buf)
 	SpinLockRelease(&freelist->freelist_lock);
 }
 
+/*
@@ -540,37 +629,44 @@ StrategyFreeBuffer(BufferDesc *buf)

 	if (complete_passes)
 	{
 
 		/*
 
-	if (num_buf_alloc)
-	{
 
 /*
@@ -658,6 +754,10 @@ StrategyShmemSize(void)
 	size = add_size(size, MAXALIGN(mul_size(sizeof(BufferStrategyFreelist),
 											num_partitions)));

@@ -676,6 +776,7 @@ StrategyInitialize(bool init)
 	int			num_nodes;
 	int			num_partitions;
 	int			num_partitions_per_node;
+	char	   *ptr;
 
 	/* */
 	BufferPartitionParams(&num_partitions, &num_nodes);
@@ -703,7 +804,8 @@ StrategyInitialize(bool init)
 	StrategyControl = (BufferStrategyControl *)
 		ShmemInitStruct("Buffer Strategy Status",
 						MAXALIGN(offsetof(BufferStrategyControl, freelists)) +
-						MAXALIGN(sizeof(BufferStrategyFreelist) * num_partitions),
+						MAXALIGN(sizeof(BufferStrategyFreelist) * num_partitions) +
+						MAXALIGN(sizeof(ClockSweep) * num_partitions),
 						&found);
 
 	if (!found)
@@ -718,12 +820,41 @@ StrategyInitialize(bool init)

+			int			node,
+						num_buffers,
+						first_buffer,
+						last_buffer;
 
-		/* Clear statistics */
-		StrategyControl->completePasses = 0;
-		pg_atomic_init_u32(&StrategyControl->numBufferAllocs, 0);
+			/* get info about the buffer partition */
+			BufferPartitionGet(i, &node, &num_buffers,
+							   &first_buffer, &last_buffer);
+
+			/*
+			 * FIXME This may not quite right, because if NBuffers is not a
+			 * perfect multiple of numBuffers, the last partition will have
+			 * numBuffers set too high. buf_init handles this by tracking the
+			 * remaining number of buffers, and not overflowing.
+			 */
+			StrategyControl->sweeps[i].numBuffers = num_buffers;
+			StrategyControl->sweeps[i].firstBuffer = first_buffer;
+
+			/* Clear statistics */
+			StrategyControl->sweeps[i].completePasses = 0;
+			pg_atomic_init_u32(&StrategyControl->sweeps[i].numBufferAllocs, 0);
+		}
 
 		/* No pending notification */
 		StrategyControl->bgwprocno = -1;
@@ -771,7 +902,6 @@ StrategyInitialize(bool init)
 				buf->freeNext = freelist->firstFreeBuffer;
 				freelist->firstFreeBuffer = i;
 			}
-
 		}
 	}
 	else
@@ -1111,9 +1241,11 @@ StrategyRejectBuffer(BufferAccessStrategy strategy, BufferDesc *buf, bool from_r
 }
 
 void
-FreelistPartitionGetInfo(int idx, uint64 *consumed, uint64 *remain, uint64 *actually_free)
+FreelistPartitionGetInfo(int idx, uint64 *consumed, uint64 *remain, uint64 *actually_free,
+						 uint32 *complete_passes, uint32 *buffer_allocs, uint32 *next_victim_buffer)
 {
 	BufferStrategyFreelist *freelist;
+	ClockSweep *sweep;
 	int			cur;
 
 	/* stats */
@@ -1123,6 +1255,7 @@ FreelistPartitionGetInfo(int idx, uint64 *consumed, uint64 *remain, uint64 *actu
 	Assert((idx >= 0) && (idx < StrategyControl->num_partitions));
 
 	freelist = &StrategyControl->freelists[idx];
+	sweep = &StrategyControl->sweeps[idx];
 
 	/* stat */
 	SpinLockAcquire(&freelist->freelist_lock);
@@ -1152,4 +1285,11 @@ FreelistPartitionGetInfo(int idx, uint64 *consumed, uint64 *remain, uint64 *actu
 
 	*remain = cnt_remain;
 	*actually_free = cnt_free;
+
+	/* get the clocksweep stats too */
+	*complete_passes = sweep->completePasses;
+	*buffer_allocs = pg_atomic_read_u32(&sweep->numBufferAllocs);
+	*next_victim_buffer = pg_atomic_read_u32(&sweep->nextVictimBuffer);
+
+	*next_victim_buffer = sweep->firstBuffer + (*next_victim_buffer % sweep->numBuffers);
 }
diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h
index 9dfbecb9fe4..907b160b4f7 100644
--- a/src/include/storage/buf_internals.h
+++ b/src/include/storage/buf_internals.h
@@ -449,7 +449,9 @@ extern void StrategyFreeBuffer(BufferDesc *buf);
 extern bool StrategyRejectBuffer(BufferAccessStrategy strategy,
 								 BufferDesc *buf, bool from_ring);

 extern Size StrategyShmemSize(void);
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index df127274190..53855d4be23 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -349,7 +349,10 @@ extern int	GetAccessStrategyPinLimit(BufferAccessStrategy strategy);
 extern void FreeAccessStrategy(BufferAccessStrategy strategy);
 extern void FreelistPartitionGetInfo(int idx,
 									 uint64 *consumed, uint64 *remain,
-									 uint64 *actually_free);
+									 uint64 *actually_free,
+									 uint32 *complete_passes,
+									 uint32 *buffer_allocs,
+									 uint32 *next_victim_buffer);
 
 /* inline functions */
 
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index c695cfa76e8..a38dd8d6242 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -428,6 +428,7 @@ ClientCertName
 ClientConnectionInfo
 ClientData
 ClientSocket
+ClockSweep
 ClonePtrType
 ClosePortalStmt
 ClosePtrType
-- 
2.50.1

From 06a43d54498ab5049c12a458cfbd4fe3b3b168c2 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Thu, 22 May 2025 18:38:41 +0200
Subject: [PATCH v3 4/7] NUMA: partition buffer freelist

This uses the buffer partitioning scheme introduced by the earlier
patch, i.e. the partitions will "align" with NUMA nodes, etc.

It also extends the "pg_buffercache_partitions" view, to include
information about each freelist (number of consumedd buffers, ...).

When allocating a buffer, it's taken from the correct freelist (same
NUMA node).

Note: This is (probably) more important than partitioning ProcArray.
---
 .../pg_buffercache--1.6--1.7.sql              |   2 +-
 contrib/pg_buffercache/pg_buffercache_pages.c |  24 +-
 src/backend/storage/buffer/freelist.c         | 360 ++++++++++++++++--
 src/backend/utils/init/globals.c              |   1 +
 src/backend/utils/misc/guc_tables.c           |  10 +
 src/include/miscadmin.h                       |   1 +
 src/include/storage/bufmgr.h                  |   4 +-
 7 files changed, 372 insertions(+), 30 deletions(-)

diff --git a/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql b/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
index bd97246f6ab..3871c261528 100644
--- a/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
+++ b/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
@@ -12,7 +12,7 @@ LANGUAGE C PARALLEL SAFE;
 -- Create a view for convenient access.
 CREATE VIEW pg_buffercache_partitions AS
 	SELECT P.* FROM pg_buffercache_partitions() AS P
-	(partition integer, numa_node integer, num_buffers integer, first_buffer integer, last_buffer integer);
+	(partition integer, numa_node integer, num_buffers integer, first_buffer integer, last_buffer integer, buffers_consumed bigint, buffers_remain bigint, buffers_free bigint);
 
 -- Don't want these to be available to public.
 REVOKE ALL ON FUNCTION pg_buffercache_partitions() FROM PUBLIC;
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 8baa7c7b543..668ada8c47b 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -27,7 +27,7 @@
 #define NUM_BUFFERCACHE_EVICT_ALL_ELEM 3
 
 #define NUM_BUFFERCACHE_NUMA_ELEM	3
-#define NUM_BUFFERCACHE_PARTITIONS_ELEM	5
+#define NUM_BUFFERCACHE_PARTITIONS_ELEM	8
 
 PG_MODULE_MAGIC_EXT(
 					.name = "pg_buffercache",
@@ -812,6 +812,12 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 						   INT4OID, -1, 0);
 		TupleDescInitEntry(tupledesc, (AttrNumber) 5, "last_buffer",
 						   INT4OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 6, "buffers_consumed",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 7, "buffers_remain",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 8, "buffers_free",
+						   INT8OID, -1, 0);
 
 		funcctx->user_fctx = BlessTupleDesc(tupledesc);
 
@@ -833,12 +839,19 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 					first_buffer,
 					last_buffer;
 
+		uint64		buffers_consumed,
+					buffers_remain,
+					buffers_free;
+
 		Datum		values[NUM_BUFFERCACHE_PARTITIONS_ELEM];
 		bool		nulls[NUM_BUFFERCACHE_PARTITIONS_ELEM];
 
 		BufferPartitionGet(i, &numa_node, &num_buffers,
 						   &first_buffer, &last_buffer);
 
+		FreelistPartitionGetInfo(i, &buffers_consumed, &buffers_remain,
+								 &buffers_free);
+
 		values[0] = Int32GetDatum(i);
 		nulls[0] = false;
 
@@ -854,6 +867,15 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		values[4] = Int32GetDatum(last_buffer);
 		nulls[4] = false;
 
+		values[5] = Int64GetDatum(buffers_consumed);
+		nulls[5] = false;
+
+		values[6] = Int64GetDatum(buffers_remain);
+		nulls[6] = false;
+
+		values[7] = Int64GetDatum(buffers_free);
+		nulls[7] = false;
+
 		/* Build and return the tuple. */
 		tuple = heap_form_tuple((TupleDesc) funcctx->user_fctx, values, nulls);
 		result = HeapTupleGetDatum(tuple);
diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index e046526c149..c3fbd651dd5 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -15,14 +15,52 @@
  */
 #include "postgres.h"
 
+#include <sched.h>

+/*
 
 /*
 
-	int			firstFreeBuffer;	/* Head of list of unused buffers */
-
 	/*
 	 * Statistics.  These counters should be wide enough that they can't
 	 * overflow during a single bgwriter cycle.
@@ -51,8 +87,19 @@ typedef struct
 	/*
 	 * Bgworker process to be notified upon activity or -1 if none. See
 	 * StrategyNotifyBgWriter.
+	 *
+	 * XXX Not sure why this needs to be aligned like this. Need to ask
+	 * Andres. Also, shouldn't the alignment be specified after, like for
+	 * "consumed"?
 	 */
-	int			bgwprocno;
+	int			__attribute__((aligned(64))) bgwprocno;
+
+	/* info about freelist partitioning */
+	int			num_nodes;		/* effectively number of NUMA nodes */
+	int			num_partitions;
+	int			num_partitions_per_node;
+
+	BufferStrategyFreelist freelists[FLEXIBLE_ARRAY_MEMBER];
 } BufferStrategyControl;
 
 /* Pointers to shared state */
@@ -157,6 +204,88 @@ ClockSweepTick(void)
 	return victim;
 }
 
+static int
+calculate_partition_index()
+{
+	int			rc;
+	unsigned	cpu;
+	unsigned	node;
+	int			index;
+
+	Assert(StrategyControl->num_partitions ==
+		   (StrategyControl->num_nodes * StrategyControl->num_partitions_per_node));
+
+	/*
+	 * freelist is partitioned, so determine the CPU/NUMA node, and pick a
+	 * list based on that.
+	 */
+	rc = getcpu(&cpu, &node);
+	if (rc != 0)
+		elog(ERROR, "getcpu failed: %m");
+
+	/*
+	 * XXX We should't get nodes that we haven't considered while building the
+	 * partitions. Maybe if we allow this (e.g. due to support adjusting the
+	 * NUMA stuff at runtime), we should just do our best to minimize the
+	 * conflicts somehow. But it'll make the mapping harder, so for now we
+	 * ignore it.
+	 */
+	if (node > StrategyControl->num_nodes)
+		elog(ERROR, "node out of range: %d > %u", cpu, StrategyControl->num_nodes);
+
+	/*
+	 * Find the partition. If we have a single partition per node, we can
+	 * calculate the index directly from node. Otherwise we need to do two
+	 * steps, using node and then cpu.
+	 */
+	if (StrategyControl->num_partitions_per_node == 1)
+	{
+		index = (node % StrategyControl->num_partitions);
+	}
+	else
+	{
+		int			index_group,
+					index_part;
+
+		/* two steps - calculate group from node, partition from cpu */
+		index_group = (node % StrategyControl->num_nodes);
+		index_part = (cpu % StrategyControl->num_partitions_per_node);
+
+		index = (index_group * StrategyControl->num_partitions_per_node)
+			+ index_part;
+	}
+
+	return index;
+}
+
+/*
+ * ChooseFreeList
+ *		Pick the buffer freelist to use, depending on the CPU and NUMA node.
+ *
+ * Without partitioned freelists (numa_partition_freelist=false), there's only
+ * a single freelist, so use that.
+ *
+ * With partitioned freelists, we have multiple ways how to pick the freelist
+ * for the backend:
+ *
+ * - one freelist per CPU, use the freelist for CPU the task executes on
+ *
+ * - one freelist per NUMA node, use the freelist for node task executes on
+ *
+ * - use fixed number of freelists, map processes to lists based on PID
+ *
+ * There may be some other strategies, not sure. The important thing is this
+ * needs to be refrecled during initialization, i.e. we need to create the
+ * right number of lists.
+ */
+static BufferStrategyFreelist *
+ChooseFreeList(void)
+{
+	int			index = calculate_partition_index();
+
+	return &StrategyControl->freelists[index];
+}
+
 /*
  * have_free_buffer -- a lockless check to see if there is a free buffer in
  *					   buffer pool.
@@ -168,10 +297,13 @@ ClockSweepTick(void)
 bool
 have_free_buffer(void)
 {
-	if (StrategyControl->firstFreeBuffer >= 0)
-		return true;
-	else
-		return false;
+	for (int i = 0; i < StrategyControl->num_partitions; i++)
+	{
+		if (StrategyControl->freelists[i].firstFreeBuffer >= 0)
+			return true;
+	}
+
+	return false;
 }
 
 /*
@@ -193,6 +325,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 	int			bgwprocno;
 	int			trycounter;
 	uint32		local_buf_state;	/* to avoid repeated (de-)referencing */
+	BufferStrategyFreelist *freelist;
 
 	*from_ring = false;
 
@@ -259,31 +392,35 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r

 

 			/*
 			 * If the buffer is pinned or has a nonzero usage_count, we cannot
@@ -305,7 +442,17 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 		}
 	}
 
-	/* Nothing on the freelist, so run the "clock sweep" algorithm */
+	/*
+	 * Nothing on the freelist, so run the "clock sweep" algorithm
+	 *
+	 * XXX Should we also make this NUMA-aware, to only access buffers from
+	 * the same NUMA node? That'd probably mean we need to make the clock
+	 * sweep NUMA-aware, perhaps by having multiple clock sweeps, each for a
+	 * subset of buffers. But that also means each process could "sweep" only
+	 * a fraction of buffers, even if the other buffers are better candidates
+	 * for eviction. Would that also mean we'd have multiple bgwriters, one
+	 * for each node, or would one bgwriter handle all of that?
+	 */
 	trycounter = NBuffers;
 	for (;;)
 	{
@@ -356,7 +503,22 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 void
 StrategyFreeBuffer(BufferDesc *buf)
 {
-	SpinLockAcquire(&StrategyControl->buffer_strategy_lock);
+	BufferStrategyFreelist *freelist;
+
+	/*
+	 * We don't want to call ChooseFreeList() again, because we might get a
+	 * completely different freelist - either a different partition in the
+	 * same group, or even a different group if the NUMA node changed. But we
+	 * can calculate the proper freelist from the buffer id.
+	 */
+	int			index = (BufferGetNode(buf->buf_id) * StrategyControl->num_partitions_per_node)
+		+ (buf->buf_id % StrategyControl->num_partitions_per_node);
+
+	Assert((index >= 0) && (index < StrategyControl->num_partitions));
+
+	freelist = &StrategyControl->freelists[index];
+
+	SpinLockAcquire(&freelist->freelist_lock);
 
 	/*
@@ -364,11 +526,11 @@ StrategyFreeBuffer(BufferDesc *buf)

 /*
@@ -432,6 +594,42 @@ StrategyNotifyBgWriter(int bgwprocno)
 	SpinLockRelease(&StrategyControl->buffer_strategy_lock);
 }
 
+/* prints some debug info / stats about freelists at shutdown */
+static void
+freelist_before_shmem_exit(int code, Datum arg)
+{
+	for (int p = 0; p < StrategyControl->num_partitions; p++)
+	{
+		BufferStrategyFreelist *freelist = &StrategyControl->freelists[p];
+		uint64		remain = 0;
+		uint64		actually_free = 0;
+		int			cur = freelist->firstFreeBuffer;
+
+		while (cur >= 0)
+		{
+			uint32		local_buf_state;
+			BufferDesc *buf;
+
+			buf = GetBufferDescriptor(cur);
+
+			remain++;
+
+			local_buf_state = LockBufHdr(buf);
+
+			if (!(local_buf_state & BM_TAG_VALID))
+				actually_free++;
+
+			UnlockBufHdr(buf, local_buf_state);
+
+			cur = buf->freeNext;
+		}
+		elog(LOG, "NUMA: freelist partition %d, firstF: %d: consumed: %lu, remain: %lu, actually free: %lu",
+			 p,
+			 freelist->firstFreeBuffer,
+			 freelist->consumed,
+			 remain, actually_free);
+	}
+}
 
 /*
  * StrategyShmemSize
@@ -445,12 +643,20 @@ Size
 StrategyShmemSize(void)
 {
 	Size		size = 0;
+	int			num_partitions;
+	int			num_nodes;
+
+	BufferPartitionParams(&num_partitions, &num_nodes);

 
 	return size;
 }
@@ -467,6 +673,18 @@ StrategyInitialize(bool init)
 {
 	bool		found;
 
+	int			num_nodes;
+	int			num_partitions;
+	int			num_partitions_per_node;
+
+	/* */
+	BufferPartitionParams(&num_partitions, &num_nodes);
+
+	/* always a multiple of NUMA nodes */
+	Assert(num_partitions % num_nodes == 0);
+
+	num_partitions_per_node = (num_partitions / num_nodes);
+
 	/*
 	 * Initialize the shared buffer lookup hashtable.
 	 *
@@ -484,7 +702,8 @@ StrategyInitialize(bool init)
 	 */
 	StrategyControl = (BufferStrategyControl *)
 		ShmemInitStruct("Buffer Strategy Status",
-						sizeof(BufferStrategyControl),
+						MAXALIGN(offsetof(BufferStrategyControl, freelists)) +
+						MAXALIGN(sizeof(BufferStrategyFreelist) * num_partitions),
 						&found);
 
 	if (!found)
@@ -494,13 +713,10 @@ StrategyInitialize(bool init)
 		 */
 		Assert(init);

-		/*
 
 		/* Initialize the clock sweep pointer */
 		pg_atomic_init_u32(&StrategyControl->nextVictimBuffer, 0);
@@ -511,6 +727,52 @@ StrategyInitialize(bool init)
 
 		/* No pending notification */
 		StrategyControl->bgwprocno = -1;
+
+		/* initialize the partitioned clocksweep */
+		StrategyControl->num_partitions = num_partitions;
+		StrategyControl->num_nodes = num_nodes;
+		StrategyControl->num_partitions_per_node = num_partitions_per_node;
+
+		/*
+		 * Rebuild the freelist - right now all buffers are in one huge list,
+		 * we want to rework that into multiple lists. Start by initializing
+		 * the strategy to have empty lists.
+		 */
+		for (int nfreelist = 0; nfreelist < num_partitions; nfreelist++)
+		{
+			int			node,
+						num_buffers,
+						first_buffer,
+						last_buffer;
+
+			BufferStrategyFreelist *freelist;
+
+			freelist = &StrategyControl->freelists[nfreelist];
+
+			freelist->firstFreeBuffer = FREENEXT_END_OF_LIST;
+
+			SpinLockInit(&freelist->freelist_lock);
+
+			/* get info about the buffer partition */
+			BufferPartitionGet(nfreelist, &node,
+							   &num_buffers, &first_buffer, &last_buffer);
+
+			/*
+			 * Walk through buffers for each partition, add them to the list.
+			 * Walk from the end, because we're adding the buffers to the
+			 * beginning.
+			 */
+
+			for (int i = last_buffer; i >= first_buffer; i--)
+			{
+				BufferDesc *buf = GetBufferDescriptor(i);
+
+				/* add to the freelist */
+				buf->freeNext = freelist->firstFreeBuffer;
+				freelist->firstFreeBuffer = i;
+			}
+
+		}
 	}
 	else
 		Assert(!init);
@@ -847,3 +1109,47 @@ StrategyRejectBuffer(BufferAccessStrategy strategy, BufferDesc *buf, bool from_r
 
 	return true;
 }
+
+void
+FreelistPartitionGetInfo(int idx, uint64 *consumed, uint64 *remain, uint64 *actually_free)
+{
+	BufferStrategyFreelist *freelist;
+	int			cur;
+
+	/* stats */
+	uint64		cnt_remain = 0;
+	uint64		cnt_free = 0;
+
+	Assert((idx >= 0) && (idx < StrategyControl->num_partitions));
+
+	freelist = &StrategyControl->freelists[idx];
+
+	/* stat */
+	SpinLockAcquire(&freelist->freelist_lock);
+
+	*consumed = freelist->consumed;
+
+	cur = freelist->firstFreeBuffer;
+	while (cur >= 0)
+	{
+		uint32		local_buf_state;
+		BufferDesc *buf;
+
+		buf = GetBufferDescriptor(cur);
+
+		cnt_remain++;
+
+		local_buf_state = LockBufHdr(buf);
+
+		if (!(local_buf_state & BM_TAG_VALID))
+			cnt_free++;
+
+		UnlockBufHdr(buf, local_buf_state);
+
+		cur = buf->freeNext;
+	}
+	SpinLockRelease(&freelist->freelist_lock);
+
+	*remain = cnt_remain;
+	*actually_free = cnt_free;
+}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index f5359db3656..a11bc71a386 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -148,6 +148,7 @@ int			MaxBackends = 0;
 /* NUMA stuff */
 bool		numa_buffers_interleave = false;
 bool		numa_localalloc = false;
+bool		numa_partition_freelist = false;

+	{

 extern PGDLLIMPORT int commit_timestamp_buffers;
 extern PGDLLIMPORT int multixact_member_buffers;
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index deaf4f19fa4..df127274190 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -347,7 +347,9 @@ extern int	GetAccessStrategyBufferCount(BufferAccessStrategy strategy);
 extern int	GetAccessStrategyPinLimit(BufferAccessStrategy strategy);
 
 extern void FreeAccessStrategy(BufferAccessStrategy strategy);
-
+extern void FreelistPartitionGetInfo(int idx,
+									 uint64 *consumed, uint64 *remain,
+									 uint64 *actually_free);
 
 /* inline functions */
 
-- 
2.50.1

From 3c7dbbec4ee3957c92bf647605768beb5473f66b Mon Sep 17 00:00:00 2001
From: Andres Freund <[email-masked]>
Date: Mon, 14 Oct 2024 14:10:13 -0400
Subject: [PATCH v3 3/7] freelist: Don't track tail of a freelist

 
 
 	/*

2.50.1

From d08a94b0f54a4c4986d351f98269905bb511624c Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Thu, 22 May 2025 18:27:06 +0200
Subject: [PATCH v3 2/7] NUMA: localalloc

XXX This patch may not be necessary, as we now locate memory to nodes
using explicit numa_tonode_memory() calls, and not by interleaving. But
it's useful for experiments during development, so I'm keeping it.
---
 src/backend/utils/init/globals.c    |  1 +
 src/backend/utils/init/miscinit.c   | 17 +++++++++++++++++
 src/backend/utils/misc/guc_tables.c | 10 ++++++++++
 src/include/miscadmin.h             |  1 +
 4 files changed, 29 insertions(+)

 /* NUMA stuff */
 
index 43b4dbccc3d..079974944e9 100644
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#endif
+
@@ -164,6 +168,19 @@ InitPostmasterChild(void)
 
 /*
 
+	{

2.50.1

From f0eb1af6fdcfd7daae26952ddc223952333f6af2 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Mon, 28 Jul 2025 14:01:37 +0200
Subject: [PATCH v3 1/7] NUMA: interleaving buffers

The effect is similar to

     numactl --interleave=all

Secondly, it considers the page and block size, and makes sure to always
put the whole buffer on a single NUMA node (even if it happens to use
multiple memory pages), and to keep the buffer and it's descriptor on
the same NUMA node. The seriousness/likelihood of these issues depends
on the memory page size (regular vs. huge pages).

The mapping of memory to NUMA nodes happens in larger chunks. This is
required to handle buffer descriptors (which are smaller than buffers),
and so many more fit onto a single memory page.

The number of buffer descriptors per memory page determines the smallest
number of buffers that can be placed on a NUMA node. With 2MB huge pages
this is 256MB, with 4KB pages this is 512KB). Nodes get a multiple of
this, and we try to keep the nodes balanced - the last node can get less
memory, though.

The "buffer partitions" may not be 1:1 with NUMA nodes. There's a
minimal number of partitions (default: 4) that will be created even with
fewer NUMA nodes, or no NUMA at all. Each node gets the same number of
partitions, to keep things simple. For example, with 2 nodes there'll be
4 partitions, with each node getting 2 of them. With 3 nodes there'll be
6 partitions (again, 2 per node).

The patch introduces a simple "registry" of buffer partitions, keeping
track of the first/last buffer, NUMA node, etc. This serves as a source
of truth, both for this patch and for later patches building on this
same buffer partition structure.

With the feature disabled (GUC set to 'off'), there'll be a single
partition for all the buffers (and it won't be mapped to a NUMA node).

Notes:

* The feature is enabled by numa_buffers_interleave GUC (default: false)

* We could partition ProcArray, with one partition per NUMA node, and
  then at connection time pick a node from the same node. The process
  could migrate to some other node later, especially for long-lived
  connections, but there's no perfect solution, Maybe we could set
  affinity to cores from the same node, or something like that?
---
 contrib/pg_buffercache/Makefile               |   2 +-
 .../pg_buffercache--1.6--1.7.sql              |  22 +
 contrib/pg_buffercache/pg_buffercache.control |   2 +-
 contrib/pg_buffercache/pg_buffercache_pages.c |  92 +++
 src/backend/storage/buffer/buf_init.c         | 626 +++++++++++++++++-
 src/backend/utils/init/globals.c              |   3 +
 src/backend/utils/misc/guc_tables.c           |  10 +
 src/include/miscadmin.h                       |   2 +
 src/include/storage/buf_internals.h           |   6 +
 src/include/storage/bufmgr.h                  |  15 +
 src/tools/pgindent/typedefs.list              |   2 +
 11 files changed, 771 insertions(+), 11 deletions(-)
 create mode 100644 contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql

diff --git a/contrib/pg_buffercache/Makefile b/contrib/pg_buffercache/Makefile
index 5f748543e2e..0e618f66aec 100644
--- a/contrib/pg_buffercache/Makefile
+++ b/contrib/pg_buffercache/Makefile
@@ -9,7 +9,7 @@ EXTENSION = pg_buffercache
 DATA = pg_buffercache--1.2.sql pg_buffercache--1.2--1.3.sql \
 	pg_buffercache--1.1--1.2.sql pg_buffercache--1.0--1.1.sql \
 	pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql \
-	pg_buffercache--1.5--1.6.sql
+	pg_buffercache--1.5--1.6.sql pg_buffercache--1.6--1.7.sql
 PGFILEDESC = "pg_buffercache - monitoring of shared buffer cache in real-time"
 
 REGRESS = pg_buffercache pg_buffercache_numa
diff --git a/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql b/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
new file mode 100644
index 00000000000..bd97246f6ab
--- /dev/null
+++ b/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
@@ -0,0 +1,22 @@
+/* contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "ALTER EXTENSION pg_buffercache UPDATE TO '1.7'" to load this file. \quit
+
+-- Register the new functions.
+CREATE OR REPLACE FUNCTION pg_buffercache_partitions()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_partitions'
+LANGUAGE C PARALLEL SAFE;
+
+-- Create a view for convenient access.
+CREATE VIEW pg_buffercache_partitions AS
+	SELECT P.* FROM pg_buffercache_partitions() AS P
+	(partition integer, numa_node integer, num_buffers integer, first_buffer integer, last_buffer integer);
+
+-- Don't want these to be available to public.
+REVOKE ALL ON FUNCTION pg_buffercache_partitions() FROM PUBLIC;
+REVOKE ALL ON pg_buffercache_partitions FROM PUBLIC;
+
+GRANT EXECUTE ON FUNCTION pg_buffercache_partitions() TO pg_monitor;
+GRANT SELECT ON pg_buffercache_partitions TO pg_monitor;
diff --git a/contrib/pg_buffercache/pg_buffercache.control b/contrib/pg_buffercache/pg_buffercache.control
index b030ba3a6fa..11499550945 100644
--- a/contrib/pg_buffercache/pg_buffercache.control
+++ b/contrib/pg_buffercache/pg_buffercache.control
@@ -1,5 +1,5 @@
 # pg_buffercache extension
 comment = 'examine the shared buffer cache'
-default_version = '1.6'
+default_version = '1.7'
 module_pathname = '$libdir/pg_buffercache'
 relocatable = true
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index ae0291e6e96..8baa7c7b543 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -27,6 +27,7 @@
 #define NUM_BUFFERCACHE_EVICT_ALL_ELEM 3
 
 #define NUM_BUFFERCACHE_NUMA_ELEM	3
+#define NUM_BUFFERCACHE_PARTITIONS_ELEM	5
 
 PG_MODULE_MAGIC_EXT(
 					.name = "pg_buffercache",
@@ -100,6 +101,7 @@ PG_FUNCTION_INFO_V1(pg_buffercache_usage_counts);
 PG_FUNCTION_INFO_V1(pg_buffercache_evict);
 PG_FUNCTION_INFO_V1(pg_buffercache_evict_relation);
 PG_FUNCTION_INFO_V1(pg_buffercache_evict_all);
+PG_FUNCTION_INFO_V1(pg_buffercache_partitions);

 /* Only need to touch memory once per backend process lifetime */
@@ -771,3 +773,93 @@ pg_buffercache_evict_all(PG_FUNCTION_ARGS)
 
 	PG_RETURN_DATUM(result);
 }
+
+/*
+ * Inquire about partitioning of buffers between NUMA nodes.
+ */
+Datum
+pg_buffercache_partitions(PG_FUNCTION_ARGS)
+{
+	FuncCallContext *funcctx;
+	MemoryContext oldcontext;
+	TupleDesc	tupledesc;
+	TupleDesc	expected_tupledesc;
+	HeapTuple	tuple;
+	Datum		result;
+
+	if (SRF_IS_FIRSTCALL())
+	{
+		funcctx = SRF_FIRSTCALL_INIT();
+
+		/* Switch context when allocating stuff to be used in later calls */
+		oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
+
+		if (get_call_result_type(fcinfo, NULL, &expected_tupledesc) != TYPEFUNC_COMPOSITE)
+			elog(ERROR, "return type must be a row type");
+
+		if (expected_tupledesc->natts != NUM_BUFFERCACHE_PARTITIONS_ELEM)
+			elog(ERROR, "incorrect number of output arguments");
+
+		/* Construct a tuple descriptor for the result rows. */
+		tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 1, "partition",
+						   INT4OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 2, "numa_node",
+						   INT4OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 3, "num_buffers",
+						   INT4OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 4, "first_buffer",
+						   INT4OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 5, "last_buffer",
+						   INT4OID, -1, 0);
+
+		funcctx->user_fctx = BlessTupleDesc(tupledesc);
+
+		/* Return to original context when allocating transient memory */
+		MemoryContextSwitchTo(oldcontext);
+
+		/* Set max calls and remember the user function context. */
+		funcctx->max_calls = BufferPartitionCount();
+	}
+
+	funcctx = SRF_PERCALL_SETUP();
+
+	if (funcctx->call_cntr < funcctx->max_calls)
+	{
+		uint32		i = funcctx->call_cntr;
+
+		int			numa_node,
+					num_buffers,
+					first_buffer,
+					last_buffer;
+
+		Datum		values[NUM_BUFFERCACHE_PARTITIONS_ELEM];
+		bool		nulls[NUM_BUFFERCACHE_PARTITIONS_ELEM];
+
+		BufferPartitionGet(i, &numa_node, &num_buffers,
+						   &first_buffer, &last_buffer);
+
+		values[0] = Int32GetDatum(i);
+		nulls[0] = false;
+
+		values[1] = Int32GetDatum(numa_node);
+		nulls[1] = false;
+
+		values[2] = Int32GetDatum(num_buffers);
+		nulls[2] = false;
+
+		values[3] = Int32GetDatum(first_buffer);
+		nulls[3] = false;
+
+		values[4] = Int32GetDatum(last_buffer);
+		nulls[4] = false;
+
+		/* Build and return the tuple. */
+		tuple = heap_form_tuple((TupleDesc) funcctx->user_fctx, values, nulls);
+		result = HeapTupleGetDatum(tuple);
+
+		SRF_RETURN_NEXT(funcctx, result);
+	}
+	else
+		SRF_RETURN_DONE(funcctx);
+}
diff --git a/src/backend/storage/buffer/buf_init.c b/src/backend/storage/buffer/buf_init.c
index ed1dc488a42..5b65a855b29 100644
--- a/src/backend/storage/buffer/buf_init.c
+++ b/src/backend/storage/buffer/buf_init.c
@@ -14,9 +14,17 @@
  */
 #include "postgres.h"
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
 
 BufferDescPadded *BufferDescriptors;
 char	   *BufferBlocks;
@@ -24,6 +32,19 @@ ConditionVariableMinimallyPadded *BufferIOCVArray;
 WritebackContext BackendWritebackContext;
 CkptSortItem *CkptBufferIds;
 
+BufferPartitions *BufferPartitionsArray;
+
+static Size get_memory_page_size(void);
+static void buffer_partitions_prepare(void);
+static void buffer_partitions_init(void);
+
+/* number of NUMA nodes (as returned by numa_num_configured_nodes) */
+static int	numa_nodes = -1;	/* number of nodes when sizing */
+static Size numa_page_size = 0; /* page used to size partitions */
+static bool numa_can_partition = false; /* can map to NUMA nodes? */
+static int	numa_buffers_per_node = -1; /* buffers per node */
+static int	numa_partitions = 0;	/* total (multiple of nodes) */
+
 
 /*
  * Data Structures:
@@ -70,19 +91,89 @@ BufferManagerShmemInit(void)
 	bool		foundBufs,
 				foundDescs,
 				foundIOCV,
-				foundBufCkpt;
+				foundBufCkpt,
+				foundParts;
+	Size		mem_page_size;
+	Size		buffer_align;
+
+	/*
+	/* allocate the partition registry first */
+	BufferPartitionsArray = (BufferPartitions *)
+		ShmemInitStruct("Buffer Partitions",
+						offsetof(BufferPartitions, partitions) +
+						mul_size(sizeof(BufferPartition), numa_partitions),
+						&foundParts);

 
 	/* Align condition variables to cacheline boundary. */
@@ -112,6 +203,12 @@ BufferManagerShmemInit(void)
 	{
 		int			i;
 
+		/*
+		 * Initialize the registry of buffer partitions, and also move the
+		 * memory to different NUMA nodes (if enabled by GUC)
+		 */
+		buffer_partitions_init();
+
 		/*
 		 * Initialize all the buffer headers.
 		 */
@@ -144,6 +241,11 @@ BufferManagerShmemInit(void)
 		GetBufferDescriptor(NBuffers - 1)->freeNext = FREENEXT_END_OF_LIST;
 	}
 
+	/*
 
@@ -152,24 +254,68 @@ BufferManagerShmemInit(void)
 						 &backend_flush_after);
 }
 
+/*
 
+	/* calculate partition info for buffers */
+	buffer_partitions_prepare();
+
 	/* size of buffer descriptors */
 	size = add_size(size, mul_size(NBuffers, sizeof(BufferDescPadded)));
 	/* to allow aligning buffer descriptors */
-	size = add_size(size, PG_CACHE_LINE_SIZE);
+	size = add_size(size, Max(numa_page_size, PG_IO_ALIGN_SIZE));
 
 	/* size of data pages, plus alignment padding */
-	size = add_size(size, PG_IO_ALIGN_SIZE);
+	size = add_size(size, Max(numa_page_size, PG_IO_ALIGN_SIZE));
 	size = add_size(size, mul_size(NBuffers, BLCKSZ));
 
 	/* size of stuff controlled by freelist.c */
@@ -184,5 +330,467 @@ BufferManagerShmemSize(void)
 	/* size of checkpoint sort array in bufmgr.c */
 	size = add_size(size, mul_size(NBuffers, sizeof(CkptSortItem)));
 
+	/* account for registry of NUMA partitions */
+	size = add_size(size, MAXALIGN(offsetof(BufferPartitions, partitions) +
+								   mul_size(sizeof(BufferPartition), numa_partitions)));
+
 	return size;
 }
+
+/*
+ * Calculate the NUMA node for a given buffer.
+ */
+int
+BufferGetNode(Buffer buffer)
+{
+	/* not NUMA interleaving */
+	if (numa_buffers_per_node == -1)
+		return 0;
+
+	return (buffer / numa_buffers_per_node);
+}
+
+/*
+ * pg_numa_interleave_memory
+ *		move memory to different NUMA nodes in larger chunks
+ *
+ * startptr - start of the region (should be aligned to page size)
+ * endptr - end of the region (doesn't need to be aligned)
+ * mem_page_size - size of the memory page size
+ * chunk_size - size of the chunk to move to a single node (should be multiple
+ *              of page size
+ * num_nodes - number of nodes to allocate memory to
+ *
+ * XXX Maybe this should use numa_tonode_memory and numa_police_memory instead?
+ * That might be more efficient than numa_move_pages, as it works on larger
+ * chunks of memory, not individual system pages, I think.
+ *
+ * XXX The "interleave" name is not quite accurate, I guess.
+ */
+static void
+pg_numa_move_to_node(char *startptr, char *endptr, int node)
+{
+	Size		mem_page_size;
+	Size		sz;
+
+	/*
+	 * Get the "actual" memory page size, not the one we used for sizing. We
+	 * might have used huge page for sizing, but only get regular pages when
+	 * allocating, so we must use the smaller pages here.
+	 *
+	 * XXX A bit weird. Do we need to worry about postmaster? Could this even
+	 * run outside postmaster? I don't think so.
+	 */
+	if (IsUnderPostmaster)
+		mem_page_size = pg_get_shmem_pagesize();
+	else
+		mem_page_size = get_memory_page_size();
+
+	Assert((int64) startptr % mem_page_size == 0);
+
+	sz = (endptr - startptr);
+	numa_tonode_memory(startptr, sz, node);
+}
+
+
+#define MIN_BUFFER_PARTITIONS	4
+
+/*
+ * buffer_partitions_prepare
+ *		Calculate parameters for partitioning buffers.
+ *
+ * We want to split the shared buffers into multiple partitions, of roughly
+ * the same size. This is meant to serve multiple purposes. We want to map
+ * the partitions to different NUMA nodes, to balance memory usage, and
+ * allow partitioning some data structures built on top of buffers, to give
+ * preference to local access (buffers on the same NUMA node). This applies
+ * mostly to freelists and clocksweep.
+ *
+ * We may want to use partitioning even on non-NUMA systems, or when running
+ * on a single NUMA node. Partitioning the freelist/clocksweep is beneficial
+ * even without the NUMA effects.
+ *
+ * So we try to always build at least 4 partitions (MIN_BUFFER_PARTITIONS)
+ * in total, or at least one partition per NUMA node. We always create the
+ * same number of partitions per NUMA node.
+ *
+ * Some examples:
+ *
+ * - non-NUMA system (or 1 NUMA node): 4 partitions for the single node
+ *
+ * - 2 NUMA nodes: 4 partitions, 2 for each node
+ *
+ * - 3 NUMA nodes: 6 partitions, 2 for each node
+ *
+ * - 4+ NUMA nodes: one partition per node
+ *
+ * NUMA works on the memory-page granularity, which determines the smallest
+ * amount of memory we can allocate to single node. This is determined by
+ * how many BufferDescriptors fit onto a single memory page, so this depends
+ * on huge page support. With 2MB huge pages (typical on x86 Linux), this is
+ * 32768 buffers (256MB). With regular 4kB pages, it's 64 buffers (512KB).
+ *
+ * Note: This is determined before the allocation, i.e. we don't know if the
+ * allocation got to use huge pages. So unless huge_pages=off we assume we're
+ * using huge pages.
+ *
+ * This minimal size requirement only matters for the per-node amount of
+ * memory, not for the individual partitions. The partitions for the same
+ * node are a contiguous chunk of memory, which can be split arbitrarily,
+ * it's independent of the NUMA granularity.
+ *
+ * XXX This patch only implements placing the buffers onto different NUMA
+ * nodes. The freelist/clocksweep partitioning is implemented in separate
+ * patches later in the patch series. Those patches however use the same
+ * buffer partition registry, to align the partitions.
+ *
+ *
+ * XXX This needs to consider the minimum chunk size, i.e. we can't split
+ * buffers beyond some point, at some point it gets we run into the size of
+ * buffer descriptors. Not sure if we should give preference to one of these
+ * (probably at least print a warning).
+ *
+ * XXX We want to do this even with numa_buffers_interleave=false, so that the
+ * other patches can do their partitioning. But in that case we don't need to
+ * enforce the min chunk size (probably)?
+ *
+ * XXX We need to only call this once, when sizing the memory. But at that
+ * point we don't know if we get to use huge pages or not (unless when huge
+ * pages are disabled). We'll proceed as if the huge pages were used, and we
+ * may have to use larger partitions. Maybe there's some sort of fallback,
+ * but for now we simply disable the NUMA partitioning - it simply means the
+ * shared buffers are too small.
+ *
+ * XXX We don't need to make each partition a multiple of min_partition_size.
+ * That's something we need to do for a node (because NUMA works at granularity
+ * of pages), but partitions for a single node can split that arbitrarily.
+ * Although keeping the sizes power-of-two would allow calculating everything
+ * as shift/mask, without expensive division/modulo operations.
+ */
+static void
+buffer_partitions_prepare(void)
+{
+	/*
+	 * Minimum number of buffers we can allocate to a NUMA node (determined by
+	 * how many BufferDescriptors fit onto a memory page).
+	 */
+	int			min_node_buffers;
+
+	/*
+	 * Maximum number of nodes we can split shared buffers to, assuming each
+	 * node gets the smallest allocatable chunk (the last node can get a
+	 * smaller amount of memory, not the full chunk).
+	 */
+	int			max_nodes;
+
+	/*
+	 * How many partitions to create per node. Could be more than 1 for small
+	 * number of nodes (of non-NUMA systems).
+	 */
+	int			num_partitions_per_node;
+
+	/* bail out if already initialized (calculate only once) */
+	if (numa_nodes != -1)
+		return;
+
+	/* XXX only gives us the number, the nodes may not be 0, 1, 2, ... */
+	numa_nodes = numa_num_configured_nodes();
+
+	/* XXX can this happen? */
+	if (numa_nodes < 1)
+		numa_nodes = 1;
+
+	elog(WARNING, "IsUnderPostmaster %d", IsUnderPostmaster);
+
+	/*
+	 * XXX A bit weird. Do we need to worry about postmaster? Could this even
+	 * run outside postmaster? I don't think so.
+	 *
+	 * XXX Another issue is we may get different values than when sizing the
+	 * the memory, because at that point we didn't know if we get huge pages,
+	 * so we assumed we will. Shouldn't cause crashes, but we might allocate
+	 * shared memory and then not use some of it (because of the alignment
+	 * that we don't actually need). Not sure about better way, good for now.
+	 */
+	if (IsUnderPostmaster)
+		numa_page_size = pg_get_shmem_pagesize();
+	else
+		numa_page_size = get_memory_page_size();
+
+	/* make sure the chunks will align nicely */
+	Assert(BLCKSZ % sizeof(BufferDescPadded) == 0);
+	Assert(numa_page_size % sizeof(BufferDescPadded) == 0);
+	Assert(((BLCKSZ % numa_page_size) == 0) || ((numa_page_size % BLCKSZ) == 0));
+
+	/*
+	 * The minimum number of buffers we can allocate from a single node, using
+	 * the memory page size (determined by buffer descriptors). NUMA allocates
+	 * memory in pages, and we need to do that for both buffers and
+	 * descriptors at the same time.
+	 *
+	 * In practice the BLCKSZ doesn't really matter, because it's much larger
+	 * than BufferDescPadded, so the result is determined buffer descriptors.
+	 */
+	min_node_buffers = (numa_page_size / sizeof(BufferDescPadded));
+
+	/*
+	 * Maximum number of nodes (each getting min_node_buffers) we can handle
+	 * given the current shared buffers size. The last node is allowed to be
+	 * smaller (half of the other nodes).
+	 */
+	max_nodes = (NBuffers + (min_node_buffers / 2)) / min_node_buffers;
+
+	/*
+	 * Can we actually do NUMA partitioning with these settings? If we can't
+	 * handle the current number of nodes, then no.
+	 *
+	 * XXX This shouldn't be a big issue in practice. NUMA systems typically
+	 * run with large shared buffers, which also makes the imbalance issues
+	 * fairly significant (it's quick to rebalance 128MB, much slower to do
+	 * that for 256GB).
+	 */
+	numa_can_partition = true;	/* assume we can allocate to nodes */
+	if (numa_nodes > max_nodes)
+	{
+		elog(WARNING, "shared buffers too small for %d nodes (max nodes %d)",
+			 numa_nodes, max_nodes);
+		numa_can_partition = false;
+	}
+
+	/*
+	 * We know we can partition to the desired number of nodes, now it's time
+	 * to figure out how many partitions we need per node. We simply add
+	 * partitions per node until we reach MIN_BUFFER_PARTITIONS.
+	 *
+	 * XXX Maybe we should make sure to keep the actual partition size a power
+	 * of 2, to make the calculations simpler (shift instead of mod).
+	 */
+	num_partitions_per_node = 1;
+
+	while (numa_nodes * num_partitions_per_node < MIN_BUFFER_PARTITIONS)
+		num_partitions_per_node++;
+
+	/* now we know the total number of partitions */
+	numa_partitions = (numa_nodes * num_partitions_per_node);
+
+	/*
+	 * Finally, calculate how many buffers we'll assign to a single NUMA node.
+	 * If we have only a single node, or can't map to that many nodes, just
+	 * take a "fair share" of buffers.
+	 *
+	 * XXX In both cases the last node can get fewer buffers.
+	 */
+	if (!numa_can_partition)
+	{
+		numa_buffers_per_node = (NBuffers + (numa_nodes - 1)) / numa_nodes;
+	}
+	else
+	{
+		numa_buffers_per_node = min_node_buffers;
+		while (numa_buffers_per_node * numa_nodes < NBuffers)
+			numa_buffers_per_node += min_node_buffers;
+
+		/* the last node should get at least some buffers */
+		Assert(NBuffers - (numa_nodes - 1) * numa_buffers_per_node > 0);
+	}
+
+	elog(LOG, "NUMA: buffers %d partitions %d num_nodes %d per_node %d buffers_per_node %d (min %d)",
+		 NBuffers, numa_partitions, numa_nodes, num_partitions_per_node,
+		 numa_buffers_per_node, min_node_buffers);
+}
+
+static void
+AssertCheckBufferPartitions(void)
+{
+#ifdef USE_ASSERT_CHECKING
+	int			num_buffers = 0;
+
+	for (int i = 0; i < numa_partitions; i++)
+	{
+		BufferPartition *part = &BufferPartitionsArray->partitions[i];
+
+		/*
+		 * We can get a single-buffer partition, if the sizing forces the last
+		 * partition to be just one buffer. But it's unlikely (and
+		 * undesirable).
+		 */
+		Assert(part->first_buffer <= part->last_buffer);
+		Assert((part->last_buffer - part->first_buffer + 1) == part->num_buffers);
+
+		num_buffers += part->num_buffers;
+
+		/*
+		 * The first partition needs to start on buffer 0. Later partitions
+		 * need to be contiguous, without skipping any buffers.
+		 */
+		if (i == 0)
+		{
+			Assert(part->first_buffer == 0);
+		}
+		else
+		{
+			BufferPartition *prev = &BufferPartitionsArray->partitions[i - 1];
+
+			Assert((part->first_buffer - 1) == prev->last_buffer);
+		}
+
+		/* the last partition needs to end on buffer (NBuffers - 1) */
+		if (i == (numa_partitions - 1))
+		{
+			Assert(part->last_buffer == (NBuffers - 1));
+		}
+	}
+
+	Assert(num_buffers == NBuffers);
+#endif
+}
+
+static void
+buffer_partitions_init(void)
+{
+	int			remaining_buffers = NBuffers;
+	int			buffer = 0;
+	int			parts_per_node = (numa_partitions / numa_nodes);
+	char	   *buffers_ptr,
+			   *descriptors_ptr;
+
+	BufferPartitionsArray->npartitions = numa_partitions;
+
+	for (int n = 0; n < numa_nodes; n++)
+	{
+		/* buffers this node should get (last node can get fewer) */
+		int			node_buffers = Min(remaining_buffers, numa_buffers_per_node);
+
+		/* split node buffers netween partitions (last one can get fewer) */
+		int			part_buffers = (node_buffers + (parts_per_node - 1)) / parts_per_node;
+
+		remaining_buffers -= node_buffers;
+
+		Assert((node_buffers > 0) && (node_buffers <= NBuffers));
+		Assert((n >= 0) && (n < numa_nodes));
+
+		for (int p = 0; p < parts_per_node; p++)
+		{
+			int			idx = (n * parts_per_node) + p;
+			BufferPartition *part = &BufferPartitionsArray->partitions[idx];
+			int			num_buffers = Min(node_buffers, part_buffers);
+
+			Assert((idx >= 0) && (idx < numa_partitions));
+			Assert((buffer >= 0) && (buffer < NBuffers));
+			Assert((num_buffers > 0) && (num_buffers <= part_buffers));
+
+			/* XXX we should get the actual node ID from the mask */
+			part->numa_node = n;
+
+			part->num_buffers = num_buffers;
+			part->first_buffer = buffer;
+			part->last_buffer = buffer + (num_buffers - 1);
+
+			elog(LOG, "NUMA: buffer %d node %d partition %d buffers %d first %d last %d", idx, n, p, num_buffers, buffer, buffer + (num_buffers - 1));
+
+			buffer += num_buffers;
+			node_buffers -= part_buffers;
+		}
+	}
+
+	AssertCheckBufferPartitions();
+
+	/*
+	 * With buffers interleaving disabled (or can't partition, because of
+	 * shared buffers being too small), we're done.
+	 */
+	if (!numa_buffers_interleave || !numa_can_partition)
+		return;
+
+	/*
+	 * Assign chunks of buffers and buffer descriptors to the available NUMA
+	 * nodes. We can't use the regular interleaving, because with regular
+	 * memory pages (smaller than BLCKSZ) we'd split all buffers to multiple
+	 * NUMA nodes. And we don't want that.
+	 *
+	 * But even with huge pages it seems like a good idea to not have mapping
+	 * for each page.
+	 *
+	 * So we always assign a larger contiguous chunk of buffers to the same
+	 * NUMA node, as calculated by choose_chunk_buffers(). We try to keep the
+	 * chunks large enough to work both for buffers and buffer descriptors,
+	 * but not too large. See the comments at choose_chunk_buffers() for
+	 * details.
+	 *
+	 * Thanks to the earlier alignment (to memory page etc.), we know the
+	 * buffers won't get split, etc.
+	 *
+	 * This also makes it easier / straightforward to calculate which NUMA
+	 * node a buffer belongs to (it's a matter of divide + mod). See
+	 * BufferGetNode().
+	 *
+	 * We need to account for partitions being of different length, when the
+	 * NBuffers is not nicely divisible. To do that we keep track of the start
+	 * of the next partition.
+	 */
+	buffers_ptr = BufferBlocks;
+	descriptors_ptr = (char *) BufferDescriptors;
+
+	for (int i = 0; i < numa_partitions; i++)
+	{
+		BufferPartition *part = &BufferPartitionsArray->partitions[i];
+		char	   *startptr,
+				   *endptr;
+
+		/* first map buffers */
+		startptr = buffers_ptr;
+		endptr = startptr + ((Size) part->num_buffers * BLCKSZ);
+		buffers_ptr = endptr;	/* start of the next partition */
+
+		elog(LOG, "NUMA: buffer_partitions_init: %d => %d buffers %d start %p end %p (size %ld)",
+			 i, part->numa_node, part->num_buffers, startptr, endptr, (endptr - startptr));
+
+		pg_numa_move_to_node(startptr, endptr, part->numa_node);
+
+		/* now do the same for buffer descriptors */
+		startptr = descriptors_ptr;
+		endptr = startptr + ((Size) part->num_buffers * sizeof(BufferDescPadded));
+		descriptors_ptr = endptr;
+
+		elog(LOG, "NUMA: buffer_partitions_init: %d => %d descriptors %d start %p end %p (size %ld)",
+			 i, part->numa_node, part->num_buffers, startptr, endptr, (endptr - startptr));
+
+		pg_numa_move_to_node(startptr, endptr, part->numa_node);
+	}
+
+	/* we should have consumed the arrays exactly */
+	Assert(buffers_ptr == BufferBlocks + (Size) NBuffers * BLCKSZ);
+	Assert(descriptors_ptr == (char *) BufferDescriptors + (Size) NBuffers * sizeof(BufferDescPadded));
+}
+
+int
+BufferPartitionCount(void)
+{
+	return BufferPartitionsArray->npartitions;
+}
+
+void
+BufferPartitionGet(int idx, int *node, int *num_buffers,
+				   int *first_buffer, int *last_buffer)
+{
+	if ((idx >= 0) && (idx < BufferPartitionsArray->npartitions))
+	{
+		BufferPartition *part = &BufferPartitionsArray->partitions[idx];
+
+		*node = part->numa_node;
+		*num_buffers = part->num_buffers;
+		*first_buffer = part->first_buffer;
+		*last_buffer = part->last_buffer;
+
+		return;
+	}
+
+	elog(ERROR, "invalid partition index");
+}
+
+void
+BufferPartitionParams(int *num_partitions, int *num_nodes)
+{
+	*num_partitions = numa_partitions;
+	*num_nodes = numa_nodes;
+}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..876cb64cf66 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -145,6 +145,9 @@ int			max_worker_processes = 8;
 int			max_parallel_workers = 8;
 int			MaxBackends = 0;
 
+/* NUMA stuff */

+	{
 
+extern PGDLLIMPORT bool numa_buffers_interleave;
+
 extern PGDLLIMPORT int commit_timestamp_buffers;
 extern PGDLLIMPORT int multixact_member_buffers;
 extern PGDLLIMPORT int multixact_offset_buffers;
diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h
index 52a71b138f7..9dfbecb9fe4 100644
--- a/src/include/storage/buf_internals.h
+++ b/src/include/storage/buf_internals.h
@@ -323,6 +323,7 @@ typedef struct WritebackContext
 
 /* in buf_init.c */
 extern PGDLLIMPORT BufferDescPadded *BufferDescriptors;
+extern PGDLLIMPORT BufferPartitions *BufferPartitionsArray;
 extern PGDLLIMPORT ConditionVariableMinimallyPadded *BufferIOCVArray;
 extern PGDLLIMPORT WritebackContext BackendWritebackContext;
 
@@ -491,4 +492,9 @@ extern void DropRelationLocalBuffers(RelFileLocator rlocator,
 extern void DropRelationAllLocalBuffers(RelFileLocator rlocator);
 extern void AtEOXact_LocalBuffers(bool isCommit);
 
+extern int	BufferPartitionCount(void);
+extern void BufferPartitionGet(int idx, int *node, int *num_buffers,
+							   int *first_buffer, int *last_buffer);
+extern void BufferPartitionParams(int *num_partitions, int *num_nodes);
+
 #endif							/* BUFMGR_INTERNALS_H */
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index 41fdc1e7693..deaf4f19fa4 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -143,6 +143,20 @@ struct ReadBuffersOperation
 
 typedef struct ReadBuffersOperation ReadBuffersOperation;
 
+typedef struct BufferPartition
+{
+	int			numa_node;
+	int			num_buffers;
+	int			first_buffer;
+	int			last_buffer;
+} BufferPartition;
+
+typedef struct BufferPartitions
+{
+	int			npartitions;
+	BufferPartition partitions[FLEXIBLE_ARRAY_MEMBER];
+} BufferPartitions;
+
 /* forward declared, to avoid having to expose buf_internals.h here */
 struct WritebackContext;
 
@@ -319,6 +333,7 @@ extern void EvictRelUnpinnedBuffers(Relation rel,
 /* in buf_init.c */
 extern void BufferManagerShmemInit(void);
 extern Size BufferManagerShmemSize(void);
+extern int	BufferGetNode(Buffer buffer);
 
 /* in localbuf.c */
 extern void AtProcExit_LocalBuffers(void);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 3daba26b237..c695cfa76e8 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -346,6 +346,8 @@ BufferDescPadded
 BufferHeapTupleTableSlot
 BufferLookupEnt
 BufferManagerRelation
+BufferPartition
+BufferPartitions
 BufferStrategyControl
 BufferTag
 BufferUsage
-- 
2.50.1

---

From: Jakub Wartak <[email-masked]>
Date: Wed, 30 Jul 2025 10:29:06 +0200

On Mon, Jul 28, 2025 at 4:22 PM Tomas Vondra <[email-masked]> wrote:

Hi Tomas,

just a quick look here:

> 2) The PGPROC part introduces a similar registry, [..]
>

If you are looking for better names: pg_shmem_pgproc_numa would sound
like a more natural name.

> 3) The PGPROC partitioning is reworked and should fix the crash with the

Thanks!

[..]
> than either "none" or "random").

Hint: real world is that network cards are usually located on some PCI
slot that is assigned to certain node (so traffic is flowing from/to
there), so probably it would make some sense to put pgbench outside
this machine and remove this as "variable" anyway and remove the need
for that pgbench --pin-cpus in script. In optimal conditions: most
optimized layout would be probably to have 2 cards on separate PCI
slots, each for different node and some LACP between those, with
xmit_hash_policy allowing traffic distribution on both of those cards
-- usually there's not just single IP/MAC out there talking to/from
such server, so that would be real-world (or lack of) affinity.

Also classic pgbench workload, seems to be poor fit for testing it out
(at least v3-0001 buffers), there I would propose sticking to just
lots of big (~s_b size) full table seq scans to put stress on shared
memory. Classic pgbench is usually not there enough to put serious
bandwidth on the interconnect by my measurements.

>       mode   s_b buffers localal no-tail freelist sweep pgproc pinning

Isn't the "pinning" column representing just numa_procs_pin=on ?
(shouldn't it be tested with numa_procs_interleave = on?)

[..]
> intentionally cause (or try to) imbalance. If you have ideas for such

Some ideas:
1. concurrent seq scans hitting s_b-sized table
2. one single giant PX-enabled seq scan with $VCPU workers (stresses
the importance of interleaving dynamic shm for workers)
3. select txid_current() with -M prepared?

[..]
> is enough huge pages and the system can allocate memory from any NUMA

Yup, similiar story as with OOMs just for per-zone/node.

> memory on the node, and realizes there's not enough huge pages. And

I think that's why options for strict policy numa allocation exist and
I had the option to use it in my patches (anyway with one big call to
numa_interleave_memory() for everything it was much more simpler and
just not micromanaging things). Good reads are numa(3) but e.g.
mbind(2) underneath will tell you that e.g. `Before Linux 5.7.
MPOL_MF_STRICT was ignored on huge page mappings.` (I was on 6.14.x,
but it could be happening for you too if you start using it). Anyway,
numa_set_strict() is just wrapper around setting this exact flag

Anyway remember that volatile pg_numa_touch_mem_if_required()? - maybe
that should be always called in your patch series to pre-populate
everything during startup, so that others testing will get proper
guaranteed layout, even without issuing any pg_buffercache calls.

> huge pages for the partitioned parts, this wouldn't be a problem.

Meh, sacrificing a couple of huge pages (worst-case 1GB ?) just to get
NUMA affinity, seems like a logical trade-off, doesn't it?
But postgres -C shared_memory_size_in_huge_pages  still works OK to
establish the exact count for vm.nr_hugepages, right?

Regards,
-J.

---

From: Tomas Vondra <[email-masked]>
Date: Wed, 30 Jul 2025 13:10:45 +0200

> such server, so that would be real-world (or lack of) affinity.

The pgbench pinning certainly reduces some of the noise / overhead you
get when using multiple machines. I use it to "isolate" patches, and
make the effects more visible.

> bandwidth on the interconnect by my measurements.

Yes, that's possible. The simple pgbench workload is a bit of a "worst
case" for the NUMA patches, in that it's can benefit less from the
improvements, and it's also fairly sensitive to regressions.

I plan to do more tests with other types of workloads, like the one
doing a lot of large sequential scans, etc.

> (shouldn't it be tested with numa_procs_interleave = on?)

Maybe I don't understand the question, but the last column (pinning)
compares two builds.

1) Build with all the patches up to "pgproc interleaving" (and all of
the GUCs set to "on").

2) Build with all the patches from (1), and "pinning" too (again, all
GUCs set to "on).

Or do I misunderstand the question?

> 3. select txid_current() with -M prepared?

Thanks. I think we'll try something like (1), but it'll need to be a bit
more elaborate, because scans on tables larger than 1/4 shared buffers
use a small circular buffer.

> guaranteed layout, even without issuing any pg_buffercache calls.

I think I tried using numa_set_strict, but it didn't change the behavior
(i.e. the numa_tonode_memory didn't error out).

> establish the exact count for vm.nr_hugepages, right?

Well, yes and no. It tells you the exact number of huge pages, but it
does not tell you how much you need to inflate it to account for the
non-shared buffer part that may get allocated on a random node.

regards

-- 
Tomas Vondra

---

From: Tomas Vondra <[email-masked]>
Date: Mon, 4 Aug 2025 16:24:40 +0200

Hi,

Here's an updated version of the patch series. The main improvement is
the new 0006 patch, adding "adaptive balancing" of allocations. I'll
also share some results from a workload doing a lot of allocations.

adaptive balancing of allocations
---------------------------------

Imagine each backend only allocates buffers from the partition on the
same NUMA node. E.g. you have 4 NUMA nodes (i.e. 4 partitions), and a
backend only allocates buffers from "home" partition (on the same NUMA
node). This is what the earlier patch versions did, and with many
backends that's mostly fine (assuming the backends get spread over all
the NUMA nodes).

But if there's only few backends doing the allocations, this can result
in very inefficient use of shared buffers - a single backend would be
limited to 25% of buffers, even if the rest is unused.

There needs to be some say to "redirect" excess allocations to other
partitions, so that the partitions are utilized about the same. This is
what the 0006 patch aims to do (I kept is separate, but it should
probably get merged into the "clocksweep partitioning" in the end).

The balancing is fairly simple:

(1) It tracks the number of allocations "requested" from each partition.

(2) In regular intervals (by bgwriter) calculate the "fair share" per
partition, and determine what fraction of "requests" to handle from the
partition itself, and how many to redirect to other partitions.

(3) Calculate coefficients to drive this for each partition.

I emphasize (1) talks about "requests", not the actual allocations. Some
of the requests could have been redirected to different partitions, and
be counted as allocations there. We want to balance allocations, but it
relies on the requests.

To give you a simple example - imagine there are 2 partitions with this
number of allocation requests:

  P1: 900,000 requests
  P2: 100,000 requests

This means the "fair share" is 500,000 allocations, so P1 needs to
redirect some requests to P2. And we end with these weights:

  P1: [ 55,  45]
  P2: [  0, 100]

Assuming the workload does not shift in some dramatic way, this should
result in both partitions handling ~500k allocations.

It's not hard to extend this algorithm to more partitions. For more
details see StrategySyncBalance(), which recalculates this.

There are a couple open questions, like:

* The algorithm combines the old/new weights by averaging, to add a bit
of hysteresis. Right now it's a simple average with 0.5 weight, to
dampen sudden changes. I think it works fine (in the long run), but I'm
open to suggestions how to do this better.

* There's probably additional things we should consider when deciding
where to redirect the allocations. For example, we may have multiple
partitions per NUMA node, in which case it's better to redirect to that
node as many allocations as possible. The current patch ignores this.

* The partitions may have slightly different sizes, but the balancing
ignores that for now. This is not very difficult to address.

clocksweep benchmark
--------------------

I ran a simple benchmark focused on allocation-heavy workloads, namely
large concurrent sequential scans. The attached scripts generate a
number of 1GB tables, and then run concurrent sequential scans with
shared buffers set to 60%, 75%, 90% and 110% of the total dataset size.

I did this for master, and with the NUMA patches applied (and the GUCs
set to 'on'). I also increased tried with the of partitions increased to
16 (so a NUMA node got multiple partitions).

There are results from three machines

1) ryzen - small non-NUMA system, mostly to see if there's regressions

2) xeon - older 2-node NUMA system

3) hb176 - big EPYC system with 176 cores / 4 NUMA nodes

The script records detailed TPS stats (e.g. percentiles), I'm attaching
CSV files with complete results, and some PDFs with charts summarizing
that (I'll get to that in a minute).

For the EPYC, the average tps for the three builds looks like this:

    clients  |  master      numa   numa-16    |    numa     numa-16
   ----------|--------------------------------|---------------------
          8  |      20        27        26    |    133%        129%
         16  |      23        39        45    |    170%        193%
         24  |      23        48        58    |    211%        252%
         32  |      21        57        68    |    268%        321%
         40  |      21        56        76    |    265%        363%
         48  |      22        59        82    |    270%        375%
         56  |      22        66        88    |    296%        397%
         64  |      23        62        93    |    277%        411%
         72  |      24        68        95    |    277%        389%
         80  |      24        72        95    |    295%        391%
         88  |      25        71        98    |    283%        392%
         96  |      26        74        97    |    282%        369%
        104  |      26        74        97    |    282%        367%
        112  |      27        77        95    |    287%        355%
        120  |      28        77        92    |    279%        335%
        128  |      27        75        89    |    277%        328%

That's not bad - the clocksweep partitioning increases the throughput
2-3x. Having 16 partitions (instead of 4) helps yet a bit more, to 3-4x.

This is for shared buffers set to 60% of the dataset, which depends on
the number of clients / tables. With 64 clients/tables, there's 64GB of
data, and shared buffers are set to ~39GB.

The results for 75% and 90% follow the same pattern. For 110% there's
much less impact - there are no allocations, so this has to be thanks to
the other NUMA patches.

The charts in the attached PDFs add a bit more detail, with various
percentiles (of per-second throughput). The bands are roughly quartiles:
5-25%, 25-50%, 50-75%, 75-95%. The thick middle line is the median.

There's only charts for 60%, 90% and 110% shared buffers, for fit it on
a single page. There 75% is not very different.

For ryzen there's little difference. Not surprising, it's not a NUMA
system. So this is positive result, as there's no regression.

For xeon the patches help a little bit. Again, not surprising. It's a
fairly old system (~2016), and the differences between NUMA nodes are
not that significant.

For epyc (hb176), the differences are pretty massive.

regards

-- 
Tomas Vondra

---

From: Tomas Vondra <[email-masked]>
Date: Thu, 7 Aug 2025 11:24:18 +0200

Hi!

Here's a slightly improved version of the patch series.

The main improvement is related to rebalancing partitions of different
sizes (which can happen because the sizes have to be a multiple of some
minimal "chunk" determined by memory page size etc.). Part 0009 deals
with that by adjusting the allocations by partition size. It works OK,
but it's also true it matters less as the shared_buffers size increases
(as the relative difference between large/small partition gets smaller).

The other improvements are related to the pg_buffercache_partitions
view, showing the weights and (running) totals of allocations.

I plan to take a break from this patch series for a while, so this would
be a good time to take a look, do a review, run some tests etc. ;-)

One detail about the balancing I forgot to mention in my last message is
how the patch "distributes" allocations to match the balancing weights.
Consider for example the example weights from that message:

  P1: [ 55,  45]
  P2: [  0, 100]

Imagine a backend located on P1 requests allocation of a buffer. The
weights say 55% buffers should be allocated from P1, and 45% should be
redirected to P2. One way to achieve that would be generating a random
number in [1, 100], and if it's [1,55] then P1, otherwise P2.

The patch does a much simpler thing - treat the weight as a "budget",
i.e. number of buffers to allocate before proceeding to the "next"
partition. So it allocates 55 buffers from P1, then 45 buffers from P2,
and then goes back to P1 in a round-robin way. The advantage is it can
do away without a PRNG.

There's two things I'm not entirely sure about:

1) memory model - I'm not quite sure the current code ensures updates to
weights are properly "communicated" to the other processes. That is, if
the bgwriter recalculates the weights, will the other backends see the
new weights right away? Using a stale weights won't cause "failures",
the consequence is just a bit of imbalance. But it shouldn't stay like
that for too long, so maybe it'd be good to add some memory barriers or
something like that.

2) I'm a bit unsure what "NUMA nodes" actually means. The patch mostly
assumes each core / piece of RAM is assigned to a particular NUMA node.
For the buffer partitioning the patch mostly cares about memory, as it
"locates" the buffers on different NUMA nodes. Which works mostly OK
(ignoring the issues with huge pages described in previous message).

But it also cares about the cores (and the node for each core), because
it uses that to pick the right partition for a backend. And here the
situation is less clear, because the CPUs don't need to be assigned to a
particular node, even on a NUMA system. Consider the rpi5 NUMA layout:

$ numactl --hardware
available: 8 nodes (0-7)
node 0 cpus: 0 1 2 3
node 0 size: 992 MB
node 0 free: 274 MB
node 1 cpus: 0 1 2 3
node 1 size: 1019 MB
node 1 free: 327 MB
node 2 cpus: 0 1 2 3
node 2 size: 1019 MB
node 2 free: 321 MB
node 3 cpus: 0 1 2 3
node 3 size: 955 MB
node 3 free: 251 MB
node 4 cpus: 0 1 2 3
node 4 size: 1019 MB
node 4 free: 332 MB
node 5 cpus: 0 1 2 3
node 5 size: 1019 MB
node 5 free: 342 MB
node 6 cpus: 0 1 2 3
node 6 size: 1019 MB
node 6 free: 352 MB
node 7 cpus: 0 1 2 3
node 7 size: 1014 MB
node 7 free: 339 MB
node distances:
node   0   1   2   3   4   5   6   7
  0:  10  10  10  10  10  10  10  10
  1:  10  10  10  10  10  10  10  10
  2:  10  10  10  10  10  10  10  10
  3:  10  10  10  10  10  10  10  10
  4:  10  10  10  10  10  10  10  10
  5:  10  10  10  10  10  10  10  10
  6:  10  10  10  10  10  10  10  10
  7:  10  10  10  10  10  10  10  10

This says there are 8 NUMA nodes, each with ~1GB of RAM. But the 4 cores
are not assigned to particular nodes - each core is mapped to all 8 NUMA
nodes. I'm not sure what to do about this (or how getcpu() or libnuma
handle this). And can the situation be even more complicated?

regards

-- 
Tomas Vondra
From 630e4f0c9ce995ff5a9f1aaf68e8c64836cadffb Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 27 May 2025 23:08:48 +0200
Subject: [PATCH v20250807 11/11] NUMA: pin backends to NUMA nodes

 
+	/*

+	{

From d2c7801011a394b5b9e00d60bf59e759b65e1bed Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Thu, 22 May 2025 18:39:08 +0200
Subject: [PATCH v20250807 10/11] NUMA: interleave PGPROC entries

 - PGPROC array / node #0
 - PGPROC array / node #1

diff --git a/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql b/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
index 5acae31b836..ba54f69eeb4 100644
--- a/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
+++ b/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
@@ -38,3 +38,22 @@ REVOKE ALL ON pg_buffercache_partitions FROM PUBLIC;
 
index 13014549d00..ee3aa8be2ce 100644
 #include "utils/array.h"
 #include "utils/builtins.h"
 #include "utils/rel.h"
@@ -30,6 +31,7 @@
 
 #define NUM_BUFFERCACHE_NUMA_ELEM	3
 #define NUM_BUFFERCACHE_PARTITIONS_ELEM	15
+#define NUM_BUFFERCACHE_PGPROC_ELEM	5
 
 PG_MODULE_MAGIC_EXT(
@@ -104,6 +106,7 @@ PG_FUNCTION_INFO_V1(pg_buffercache_evict);

@@ -946,3 +949,94 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 
 		/*

 
 
 /*
 
index bbe29bc9729..878d1e33f61 100644
@@ -510,7 +510,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 
 	/*

 
 /*

 

 	/*

 

 	/*

 
 
-
 /*

 			/*

 

 			/*

 

 		if (proc->pid == pid)
 		{
 
 			result = proc->pid;
 			break;
 
 		if (proc == MyProc)

 
 		/*
 
 		if (proc->pid == 0)
 
 		if (proc->pid == 0)

 		if (proc->pid == 0)

 

 
+#include <sched.h>
 #include <signal.h>
 #include <unistd.h>
 #include <sys/time.h>
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+

 
+	/*

 
+	/*

 

 
 
+	/*

 
 		/*

 

 

 /*
 
 	return ok;
 }
+

+	{

 

 
 /*
 
+extern int	ProcPartitionCount(void);
+extern void ProcPartitionGet(int idx, int *node, int *nprocs, void **procsptr, void **fpptr);
+
 #endif							/* _PROC_H_ */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 8540d537a3e..ded2db30422 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1876,6 +1876,7 @@ PGP_MPI
 PGP_PubKey
 PGP_S2K
 PGPing
+PGProcPartition
 PGQueryClass
 PGRUsage
 PGSemaphore
-- 
2.50.1

From 170673d4fe89bb9436c2af41216884fefa0a48ee Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 6 Aug 2025 01:09:57 +0200
Subject: [PATCH v20250807 09/11] NUMA: weighted clocksweep balancing

The partitions may not be of exactly the same size, so consider that
when balancing clocksweep allocations.
---
 src/backend/storage/buffer/freelist.c | 63 ++++++++++++++++++++++-----
 1 file changed, 53 insertions(+), 10 deletions(-)

diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index f2203cebcc8..bbe29bc9729 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -738,6 +738,20 @@ StrategySyncBalance(void)
 			avg_allocs,			/* average allocations (per partition) */
 			delta_allocs = 0;	/* sum of allocs above average */
 
+	/*
+	 * Size of a partition, used to calculate weighted average (the first
+	 * partition is expected to be the largest one, and so will be counted
+	 * as a "unit" partition with weight 1.0).
+	 */
+	int32	num_buffers = StrategyControl->sweeps[0].numBuffers;
+
+	/*
+	 * Total weight of partitions. If the partitions have the same size,
+	 * the weight should be equal the partition count (modulo rounding
+	 * errors, etc.)
+	 */
+	double	weight = 0.0;
+
 	/*
 	 * Collect the number of allocations requested in the past interval.
 	 * While at it, reset the counter to start the new interval.
@@ -764,16 +778,27 @@ StrategySyncBalance(void)
 		pg_atomic_fetch_add_u64(&sweep->numTotalRequestedAllocs, allocs[i]);
 
 		total_allocs += allocs[i];
+
+		/* weight of the partition, relative to the "unit" partition */
+		weight += (sweep->numBuffers * 1.0 / num_buffers);
 	}
 
 	/*
-	 * Calculate the "fair share" of allocations per partition.
+	 * XXX Not sure if the total_weight might exceed num_partitions due to
+	 * rounding errors.
+	 */
+	Assert((weight > 0.0) && (weight <= StrategyControl->num_partitions));
+
+	/*
+	 * Calculate the "fair share" of allocations per partition. This is the
+	 * number of allocations for the "unit" partition with num_buffers, we'll
+	 * need to adjust it using the per-partition weight.
 	 *
 	 * XXX The last partition could be smaller, in which case it should be
 	 * expected to handle fewer allocations. So this should be a weighted
 	 * average. But for now a simple average is good enough.
 	 */
-	avg_allocs = (total_allocs / StrategyControl->num_partitions);
+	avg_allocs = (total_allocs / weight);
 
 	/*
 	 * Calculate the "delta" from balanced state, i.e. how many allocations
@@ -781,8 +806,14 @@ StrategySyncBalance(void)
 	 */
 	for (int i = 0; i < StrategyControl->num_partitions; i++)
 	{
-		if (allocs[i] > avg_allocs)
-			delta_allocs += (allocs[i] - avg_allocs);
+		ClockSweep *sweep = &StrategyControl->sweeps[i];
+
+		/* number of allocations expected for this partition */
+		double	part_weight = (sweep->numBuffers * 1.0 / num_buffers);
+		uint32	part_allocs = avg_allocs * part_weight;
+
+		if (allocs[i] > part_allocs)
+			delta_allocs += (allocs[i] - part_allocs);
 	}
 
 	/*
@@ -845,6 +876,10 @@ StrategySyncBalance(void)
 		ClockSweep *sweep = &StrategyControl->sweeps[i];
 		uint8		balance[MAX_BUFFER_PARTITIONS];
 
+		/* number of allocations expected for this partition */
+		double	part_weight = (sweep->numBuffers * 1.0 / num_buffers);
+		uint32	part_allocs = avg_allocs * part_weight;
+
 		/* lock, we're going to modify the balance weights */
 		SpinLockAcquire(&sweep->clock_sweep_lock);
 
@@ -852,7 +887,7 @@ StrategySyncBalance(void)
 		memset(balance, 0, sizeof(uint8) * MAX_BUFFER_PARTITIONS);
 
 		/* does this partition has fewer or more than avg_allocs? */
-		if (allocs[i] < avg_allocs)
+		if (allocs[i] < part_allocs)
 		{
 			/* fewer - don't redirect any allocations elsewhere */
 			balance[i] = 100;
@@ -866,22 +901,30 @@ StrategySyncBalance(void)
 			 * a fraction proportional to (excess/delta) from this one.
 			 */
 
-			/* fraction of the "total" delta */
-			double	delta_frac = (allocs[i] - avg_allocs) * 1.0 / delta_allocs;
+			/* fraction of the "total" delta represented by "excess" allocations */
+			double	delta_frac = (allocs[i] - part_allocs) * 1.0 / delta_allocs;
 
 			/* keep just enough allocations to meet the target */
-			balance[i] = (100.0 * avg_allocs / allocs[i]);
+			balance[i] = (100.0 * part_allocs / allocs[i]);
 
 			/* redirect the extra allocations */
 			for (int j = 0; j < StrategyControl->num_partitions; j++)
 			{
+				ClockSweep *sweep2 = &StrategyControl->sweeps[j];
+
+				/* number of allocations expected for this partition */
+				double	part_weight_2 = (sweep2->numBuffers * 1.0 / num_buffers);
+				uint32	part_allocs_2 = avg_allocs * part_weight_2;
+
 				/* How many allocations to receive from i-th partition? */
-				uint32	receive_allocs = delta_frac * (avg_allocs - allocs[j]);
+				uint32	receive_allocs = delta_frac * (part_allocs_2 - allocs[j]);
 
 				/* ignore partitions that don't need additional allocations */
-				if (allocs[j] > avg_allocs)
+				if (allocs[j] > part_allocs_2)
 					continue;
 
+				Assert(receive_allocs >= 0);
+
 				/* fraction to redirect */
 				balance[j] = (100.0 * receive_allocs / allocs[i]) + 0.5;
 			}
-- 
2.50.1

From f832fedb847004990e7e806691ab4338752a5db5 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Thu, 31 Jul 2025 19:50:05 +0200
Subject: [PATCH v20250807 08/11] NUMA: clocksweep allocation balancing

If backends only allocate buffers from the "local" partition, this could
cause significant misbalance - some partitions might be overused, while
other partitions would be left unused. In other words, shared buffers
would not be used efficiently.

We want all partitions to be used about the same, i.e. serve about the
same number of allocations. To achieve that, allocations from partitions
that are "too busy" may get redirected to other partitions. The system
counts allocations requested from each partition, calculates the "fair
share" (average per partition), and then redirectsexcess allocations to
other partitions.

Each partition gets a set of coefficients determining the fraction of
allocations to redirect to other partitions. The coefficients may be
interpreted as a "budget" for each of the partition, i.e. the number of
allocations to serve from that partition, before moving to the next
partition (in a round-robin manner).

All of this is tied to the partition where the allocation was requested.
Each partition has a separate set of coefficients.

We might also treat the coefficients as probabilities, and use PRNG to
determine where to direct individual requests. But a PRNG seems fairly
expensive, and the budget approach works well.

We intentionally keep the "budget" fairly low, with the sum for a given
partition 100. That means we get to the same partition after only 100
allocations, keeping it more balanced. It wouldn't be hard to make the
budgets higher (e.g. matching the number of allocations per round), but
it might also make the behavior less smooth (long period of allocations
from each partition).

This is very simple/cheap, and over many allocations it has the same
effect. For periods of low activity it may diverge, but that does not
matter much (we care about high-activity periods much more).
---
 .../pg_buffercache--1.6--1.7.sql              |   5 +-
 contrib/pg_buffercache/pg_buffercache_pages.c |  41 +-
 src/backend/storage/buffer/bufmgr.c           |   3 +
 src/backend/storage/buffer/freelist.c         | 396 ++++++++++++++++--
 src/include/storage/buf_internals.h           |   1 +
 src/include/storage/bufmgr.h                  |   5 +-
 6 files changed, 420 insertions(+), 31 deletions(-)

diff --git a/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql b/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
index 999bb2128f0..5acae31b836 100644
--- a/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
+++ b/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
@@ -27,7 +27,10 @@ CREATE VIEW pg_buffercache_partitions AS
 	 num_passes bigint,			-- clocksweep passes
 	 next_buffer integer,		-- next victim buffer for clocksweep
 	 total_allocs bigint,		-- handled allocs (running total)
-	 num_allocs bigint);		-- handled allocs (current cycle)
+	 num_allocs bigint,			-- handled allocs (current cycle)
+	 total_req_allocs bigint,	-- requested allocs (running total)
+	 num_req_allocs bigint,		-- handled allocs (current cycle)
+	 weights int[]);			-- balancing weights
 
 -- Don't want these to be available to public.
 REVOKE ALL ON FUNCTION pg_buffercache_partitions() FROM PUBLIC;
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 7ca075e6164..13014549d00 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -15,6 +15,8 @@
 #include "port/pg_numa.h"
 #include "storage/buf_internals.h"
 #include "storage/bufmgr.h"
+#include "utils/array.h"
+#include "utils/builtins.h"
 #include "utils/rel.h"

@@ -27,7 +29,7 @@
 
 #define NUM_BUFFERCACHE_NUMA_ELEM	3
-#define NUM_BUFFERCACHE_PARTITIONS_ELEM	12
+#define NUM_BUFFERCACHE_PARTITIONS_ELEM	15
 
 PG_MODULE_MAGIC_EXT(
 					.name = "pg_buffercache",
@@ -789,6 +791,8 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 
 	if (SRF_IS_FIRSTCALL())
 	{
+		TypeCacheEntry *typentry = lookup_type_cache(INT4OID, 0);
+
 		funcctx = SRF_FIRSTCALL_INIT();
 
 		/* Switch context when allocating stuff to be used in later calls */
@@ -826,6 +830,12 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 						   INT8OID, -1, 0);
 		TupleDescInitEntry(tupledesc, (AttrNumber) 12, "num_allocs",
 						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 13, "total_req_allocs",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 14, "num_req_allocs",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 15, "weigths",
+						   typentry->typarray, -1, 0);

@@ -851,11 +861,17 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 					buffers_remain,
 					buffers_free;
 
-		uint64		buffer_total_allocs;
+		uint64		buffer_total_allocs,
+					buffer_total_req_allocs;
 
 		uint32		complete_passes,
 					next_victim_buffer,
-					buffer_allocs;
+					buffer_allocs,
+					buffer_req_allocs;
+
+		int		   *weights;
+		Datum	   *dweights;
+		ArrayType  *array;
 
 		Datum		values[NUM_BUFFERCACHE_PARTITIONS_ELEM];
 		bool		nulls[NUM_BUFFERCACHE_PARTITIONS_ELEM];
@@ -866,7 +882,15 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		FreelistPartitionGetInfo(i, &buffers_consumed, &buffers_remain,
 								 &buffers_free,
 								 &complete_passes, &next_victim_buffer,
-								 &buffer_total_allocs, &buffer_allocs);
+								 &buffer_total_allocs, &buffer_allocs,
+								 &buffer_total_req_allocs, &buffer_req_allocs,
+								 &weights);
+
+		dweights = palloc_array(Datum, funcctx->max_calls);
+		for (int i = 0; i < funcctx->max_calls; i++)
+			dweights[i] = Int32GetDatum(weights[i]);
+
+		array = construct_array_builtin(dweights, funcctx->max_calls, INT4OID);
 
 		values[0] = Int32GetDatum(i);
 		nulls[0] = false;
@@ -904,6 +928,15 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		values[11] = Int64GetDatum(buffer_allocs);
 		nulls[11] = false;
 
+		values[12] = Int64GetDatum(buffer_total_req_allocs);
+		nulls[12] = false;
+
+		values[13] = Int64GetDatum(buffer_req_allocs);
+		nulls[13] = false;
+
+		values[14] = PointerGetDatum(array);
+		nulls[14] = false;
+
 		/* Build and return the tuple. */
 		tuple = heap_form_tuple((TupleDesc) funcctx->user_fctx, values, nulls);
 		result = HeapTupleGetDatum(tuple);
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 7a8c45ac59c..97b6f973c26 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3623,6 +3623,9 @@ BgBufferSync(WritebackContext *wb_context)
 	/* assume we can hibernate, any partition can set to false */
 	bool		hibernate = true;
 
+	/* trigger partition rebalancing first */
+	StrategySyncBalance();
+
 	/* get the number of clocksweep partitions, and total alloc count */
 	StrategySyncPrepare(&num_partitions, &recent_alloc);
 
diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index 17988b4fd53..f2203cebcc8 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -51,6 +51,22 @@ typedef struct BufferStrategyFreelist
 	uint64		consumed;
 }			BufferStrategyFreelist;
 
+/*
+ * XXX needed for make ClockSweep fixed-size, should be tied to the number
+ * of buffer partitions.
+ */
+#define MAX_BUFFER_PARTITIONS		16
+
+/*
+ * Coefficient used to combine the old and new balance coefficients, using
+ * weighted average. The higher the value, the more the old value affects the
+ * result.
+ *
+ * XXX Doesn't this invalidate the interpretation as a probability to allocate
+ * from a given partition? Does it still sum to 100%?
+ */
+#define CLOCKSWEEP_HISTORY_COEFF	0.5
+
 /*
  * Information about one partition of the ClockSweep (on a subset of buffers).
  *
@@ -83,9 +99,28 @@ typedef struct
 	uint32		completePasses; /* Complete cycles of the clock sweep */
 	pg_atomic_uint32 numBufferAllocs;	/* Buffers allocated since last reset */
 
+	/*
+	 * Buffers that should have been allocated in this partition (but might
+	 * have been redirected to keep allocations balanced).
+	 */
+	pg_atomic_uint32 numRequestedAllocs;
+
 	/* running total of allocs */
 	pg_atomic_uint64 numTotalAllocs;
+	pg_atomic_uint64 numTotalRequestedAllocs;
 
+	/*
+	 * Weights to balance buffer allocations for all the partitions. Each
+	 * partition gets a vector of weights 0-100, determining what fraction
+	 * of buffers to allocate from that particular. So [75, 15, 5, 5] would
+	 * mean 75% allocations should go from partition 0, 15% from partition
+	 * 1, and 5% from partitions 2&3. Each partition gets a different vector
+	 * of weights.
+	 *
+	 * XXX Allocate a fixed-length array, to simplify working with array of
+	 * the structs, etc.
+	 */
+	uint8		balance[MAX_BUFFER_PARTITIONS];
 } ClockSweep;
 
 /*
@@ -153,7 +188,33 @@ static BufferDesc *GetBufferFromRing(BufferAccessStrategy strategy,
 									 uint32 *buf_state);
 static void AddBufferToRing(BufferAccessStrategy strategy,
 							BufferDesc *buf);
-static ClockSweep *ChooseClockSweep(void);
+static ClockSweep *ChooseClockSweep(bool balance);
+
+/*
+ * clocksweep allocation balancing
+ *
+ * To balance allocations from clocksweep partitions, each partition gets
+ * a set of "weights" determining the fraction of allocations to redirect
+ * to other partitions.
+ *
+ * We could do that based on a random number generator, but that seems too
+ * expensive. So instead we simply treat the probabilities as a budget, i.e.
+ * a number of allocations to serve from that partition, before moving to
+ * the next partition (in a round-robin manner).
+ *
+ * This is very simple/cheap, and over many allocations it has the same
+ * effect. For periods of low activity it may diverge, but that does not
+ * matter much (we care about high-activity periods much more).
+ *
+ * We intentionally keep the "budget" fairly low, with the sum for a given
+ * partition 100. That means we get to the same partition after only 100
+ * allocations, keeping it more balances. It wouldn't be hard to make the
+ * budgets higher (say, to match the expected number of allocations, i.e.
+ * about the average number of allocations from the past interval).
+ */
+static int clocksweep_partition_optimal = -1;
+static int clocksweep_partition = -1;
+static int clocksweep_count = 0;
 
 /*
  * ClockSweepTick - Helper routine for StrategyGetBuffer()
@@ -165,7 +226,7 @@ static inline uint32
 ClockSweepTick(void)
 {
 	uint32		victim;
-	ClockSweep *sweep = ChooseClockSweep();
+	ClockSweep *sweep = ChooseClockSweep(true);
 
 	/*
 	 * Atomically move hand ahead one buffer - if there's several processes
@@ -294,32 +355,72 @@ calculate_partition_index()
  * and that's cheaper. But how would that deal with odd number of nodes?
  */
 static ClockSweep *
-ChooseClockSweep(void)
+ChooseClockSweep(bool balance)
 {
-	int			index = calculate_partition_index();
+	/* What's the "optimal" partition? */
+	int		index = calculate_partition_index();
+	ClockSweep *sweep = &StrategyControl->sweeps[index];
+
+	/*
+	 * Did we migrate to a different core / NUMA node, affecting the
+	 * clocksweep partition we should use? Switch to that partition.
+	 */
+	if (clocksweep_partition_optimal != index)
+	{
+		clocksweep_partition_optimal = index;
+		clocksweep_partition = index;
+		clocksweep_count = sweep->balance[index];
+	}
+
+	/* we should have a valid partition */
+	Assert(clocksweep_partition_optimal != -1);
+	Assert(clocksweep_partition != -1);
+
+	/*
+	 * If rebalancing is enabled, use the weights to redirect the allocations
+	 * to match the desired distribution. We do that by using the partitions
+	 * in a round-robin way, after allocating the "weight" of allocations
+	 * from each partitions.
+	 */
+	if (balance)
+	{
+		/*
+		 * Ran out of allocations from the current partition? Move to the
+		 * next partition with non-zero weight, and use the weight as a
+		 * budget for allocations.
+		 */
+		while (clocksweep_count == 0)
+		{
+			clocksweep_partition
+				= (clocksweep_partition + 1) % StrategyControl->num_partitions;
+
+			Assert((clocksweep_partition >= 0) &&
+				   (clocksweep_partition < StrategyControl->num_partitions));
+
+			clocksweep_count = sweep->balance[clocksweep_partition];
+		}
+
+		/* account for the allocation - take it from the budget */
+		--clocksweep_count;
 
-	return &StrategyControl->sweeps[index];
+		/* account for the alloc in the "optimal" (original) partition */
+		pg_atomic_fetch_add_u32(&sweep->numRequestedAllocs, 1);
+	}
+
+	return &StrategyControl->sweeps[clocksweep_partition];
 }
 
 /*
  * ChooseFreeList
- *		Pick the buffer freelist to use, depending on the CPU and NUMA node.
- *
- * Without partitioned freelists (numa_partition_freelist=false), there's only
- * a single freelist, so use that.
- *
- * With partitioned freelists, we have multiple ways how to pick the freelist
- * for the backend:
- *
- * - one freelist per CPU, use the freelist for CPU the task executes on
- *
- * - one freelist per NUMA node, use the freelist for node task executes on
+ *		pick a clocksweep partition based on NUMA node and CPU
  *
- * - use fixed number of freelists, map processes to lists based on PID
+ * The number of freelist partitions may not match the number of NUMA
+ * nodes, but it should not be lower. Each partition should be mapped to
+ * a single NUMA node, but a node may have multiple partitions. If there
+ * are multiple partitions per node (all nodes have the same number of
+ * partitions), we pick the partition using CPU.
  *
- * There may be some other strategies, not sure. The important thing is this
- * needs to be refrecled during initialization, i.e. we need to create the
- * right number of lists.
+ * XXX Maybe this should use the same balancing strategy as clocksweep?
  */
 static BufferStrategyFreelist *
 ChooseFreeList(void)
@@ -417,7 +518,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 	 * the rate of buffer consumption.  Note that buffers recycled by a
 	 * strategy object are intentionally not counted here.
 	 */
-	pg_atomic_fetch_add_u32(&ChooseClockSweep()->numBufferAllocs, 1);
+	pg_atomic_fetch_add_u32(&ChooseClockSweep(false)->numBufferAllocs, 1);
 
 	/*
 	 * First check, without acquiring the lock, whether there's buffers in the
@@ -580,6 +681,224 @@ StrategyFreeBuffer(BufferDesc *buf)
 	SpinLockRelease(&freelist->freelist_lock);
 }
 
+/*
+ * StrategySyncBalance
+ *		update partition weights, to balance the buffer allocations
+ *
+ * We want to give preference to allocating buffers on the same NUMA node,
+ * but that might lead to imbalance - a single process would only use a
+ * fraction of shared buffers. We don't want that, we want to utilize the
+ * whole shared buffers. The number of allocations in each partition may
+ * also change over time, so we need to adapt to that.
+ *
+ * To allow this "adaptive balancing", each partition has a set of weights,
+ * determining what fraction of allocations to direct to other partitions.
+ * For simplicity the coefficients are integers 0-100, expressing the
+ * percentage of allocations redirected to that partition.
+ *
+ * Consider for example weights [50, 25, 25, 0] for one of 4 partitions.
+ * This means 50% of allocations will be redirected to partition 0, 25%
+ * to partitions 1 and 2, and no allocations will go to partition 3.
+ *
+ * To calculate these weights, assume we know the number of allocations
+ * requested for each partition in the past interval. We can use this to
+ * calculate weights for the following interval, aiming to allocate the
+ * same (fair share) number of buffers from each partition.
+ *
+ * Note: This is based on number of allocations "originating" in a given
+ * partition. If an allocation is requested in a partition A, it's counted
+ * as allocation for A, even if it gets redirected to some other partition.
+ * The patch addes a new counter to track this.
+ *
+ * The main observation is that partitions get divided into two groups,
+ * depending on whether the number allocations is higher or lower than the
+ * target average. But the "total delta" for these two groups is the
+ * same, i.e. sum(abs(allocs - avg_allocs)) is the same. Therefore, the
+ * task is to "distribute" the excess allocations between the partitions
+ * with not enough allocations.
+ *
+ * Partitions with (nallocs < avg_nallocs) don't redirect any allocations.
+ *
+ * Partitions with (nallocs > avg_nallocs) redirect the extra allocations,
+ * with each target allocation getting a proportional part (with respect
+ * to the total delta).
+ *
+ * XXX In principle we might do without the new "requestedAllocs" counter,
+ * but we'd need to solve the matrix equation Ax=b, with [A,b] known
+ * (weights and allocs), and calculate x (requested allocs). But it's not
+ * quite clear this'd always have a solution.
+ */
+void
+StrategySyncBalance(void)
+{
+	/* snapshot of allocs for partitions */
+	uint32	allocs[MAX_BUFFER_PARTITIONS];
+
+	uint32	total_allocs = 0,	/* total number of allocations */
+			avg_allocs,			/* average allocations (per partition) */
+			delta_allocs = 0;	/* sum of allocs above average */
+
+	/*
+	 * Collect the number of allocations requested in the past interval.
+	 * While at it, reset the counter to start the new interval.
+	 *
+	 * We lock the partitions one by one, so this is not exactly consistent
+	 * snapshot of the counts, and the resets happen before we update the
+	 * weights too. But we're only looking for heuristics anyway, so this
+	 * should be good enough.
+	 *
+	 * A similar issue applies to the counter reset - we haven't updated
+	 * the weights yet. Should be fine, we'll simply consider this in the
+	 * next balancing cycle.
+	 *
+	 * XXX Does this need to worry about the completePasses too?
+	 */
+	for (int i = 0; i < StrategyControl->num_partitions; i++)
+	{
+		ClockSweep *sweep = &StrategyControl->sweeps[i];
+
+		/* no need for a spinlock */
+		allocs[i] = pg_atomic_exchange_u32(&sweep->numRequestedAllocs, 0);
+
+		/* add the allocs to running total */
+		pg_atomic_fetch_add_u64(&sweep->numTotalRequestedAllocs, allocs[i]);
+
+		total_allocs += allocs[i];
+	}
+
+	/*
+	 * Calculate the "fair share" of allocations per partition.
+	 *
+	 * XXX The last partition could be smaller, in which case it should be
+	 * expected to handle fewer allocations. So this should be a weighted
+	 * average. But for now a simple average is good enough.
+	 */
+	avg_allocs = (total_allocs / StrategyControl->num_partitions);
+
+	/*
+	 * Calculate the "delta" from balanced state, i.e. how many allocations
+	 * we'd need to redistribute.
+	 */
+	for (int i = 0; i < StrategyControl->num_partitions; i++)
+	{
+		if (allocs[i] > avg_allocs)
+			delta_allocs += (allocs[i] - avg_allocs);
+	}
+
+	/*
+	 * Skip the rebalancing when there's not enough activity. In this case
+	 * we just keep the current weights.
+	 *
+	 * XXX The threshold of 100 allocation is pretty arbitrary.
+	 *
+	 * XXX Maybe a better strategy would be to slowly return to the default
+	 * weights, with each partition allocation only from itself?
+	 *
+	 * XXX Maybe we shouldn't even reset the counters in this case? But it
+	 * should not matter, if the activity is low.
+	 */
+	if (avg_allocs < 100)
+	{
+		elog(LOG, "rebalance skipped: not enough allocations (allocs: %u)",
+			 avg_allocs);
+		return;
+	}
+
+	/*
+	 * Likewise, skip rebalancing if the misbalance is not significant. We
+	 * consider it acceptable if the amount of allocations we'd need to
+	 * redistribute is less than 10% of the average.
+	 *
+	 * XXX Again, these threshold are rather arbitrary.
+	 */
+	if (delta_allocs < (avg_allocs * 0.1))
+	{
+		elog(LOG, "rebalance skipped: delta within limit (delta: %u, threshold: %u)",
+			 delta_allocs, (uint32) (avg_allocs * 0.1));
+		return;
+	}
+
+	/*
+	 * Got to do the rebalancing. Go through the partitions, and for each
+	 * partition decide if it gets to redirect or receive allocations.
+	 *
+	 * If a partition has fewer than average allocations, it won't redirect
+	 * any allocations to other partitions. So it only has a single non-zero
+	 * weight, and that's for itself.
+	 *
+	 * If a parttion has more than average allocations, it won't receive
+	 * any redirected allocations. Instead, the excess allocations are
+	 * redirected to the other partitions.
+	 *
+	 * The redistribution is "proportional" - if the excess allocations of
+	 * a partition represent 10% of the "delta", then each partition that
+	 * needs more allocations will get 10% of the gap from this one.
+	 *
+	 * XXX We should add hysteresis, to "dampen" the changes, and make
+	 * sure it does not oscillate too much.
+	 *
+	 * XXX Ideally, the alternative partitions to use first would be the
+	 * other partitions for the same node (if any).
+	 */
+	for (int i = 0; i < StrategyControl->num_partitions; i++)
+	{
+		ClockSweep *sweep = &StrategyControl->sweeps[i];
+		uint8		balance[MAX_BUFFER_PARTITIONS];
+
+		/* lock, we're going to modify the balance weights */
+		SpinLockAcquire(&sweep->clock_sweep_lock);
+
+		/* reset the weights to start from scratch */
+		memset(balance, 0, sizeof(uint8) * MAX_BUFFER_PARTITIONS);
+
+		/* does this partition has fewer or more than avg_allocs? */
+		if (allocs[i] < avg_allocs)
+		{
+			/* fewer - don't redirect any allocations elsewhere */
+			balance[i] = 100;
+		}
+		else
+		{
+			/*
+			 * more - redistribute the excess allocations
+			 *
+			 * Each "target" partition (with less than avg_allocs) should get
+			 * a fraction proportional to (excess/delta) from this one.
+			 */
+
+			/* fraction of the "total" delta */
+			double	delta_frac = (allocs[i] - avg_allocs) * 1.0 / delta_allocs;
+
+			/* keep just enough allocations to meet the target */
+			balance[i] = (100.0 * avg_allocs / allocs[i]);
+
+			/* redirect the extra allocations */
+			for (int j = 0; j < StrategyControl->num_partitions; j++)
+			{
+				/* How many allocations to receive from i-th partition? */
+				uint32	receive_allocs = delta_frac * (avg_allocs - allocs[j]);
+
+				/* ignore partitions that don't need additional allocations */
+				if (allocs[j] > avg_allocs)
+					continue;
+
+				/* fraction to redirect */
+				balance[j] = (100.0 * receive_allocs / allocs[i]) + 0.5;
+			}
+		}
+
+		/* combine the old and new weights (hysteresis) */
+		for (int j = 0; j < MAX_BUFFER_PARTITIONS; j++)
+		{
+			sweep->balance[j]
+				= CLOCKSWEEP_HISTORY_COEFF * sweep->balance[j] +
+				  (1.0 - CLOCKSWEEP_HISTORY_COEFF) * balance[j];
+		}
+
+		SpinLockRelease(&sweep->clock_sweep_lock);
+	}
+}
+
 /*
  * StrategySyncStart -- prepare for sync of all partitions
  *
@@ -606,6 +925,7 @@ StrategySyncPrepare(int *num_parts, uint32 *num_buf_alloc)
 	{
 		ClockSweep *sweep = &StrategyControl->sweeps[i];
 
+		/* XXX we don't need the spinlock to read atomics, no? */
 		SpinLockAcquire(&sweep->clock_sweep_lock);
 		if (num_buf_alloc)
 		{
@@ -852,7 +1172,21 @@ StrategyInitialize(bool init)
 			/* Clear statistics */
 			StrategyControl->sweeps[i].completePasses = 0;
 			pg_atomic_init_u32(&StrategyControl->sweeps[i].numBufferAllocs, 0);
+			pg_atomic_init_u32(&StrategyControl->sweeps[i].numRequestedAllocs, 0);
 			pg_atomic_init_u64(&StrategyControl->sweeps[i].numTotalAllocs, 0);
+			pg_atomic_init_u64(&StrategyControl->sweeps[i].numTotalRequestedAllocs, 0);
+
+			/*
+			 * Initialize the weights - start by allocating 100% buffers from
+			 * the current node / partition.
+			 */
+			for (int j = 0; j < MAX_BUFFER_PARTITIONS; j++)
+			{
+				if (i == j)
+					StrategyControl->sweeps[i].balance[i] = 100;
+				else
+					StrategyControl->sweeps[i].balance[j] = 0;
+			}
 		}
 
 		/* No pending notification */
@@ -1242,7 +1576,9 @@ StrategyRejectBuffer(BufferAccessStrategy strategy, BufferDesc *buf, bool from_r
 void
 FreelistPartitionGetInfo(int idx, uint64 *consumed, uint64 *remain, uint64 *actually_free,
 						 uint32 *complete_passes, uint32 *next_victim_buffer,
-						 uint64 *buffer_total_allocs, uint32 *buffer_allocs)
+						 uint64 *buffer_total_allocs, uint32 *buffer_allocs,
+						 uint64 *buffer_total_req_allocs, uint32 *buffer_req_allocs,
+						 int **weights)
 {
 	BufferStrategyFreelist *freelist;
 	ClockSweep *sweep;
@@ -1288,11 +1624,21 @@ FreelistPartitionGetInfo(int idx, uint64 *consumed, uint64 *remain, uint64 *actu
 
 	/* get the clocksweep stats too */
 	*complete_passes = sweep->completePasses;
+
+	/* calculate the actual buffer ID */
 	*next_victim_buffer = pg_atomic_read_u32(&sweep->nextVictimBuffer);
+	*next_victim_buffer = sweep->firstBuffer + (*next_victim_buffer % sweep->numBuffers);
 
-	*buffer_allocs = pg_atomic_read_u32(&sweep->numBufferAllocs);
 	*buffer_total_allocs = pg_atomic_read_u64(&sweep->numTotalAllocs);
+	*buffer_allocs = pg_atomic_read_u32(&sweep->numBufferAllocs);
 
-	/* calculate the actual buffer ID */
-	*next_victim_buffer = sweep->firstBuffer + (*next_victim_buffer % sweep->numBuffers);
+	*buffer_total_req_allocs = pg_atomic_read_u64(&sweep->numTotalRequestedAllocs);
+	*buffer_req_allocs = pg_atomic_read_u32(&sweep->numRequestedAllocs);
+
+	/* return the weights in a newly allocated array */
+	*weights = palloc_array(int, StrategyControl->num_partitions);
+	for (int i = 0; i < StrategyControl->num_partitions; i++)
+	{
+		(*weights)[i] = (int) sweep->balance[i];
+	}
 }
diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h
index 907b160b4f7..38bd5511048 100644
--- a/src/include/storage/buf_internals.h
+++ b/src/include/storage/buf_internals.h
@@ -449,6 +449,7 @@ extern void StrategyFreeBuffer(BufferDesc *buf);
 extern bool StrategyRejectBuffer(BufferAccessStrategy strategy,
 								 BufferDesc *buf, bool from_ring);
 
+extern void StrategySyncBalance(void);
 extern void StrategySyncPrepare(int *num_parts, uint32 *num_buf_alloc);
 extern int	StrategySyncStart(int partition, uint32 *complete_passes,
 							  int *first_buffer, int *num_buffers);
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index a6795c5fee9..e1729f0ee14 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -353,7 +353,10 @@ extern void FreelistPartitionGetInfo(int idx,
 									 uint32 *complete_passes,
 									 uint32 *next_victim_buffer,
 									 uint64 *buffer_total_allocs,
-									 uint32 *buffer_allocs);
+									 uint32 *buffer_allocs,
+									 uint64 *buffer_total_req_allocs,
+									 uint32 *buffer_req_allocs,
+									 int **weights);
 
 /* inline functions */
 
-- 
2.50.1

From c2dbd991bf2720eb6d9295bffd61744f525a19c9 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Sun, 8 Jun 2025 18:53:12 +0200
Subject: [PATCH v20250807 07/11] NUMA: clockweep partitioning

Note: This needs some sort of "balancing" when one of the partitions is
much busier than the rest (e.g. because there's a single backend consuming
a lot of buffers from it).
---
 .../pg_buffercache--1.6--1.7.sql              |   8 +-
 contrib/pg_buffercache/pg_buffercache_pages.c |  32 +-
 src/backend/storage/buffer/bufmgr.c           | 476 ++++++++++--------
 src/backend/storage/buffer/freelist.c         | 238 +++++++--
 src/include/storage/buf_internals.h           |   4 +-
 src/include/storage/bufmgr.h                  |   6 +-
 src/tools/pgindent/typedefs.list              |   1 +
 7 files changed, 504 insertions(+), 261 deletions(-)

diff --git a/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql b/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
index 95fd2d2a226..999bb2128f0 100644
--- a/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
+++ b/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
@@ -21,7 +21,13 @@ CREATE VIEW pg_buffercache_partitions AS
 	 -- freelists
 	 list_consumed bigint,		-- buffers consumed from a freelist
 	 list_remain bigint,		-- buffers left in a freelist
-	 list_free bigint);			-- number of free buffers
+	 list_free bigint,			-- number of free buffers
+
+	 -- clocksweep counters
+	 num_passes bigint,			-- clocksweep passes
+	 next_buffer integer,		-- next victim buffer for clocksweep
+	 total_allocs bigint,		-- handled allocs (running total)
+	 num_allocs bigint);		-- handled allocs (current cycle)
 
index 6d734464a22..7ca075e6164 100644
 
 #define NUM_BUFFERCACHE_NUMA_ELEM	3
-#define NUM_BUFFERCACHE_PARTITIONS_ELEM	8
+#define NUM_BUFFERCACHE_PARTITIONS_ELEM	12
 
 PG_MODULE_MAGIC_EXT(
 					.name = "pg_buffercache",
@@ -818,6 +818,14 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 						   INT8OID, -1, 0);
 		TupleDescInitEntry(tupledesc, (AttrNumber) 8, "list_free",
 						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 9, "num_passes",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 10, "next_buffer",
+						   INT4OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 11, "total_allocs",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 12, "num_allocs",
+						   INT8OID, -1, 0);

@@ -843,6 +851,12 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 					buffers_remain,
 					buffers_free;
 
+		uint64		buffer_total_allocs;
+
+		uint32		complete_passes,
+					next_victim_buffer,
+					buffer_allocs;
+
 		Datum		values[NUM_BUFFERCACHE_PARTITIONS_ELEM];
 		bool		nulls[NUM_BUFFERCACHE_PARTITIONS_ELEM];
 
@@ -850,7 +864,9 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 						   &first_buffer, &last_buffer);
 
 		FreelistPartitionGetInfo(i, &buffers_consumed, &buffers_remain,
-								 &buffers_free);
+								 &buffers_free,
+								 &complete_passes, &next_victim_buffer,
+								 &buffer_total_allocs, &buffer_allocs);
 
 		values[0] = Int32GetDatum(i);
 		nulls[0] = false;
@@ -876,6 +892,18 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		values[7] = Int64GetDatum(buffers_free);
 		nulls[7] = false;
 
+		values[8] = Int64GetDatum(complete_passes);
+		nulls[8] = false;
+
+		values[9] = Int32GetDatum(next_victim_buffer);
+		nulls[9] = false;
+
+		values[10] = Int64GetDatum(buffer_total_allocs);
+		nulls[10] = false;
+
+		values[11] = Int64GetDatum(buffer_allocs);
+		nulls[11] = false;
+
 		/* Build and return the tuple. */
 		tuple = heap_form_tuple((TupleDesc) funcctx->user_fctx, values, nulls);
 		result = HeapTupleGetDatum(tuple);
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index bd50535385f..7a8c45ac59c 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3584,6 +3584,23 @@ BufferSync(int flags)
 	TRACE_POSTGRESQL_BUFFER_SYNC_DONE(NBuffers, num_written, num_to_scan);
 }
 
+/*
@@ -3599,55 +3616,24 @@ bool
 
-	/*

-	/*

@@ -3660,223 +3646,285 @@ BgBufferSync(WritebackContext *wb_context)
 	}
 
 	/*

 
 
-	/*
 
-	/*
 
-	/*

-	/*
 
-	/*

 

 
 
 #ifdef BGW_DEBUG
 
-	/*
 
 #ifdef BGW_DEBUG

 /*
diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index 5a63dad7f2c..17988b4fd53 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -52,17 +52,27 @@ typedef struct BufferStrategyFreelist
 }			BufferStrategyFreelist;
 
 /*
 
 	/*
 
@@ -73,6 +83,19 @@ typedef struct
 	uint32		completePasses; /* Complete cycles of the clock sweep */
 	pg_atomic_uint32 numBufferAllocs;	/* Buffers allocated since last reset */
 
+	/* running total of allocs */
+	pg_atomic_uint64 numTotalAllocs;
+
+} ClockSweep;
+
+/*
+ * The shared freelist control information.
+ */
+typedef struct
+{
+	/* Spinlock: protects the values below */
+	slock_t		buffer_strategy_lock;
+
 	/*
 	 * Bgworker process to be notified upon activity or -1 if none. See
 	 * StrategyNotifyBgWriter.
@@ -88,6 +111,9 @@ typedef struct
 	int			num_partitions;
 	int			num_partitions_per_node;

@@ -127,6 +153,7 @@ static BufferDesc *GetBufferFromRing(BufferAccessStrategy strategy,
 									 uint32 *buf_state);
 static void AddBufferToRing(BufferAccessStrategy strategy,
 							BufferDesc *buf);
+static ClockSweep *ChooseClockSweep(void);
 
 /*
  * ClockSweepTick - Helper routine for StrategyGetBuffer()
@@ -138,6 +165,7 @@ static inline uint32
 ClockSweepTick(void)
 {
 	uint32		victim;
+	ClockSweep *sweep = ChooseClockSweep();
 
 	/*
 	 * Atomically move hand ahead one buffer - if there's several processes
@@ -145,14 +173,14 @@ ClockSweepTick(void)
 	 * apparent order.
 	 */
 	victim =
-		pg_atomic_fetch_add_u32(&StrategyControl->nextVictimBuffer, 1);
+		pg_atomic_fetch_add_u32(&sweep->nextVictimBuffer, 1);

 
 		/*
@@ -178,19 +206,23 @@ ClockSweepTick(void)

 
 static int
@@ -247,6 +279,28 @@ calculate_partition_index()
 	return index;
 }
 
+/*
+ * ChooseClockSweep
@@ -363,7 +417,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 
 	/*
@@ -434,13 +488,17 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
@@ -522,6 +580,46 @@ StrategyFreeBuffer(BufferDesc *buf)
 
+/*
+			uint32	allocs = pg_atomic_exchange_u32(&sweep->numBufferAllocs, 0);
+			/* include the count in the running total */
+			pg_atomic_fetch_add_u64(&sweep->numTotalAllocs, allocs);
+			*num_buf_alloc += allocs;
@@ -529,37 +627,44 @@ StrategyFreeBuffer(BufferDesc *buf)

 	if (complete_passes)
 	{
 
 		/*
 
-	if (num_buf_alloc)
-	{
 
 /*
@@ -647,6 +752,10 @@ StrategyShmemSize(void)
 	size = add_size(size, MAXALIGN(mul_size(sizeof(BufferStrategyFreelist),
 											num_partitions)));

@@ -665,6 +774,7 @@ StrategyInitialize(bool init)
 	int			num_nodes;
 	int			num_partitions;
 	int			num_partitions_per_node;
+	char	   *ptr;
 
 	/* */
@@ -692,7 +802,8 @@ StrategyInitialize(bool init)
 
 	if (!found)
@@ -707,12 +818,42 @@ StrategyInitialize(bool init)

 
-		/* Clear statistics */
+			pg_atomic_init_u64(&StrategyControl->sweeps[i].numTotalAllocs, 0);
+		}
 
 		/* No pending notification */
 		StrategyControl->bgwprocno = -1;
@@ -760,7 +901,6 @@ StrategyInitialize(bool init)
 				buf->freeNext = freelist->firstFreeBuffer;
 				freelist->firstFreeBuffer = i;
 			}
-
 		}
 	}
 	else
@@ -1100,9 +1240,12 @@ StrategyRejectBuffer(BufferAccessStrategy strategy, BufferDesc *buf, bool from_r
 }
 
 void
-FreelistPartitionGetInfo(int idx, uint64 *consumed, uint64 *remain, uint64 *actually_free)
+FreelistPartitionGetInfo(int idx, uint64 *consumed, uint64 *remain, uint64 *actually_free,
+						 uint32 *complete_passes, uint32 *next_victim_buffer,
+						 uint64 *buffer_total_allocs, uint32 *buffer_allocs)
 {
 	BufferStrategyFreelist *freelist;
+	ClockSweep *sweep;
 	int			cur;
 
 	/* stats */
@@ -1112,6 +1255,7 @@ FreelistPartitionGetInfo(int idx, uint64 *consumed, uint64 *remain, uint64 *actu
 	Assert((idx >= 0) && (idx < StrategyControl->num_partitions));

 	/* stat */
 	SpinLockAcquire(&freelist->freelist_lock);
@@ -1141,4 +1285,14 @@ FreelistPartitionGetInfo(int idx, uint64 *consumed, uint64 *remain, uint64 *actu
 
 	*remain = cnt_remain;
+	*buffer_total_allocs = pg_atomic_read_u64(&sweep->numTotalAllocs);
+	/* calculate the actual buffer ID */

 extern Size StrategyShmemSize(void);
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index df127274190..a6795c5fee9 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -349,7 +349,11 @@ extern int	GetAccessStrategyPinLimit(BufferAccessStrategy strategy);
 extern void FreeAccessStrategy(BufferAccessStrategy strategy);
 extern void FreelistPartitionGetInfo(int idx,
 									 uint64 *consumed, uint64 *remain,
-									 uint64 *actually_free);
+									 uint64 *actually_free,
+									 uint32 *complete_passes,
+									 uint32 *next_victim_buffer,
+									 uint64 *buffer_total_allocs,
+									 uint32 *buffer_allocs);
 
 /* inline functions */
 
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 03ca3b7c8bc..8540d537a3e 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -427,6 +427,7 @@ ClientCertName
 ClientConnectionInfo
 ClientData
 ClientSocket
+ClockSweep
 ClonePtrType
 ClosePortalStmt
 ClosePtrType
-- 
2.50.1

From a505c4e23d60d5aac911c24391444ba1b8320bf0 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Thu, 22 May 2025 18:38:41 +0200
Subject: [PATCH v20250807 06/11] NUMA: partition buffer freelist

Note: This is (probably) more important than partitioning ProcArray.
---
 .../pg_buffercache--1.6--1.7.sql              |   7 +-
 contrib/pg_buffercache/pg_buffercache_pages.c |  24 +-
 src/backend/storage/buffer/freelist.c         | 349 ++++++++++++++++--
 src/backend/utils/init/globals.c              |   1 +
 src/backend/utils/misc/guc_tables.c           |  10 +
 src/include/miscadmin.h                       |   1 +
 src/include/storage/bufmgr.h                  |   4 +-
 7 files changed, 366 insertions(+), 30 deletions(-)

diff --git a/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql b/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
index fb9003c011e..95fd2d2a226 100644
--- a/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
+++ b/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
@@ -16,7 +16,12 @@ CREATE VIEW pg_buffercache_partitions AS
 	 numa_node integer,			-- NUMA node of the partitioon
 	 num_buffers integer,		-- number of buffers in the partition
 	 first_buffer integer,		-- first buffer of partition
-	 last_buffer integer);		-- last buffer of partition
+	 last_buffer integer,		-- last buffer of partition
+
+	 -- freelists
+	 list_consumed bigint,		-- buffers consumed from a freelist
+	 list_remain bigint,		-- buffers left in a freelist
+	 list_free bigint);			-- number of free buffers
 
index 8baa7c7b543..6d734464a22 100644

 PG_MODULE_MAGIC_EXT(
 					.name = "pg_buffercache",
@@ -812,6 +812,12 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 						   INT4OID, -1, 0);
 		TupleDescInitEntry(tupledesc, (AttrNumber) 5, "last_buffer",
 						   INT4OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 6, "list_consumed",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 7, "list_remain",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 8, "list_free",
+						   INT8OID, -1, 0);

 

 
 
index e046526c149..5a63dad7f2c 100644
 
+#include <sched.h>

+/*
 
 /*
 
@@ -51,8 +76,19 @@ typedef struct
 
 /* Pointers to shared state */
@@ -157,6 +193,88 @@ ClockSweepTick(void)
 	return victim;
 }
 
+static int
@@ -168,10 +286,13 @@ ClockSweepTick(void)
 
 /*
@@ -193,6 +314,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 	int			bgwprocno;
 	int			trycounter;
 	uint32		local_buf_state;	/* to avoid repeated (de-)referencing */
+	BufferStrategyFreelist *freelist;
 
 	*from_ring = false;
 
@@ -259,31 +381,35 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r

 

 			/*
 			 * If the buffer is pinned or has a nonzero usage_count, we cannot
@@ -305,7 +431,17 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 		}
 	}
 
@@ -356,7 +492,22 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 
 	/*
@@ -364,11 +515,11 @@ StrategyFreeBuffer(BufferDesc *buf)

 /*
@@ -432,6 +583,42 @@ StrategyNotifyBgWriter(int bgwprocno)
 	SpinLockRelease(&StrategyControl->buffer_strategy_lock);
 }

 /*
  * StrategyShmemSize
@@ -445,12 +632,20 @@ Size
 StrategyShmemSize(void)
 {
 	Size		size = 0;
+	int			num_partitions;
+	int			num_nodes;
+
+	BufferPartitionParams(&num_partitions, &num_nodes);

 
 	return size;
 }
@@ -467,6 +662,18 @@ StrategyInitialize(bool init)
 {
 	bool		found;
 
+	int			num_nodes;
+	int			num_partitions;
@@ -484,7 +691,8 @@ StrategyInitialize(bool init)
 
 	if (!found)
@@ -494,13 +702,10 @@ StrategyInitialize(bool init)
 		 */
 		Assert(init);

-		/*
 
 		/* Initialize the clock sweep pointer */
 		pg_atomic_init_u32(&StrategyControl->nextVictimBuffer, 0);
@@ -511,6 +716,52 @@ StrategyInitialize(bool init)
 
@@ -847,3 +1098,47 @@ StrategyRejectBuffer(BufferAccessStrategy strategy, BufferDesc *buf, bool from_r
 
 	return true;
 }
+
+void

+	{

 
 
 /* inline functions */
 
-- 
2.50.1

From e0fa771531b92dc096fd5a1580ded3224524fa65 Mon Sep 17 00:00:00 2001
From: Andres Freund <[email-masked]>
Date: Mon, 14 Oct 2024 14:10:13 -0400
Subject: [PATCH v20250807 05/11] freelist: Don't track tail of a freelist

 
 
 	/*

From 7fec3b33652e92903a20276536477df33be722c8 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Thu, 22 May 2025 18:27:06 +0200
Subject: [PATCH v20250807 04/11] NUMA: localalloc

 
 /* NUMA stuff */
 
index 65d8cbfaed5..d986a1d18cf 100644
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#endif
+
 
 /*
 
+	{

From bbe6427b4e0e871bcb7b2cc4ce11ad8aba62799c Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Mon, 28 Jul 2025 14:01:37 +0200
Subject: [PATCH v20250807 03/11] NUMA: interleaving buffers

The effect is similar to

     numactl --interleave=all

Notes:

 .../pg_buffercache--1.6--1.7.sql              |  26 +
 11 files changed, 775 insertions(+), 11 deletions(-)

index 00000000000..fb9003c011e
@@ -0,0 +1,26 @@
+	(partition integer,			-- partition index
+	 numa_node integer,			-- NUMA node of the partitioon
+	 num_buffers integer,		-- number of buffers in the partition
+	 first_buffer integer,		-- first buffer of partition
+	 last_buffer integer);		-- last buffer of partition

 PG_MODULE_MAGIC_EXT(

 	PG_RETURN_DATUM(result);
 }
+
+/*
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+

 
 /*
  * Data Structures:

 
 
+		/*
 
+	/*

+/*

 

+/* NUMA stuff */

+	{

 /* in buf_init.c */

 

 
 /* in localbuf.c */
index e6f2e93b2d6..03ca3b7c8bc 100644

From 730467cfcf1d1b7f0a22f61bd48a37371f80cce9 Mon Sep 17 00:00:00 2001
From: Andres Freund <[email-masked]>
Date: Tue, 27 Jun 2023 14:07:34 -0700
Subject: [PATCH v20250807 02/11] nbtree: Use ReadRecentBuffer()
 in_bt_getroot()

We tend to access the btree root page over and over, when descending the
index. It's quite expensive. Not really specific to NUMA, but it hurts
there much more.

Thomas Munro worked on this.

Discussion: https://www.postgresql.org/message-id/20230627020546.t6z4tntmj7wmjrfh%40awork3.anarazel.de
Discussion: https://www.postgresql.org/message-id/CA%2BhUKGJ8N_DRSB0YioinWjS2ycMpmOLy32mbBqVVztwBvXgyJA%40mail.gmail.com
---
 src/backend/access/nbtree/nbtpage.c   | 21 ++++++++++++++--
 src/backend/access/nbtree/nbtsearch.c | 17 +++++++++----
 src/backend/storage/buffer/bufmgr.c   | 35 +++++----------------------
 src/include/utils/backend_status.h    |  4 +++
 src/include/utils/rel.h               |  4 +++
 5 files changed, 45 insertions(+), 36 deletions(-)

diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c
index c79dd38ee18..3cc26490f06 100644
--- a/src/backend/access/nbtree/nbtpage.c
+++ b/src/backend/access/nbtree/nbtpage.c
@@ -35,6 +35,7 @@
 #include "storage/procarray.h"
 #include "utils/memdebug.h"
 #include "utils/memutils.h"
+#include "utils/rel.h"
 #include "utils/snapmgr.h"
 
 static BTMetaPageData *_bt_getmeta(Relation rel, Buffer metabuf);
@@ -344,7 +345,7 @@ Buffer
 _bt_getroot(Relation rel, Relation heaprel, int access)
 {
 	Buffer		metabuf;
-	Buffer		rootbuf;
+	Buffer		rootbuf = InvalidBuffer;
 	Page		rootpage;
 	BTPageOpaque rootopaque;
 	BlockNumber rootblkno;
@@ -373,7 +374,20 @@ _bt_getroot(Relation rel, Relation heaprel, int access)
 		Assert(rootblkno != P_NONE);
 		rootlevel = metad->btm_fastlevel;
 
-		rootbuf = _bt_getbuf(rel, rootblkno, BT_READ);
+
+		if (BufferIsValid(rel->rd_recent_root))
+		{
+			if (ReadRecentBuffer(rel->rd_locator, MAIN_FORKNUM, rootblkno,
+								 rel->rd_recent_root))
+			{
+				rootbuf = rel->rd_recent_root;
+				_bt_lockbuf(rel, rootbuf, BT_READ);
+				_bt_checkpage(rel, rootbuf);
+			}
+		}
+
+		if (rootbuf == InvalidBuffer)
+			rootbuf = _bt_getbuf(rel, rootblkno, BT_READ);
 		rootpage = BufferGetPage(rootbuf);
 		rootopaque = BTPageGetOpaque(rootpage);
 
@@ -390,6 +404,7 @@ _bt_getroot(Relation rel, Relation heaprel, int access)
 			P_RIGHTMOST(rootopaque))
 		{
 			/* OK, accept cached page as the root */
+			rel->rd_recent_root = rootbuf;
 			return rootbuf;
 		}
 		_bt_relbuf(rel, rootbuf);
@@ -555,6 +570,8 @@ _bt_getroot(Relation rel, Relation heaprel, int access)
 				 rootopaque->btpo_level, rootlevel);
 	}
 
+	rel->rd_recent_root = rootbuf;
+
 	/*
 	 * By here, we have a pin and read lock on the root page, and no lock set
 	 * on the metadata page.  Return the root page's buffer.
diff --git a/src/backend/access/nbtree/nbtsearch.c b/src/backend/access/nbtree/nbtsearch.c
index d69798795b4..e1e73b4aaf3 100644
--- a/src/backend/access/nbtree/nbtsearch.c
+++ b/src/backend/access/nbtree/nbtsearch.c
@@ -168,11 +168,17 @@ _bt_search(Relation rel, Relation heaprel, BTScanInsert key, Buffer *bufP,
 		 * stack entry for this page/level.  If caller ends up splitting a
 		 * page one level down, it usually ends up inserting a new pivot
 		 * tuple/downlink immediately after the location recorded here.
+		 *
+		 * FIXME: Unfortunately this isn't a usable gating condition, as
+		 * vacuum uses BT_READ and needs the stack.
 		 */
-		new_stack = (BTStack) palloc(sizeof(BTStackData));
-		new_stack->bts_blkno = BufferGetBlockNumber(*bufP);
-		new_stack->bts_offset = offnum;
-		new_stack->bts_parent = stack_in;
+		if (false && access == BT_WRITE)
+		{
+			new_stack = (BTStack) palloc(sizeof(BTStackData));
+			new_stack->bts_blkno = BufferGetBlockNumber(*bufP);
+			new_stack->bts_offset = offnum;
+			new_stack->bts_parent = stack_in;
+		}
 
 		/*
 		 * Page level 1 is lowest non-leaf page level prior to leaves.  So, if
@@ -186,7 +192,8 @@ _bt_search(Relation rel, Relation heaprel, BTScanInsert key, Buffer *bufP,
 		*bufP = _bt_relandgetbuf(rel, *bufP, child, page_access);
 
 		/* okay, all set to move down a level */
-		stack_in = new_stack;
+		if (false && access == BT_WRITE)
+			stack_in = new_stack;
 	}
 
 	/*
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 67431208e7f..bd50535385f 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -685,7 +685,6 @@ ReadRecentBuffer(RelFileLocator rlocator, ForkNumber forkNum, BlockNumber blockN
 	BufferDesc *bufHdr;
 	BufferTag	tag;
 	uint32		buf_state;
-	bool		have_private_ref;

@@ -713,38 +712,16 @@ ReadRecentBuffer(RelFileLocator rlocator, ForkNumber forkNum, BlockNumber blockN
 	else
 	{
 		bufHdr = GetBufferDescriptor(recent_buffer - 1);
-		have_private_ref = GetPrivateRefCount(recent_buffer) > 0;
 
-		/*
-		 * Do we already have this buffer pinned with a private reference?  If
-		 * so, it must be valid and it is safe to check the tag without
-		 * locking.  If not, we have to lock the header first and then check.
-		 */
-		if (have_private_ref)
-			buf_state = pg_atomic_read_u32(&bufHdr->state);
-		else
-			buf_state = LockBufHdr(bufHdr);
-
-		if ((buf_state & BM_VALID) && BufferTagsEqual(&tag, &bufHdr->tag))
+		if (!PinBuffer(bufHdr, NULL) ||
+			!BufferTagsEqual(&tag, &bufHdr->tag))
 		{
-			/*
-			 * It's now safe to pin the buffer.  We can't pin first and ask
-			 * questions later, because it might confuse code paths like
-			 * InvalidateBuffer() if we pinned a random non-matching buffer.
-			 */
-			if (have_private_ref)
-				PinBuffer(bufHdr, NULL);	/* bump pin count */
-			else
-				PinBuffer_Locked(bufHdr);	/* pin for first time */
-
-			pgBufferUsage.shared_blks_hit++;
-
-			return true;
+			UnpinBuffer(bufHdr);
+			return false;
 		}
 
-		/* If we locked the header above, now unlock. */
-		if (!have_private_ref)
-			UnlockBufHdr(bufHdr, buf_state);
+		pgBufferUsage.shared_blks_hit++;
+		return true;
 	}
 
 	return false;
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index 3016501ac05..443c492cf34 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -97,6 +97,10 @@ typedef struct PgBackendGSSStatus
  */
 typedef struct PgBackendStatus
 {
+#ifdef pg_attribute_aligned
+	pg_attribute_aligned(PG_CACHE_LINE_SIZE)
+#endif
+
 	/*
 	 * To avoid locking overhead, we use the following protocol: a backend
 	 * increments st_changecount before modifying its entry, and again after
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index b552359915f..7ac2fb1512f 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -25,6 +25,7 @@
 #include "rewrite/prs2lock.h"
 #include "storage/block.h"
 #include "storage/relfilelocator.h"
+#include "storage/buf.h"
 #include "storage/smgr.h"
 #include "utils/relcache.h"
 #include "utils/reltrigger.h"
@@ -159,6 +160,9 @@ typedef struct RelationData
 
 	/* data managed by RelationGetIndexAttrBitmap: */
 	bool		rd_attrsvalid;	/* are bitmaps of attrs valid? */
+
+	Buffer		rd_recent_root;
+
 	Bitmapset  *rd_keyattr;		/* cols that can be ref'd by foreign keys */
 	Bitmapset  *rd_pkattr;		/* cols included in primary key */
 	Bitmapset  *rd_idattr;		/* included in replica identity index */
-- 
2.50.1

From 9097f04069ab604146606a7f6eabe9a2ea78dca0 Mon Sep 17 00:00:00 2001
From: Ubuntu
 <[email-masked]>
Date: Sat, 31 May 2025 00:45:47 +0000
Subject: [PATCH v20250807 01/11] allow pgbench to pin backends/threads

Supported policies are "none" (default), "colocated" (both processes on
the same random core) and "random" (both processes on different cores).
---
 src/bin/pgbench/pgbench.c | 348 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 348 insertions(+)

diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 125f3c7bbbe..95479861177 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -41,6 +41,7 @@
 #include <time.h>
 #include <sys/time.h>
 #include <sys/resource.h>		/* for getrlimit */
+#include <sys/sysinfo.h>
 
 /* For testing, PGBENCH_USE_SELECT can be defined to force use of that code */
 #if defined(HAVE_PPOLL) && !defined(PGBENCH_USE_SELECT)
@@ -300,6 +301,12 @@ static const char *dbName = NULL;
 static char *logfile_prefix = NULL;
 static const char *progname;
 
+#define	CPU_PINNING_NONE		0
+#define	CPU_PINNING_RANDOM		1
+#define	CPU_PINNING_COLOCATED	2
+
+static int	pinning_mode = CPU_PINNING_NONE;
+
 #define WSEP '@'				/* weight separator */
 
 static volatile sig_atomic_t timer_exceeded = false;	/* flag from signal
@@ -639,6 +646,8 @@ typedef struct
 	int64		cnt;			/* client transaction count, for -t; skipped
 								 * and failed transactions are also counted
 								 * here */
+
+	int			cpu;			/* CPU to pin to, -1 = no pinning */
 } CState;
 
 /*
@@ -672,6 +681,8 @@ typedef struct
 
 	StatsData	stats;
 	int64		latency_late;	/* count executed but late transactions */
+
+	int			cpu;			/* CPU to pin to, -1 = no pinning */
 } TState;
 
 /*
@@ -841,6 +852,45 @@ static void add_socket_to_set(socket_set *sa, int fd, int idx);
 static int	wait_on_socket_set(socket_set *sa, int64 usecs);
 static bool socket_has_input(socket_set *sa, int fd, int idx);
 
+/*
+ * random number generator for assigning CPUs to processes/threads
+ *
+ * This is not a simple generator, because we want to keep the outcome
+ * balanced - it's not just about the total number of threads/processes
+ * assigned to each core, because a thread is likely much lighter than a
+ * backend. So we want to distribute each of those uniformly.
+ *
+ * For example, assume we have 2 cores, and want to pin 2 threads and 2
+ * backends. We could pin 2 threads to core 0 and 2 backends to core 1.
+ * But that would not be balanced, because core 1 will have to execute
+ * much more work. Instead, we want to assign 1+1 to each core.
+ *
+ * Another restriction is that we don't want to assign both sides of
+ * a connection to the same core.
+ *
+ * All of that however applies to the 'random' mode. In the 'colocated'
+ * mode, we want to assign both sides of a connection to the same core.
+ * But we still pick the core randomly.
+ */
+typedef struct cpu_generator_state
+{
+	int			ncpus;			/* number of CPUs available */
+	int			nitems;			/* number of items in the queue */
+	int		   *nthreads;		/* number of threads for each CPU */
+	int		   *nclients;		/* number of processes for each CPU */
+	int		   *items;			/* queue of CPUs to pick from */
+}			cpu_generator_state;
+
+static cpu_generator_state cpu_generator_init(int ncpus);
+static void cpu_generator_refill(cpu_generator_state * state);
+static void cpu_generator_reset(cpu_generator_state * state);
+static int	cpu_generator_thread(cpu_generator_state * state);
+static int	cpu_generator_client(cpu_generator_state * state, int thread_cpu);
+static void cpu_generator_print(cpu_generator_state * state);
+static bool cpu_generator_check(cpu_generator_state * state);
+
+static void reset_pinning(TState *threads, int nthreads);
+
 /* callback used to build rows for COPY during data loading */
 typedef void (*initRowMethod) (PQExpBufferData *sql, int64 curr);
 
@@ -959,6 +1009,7 @@ usage(void)
 		   "  --sampling-rate=NUM      fraction of transactions to log (e.g., 0.01 for 1%%)\n"
 		   "  --show-script=NAME       show builtin script code, then exit\n"
 		   "  --verbose-errors         print messages of all errors\n"
+		   "  --pin-cpus MODE          pin threads and backends to CPU (random or colocated)\n"
 		   "\nCommon options:\n"
 		   "  --debug                  print debugging output\n"
 		   "  -d, --dbname=DBNAME      database name to connect to\n"
@@ -6718,6 +6769,7 @@ main(int argc, char **argv)
 		{"verbose-errors", no_argument, NULL, 15},
 		{"exit-on-abort", no_argument, NULL, 16},
 		{"debug", no_argument, NULL, 17},
+		{"pin-cpus", required_argument, NULL, 18},
 		{NULL, 0, NULL, 0}
 	};
 
@@ -7071,6 +7123,15 @@ main(int argc, char **argv)
 			case 17:			/* debug */
 				pg_logging_increase_verbosity();
 				break;
+			case 18:			/* pin CPUs */
+				if (pg_strcasecmp(optarg, "random") == 0)
+					pinning_mode = CPU_PINNING_RANDOM;
+				else if (pg_strcasecmp(optarg, "colocated") == 0)
+					pinning_mode = CPU_PINNING_COLOCATED;
+				else
+					pg_fatal("invalid pinning method, expecting \"random\" or \"colocated\", got: \"%s\"",
+							 optarg);
+				break;
 			default:
 				/* getopt_long already emitted a complaint */
 				pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -7368,6 +7429,96 @@ main(int argc, char **argv)
 		nclients_dealt += thread->nstate;
 	}
 
+	/* reset the CPU assignment, i.e. no pinning by default */
+	reset_pinning(threads, nthreads);
+
+	/* try to assign threads/clients to CPUs */
+	if (pinning_mode != CPU_PINNING_NONE)
+	{
+		int			nprocs = get_nprocs();
+		cpu_generator_state state = cpu_generator_init(nprocs);
+
+retry:
+		/* start from scratch */
+		cpu_generator_reset(&state);
+
+		/* assign CPU to all threads */
+		for (i = 0; i < nthreads; i++)
+		{
+			TState	   *thread = &threads[i];
+
+			thread->cpu = cpu_generator_thread(&state);
+		}
+
+		/*
+		 * assign CPUs to backends, one at a time (for each thread)
+		 *
+		 * This helps to keep it balanced in the 'random' mode.
+		 */
+		while (true)
+		{
+			/* did we find any unassigned backend? */
+			bool		found = false;
+
+			for (i = 0; i < nthreads; i++)
+			{
+				TState	   *thread = &threads[i];
+
+				for (int j = 0; j < thread->nstate; j++)
+				{
+					/* skip backends with already assigned CPUs */
+					if (thread->state[j].cpu != -1)
+						continue;
+
+					if (pinning_mode == CPU_PINNING_RANDOM)
+						thread->state[j].cpu = cpu_generator_client(&state, thread->cpu);
+					else
+					{
+						state.nclients[thread->cpu]++;
+						thread->state[j].cpu = thread->cpu;
+					}
+
+					/* move to the next thread */
+					found = true;
+					break;
+				}
+			}
+
+			/* if we haven't found any unassigned backend, we're done */
+			if (!found)
+				break;
+		}
+
+		/* validate the assignments don't violate any of the restrictions */
+		if ((pinning_mode == CPU_PINNING_RANDOM) &&
+			!cpu_generator_check(&state))
+		{
+			reset_pinning(threads, nthreads);
+			goto retry;
+		}
+
+		/* print info about CPU assignments */
+		printf("============================\n");
+
+		cpu_generator_print(&state);
+
+		printf("----------------------------\n");
+
+		for (i = 0; i < nthreads; i++)
+		{
+			TState	   *thread = &threads[i];
+
+			printf("thread %d CPU %d\n", i, thread->cpu);
+
+			for (int j = 0; j < thread->nstate; j++)
+			{
+				printf("  client %d CPU %d\n", j, thread->state[j].cpu);
+			}
+		}
+
+		printf("============================\n");
+	}
+
 	/* all clients must be assigned to a thread */
 	Assert(nclients_dealt == nclients);
 
@@ -7519,6 +7670,39 @@ threadRun(void *arg)
 		}
 	}
 
+	/* pin the thread and backends for it's connections to the same CPU */
+	if (pinning_mode != CPU_PINNING_NONE)
+	{
+		cpu_set_t  *cpusetp;
+		size_t		cpusetsize;
+		size_t		nprocs = get_nprocs();
+
+		cpusetp = CPU_ALLOC(nprocs);
+		cpusetsize = CPU_ALLOC_SIZE(nprocs);
+
+		CPU_ZERO_S(cpusetsize, cpusetp);
+		CPU_SET_S(thread->cpu, cpusetsize, cpusetp);
+
+		/* thread 0 dess not have an actual pthread */
+		if (thread->thread)
+			pthread_setaffinity_np(thread->thread, cpusetsize, cpusetp);
+		else
+			sched_setaffinity(getpid(), cpusetsize, cpusetp);
+
+		/* determine PID of the backend, pin it to the same CPU */
+		for (int i = 0; i < nstate; i++)
+		{
+			pid_t		pid = PQbackendPID(state[i].con);
+
+			CPU_ZERO_S(cpusetsize, cpusetp);
+			CPU_SET_S(state[i].cpu, cpusetsize, cpusetp);
+
+			sched_setaffinity(pid, cpusetsize, cpusetp);
+		}
+
+		CPU_FREE(cpusetp);
+	}
+
 	/* GO */
 	THREAD_BARRIER_WAIT(&barrier);
 
@@ -7992,3 +8176,167 @@ socket_has_input(socket_set *sa, int fd, int idx)
 }
 
 #endif							/* POLL_USING_SELECT */
+
+static cpu_generator_state
+cpu_generator_init(int ncpus)
+{
+	struct timeval tv;
+
+	cpu_generator_state state;
+
+	state.ncpus = ncpus;
+
+	state.nthreads = pg_malloc(sizeof(int) * ncpus);
+	state.nclients = pg_malloc(sizeof(int) * ncpus);
+
+	state.nitems = ncpus;
+	state.items = pg_malloc(sizeof(int) * ncpus);
+	for (int i = 0; i < ncpus; i++)
+	{
+		state.nthreads[i] = 0;
+		state.nclients[i] = 0;
+		state.items[i] = i;
+	}
+
+	gettimeofday(&tv, NULL);
+	srand48(tv.tv_usec);
+
+	return state;
+}
+
+static void
+cpu_generator_refill(cpu_generator_state * state)
+{
+	struct timeval tv;
+
+	state->items = pg_realloc(state->items,
+							  (state->nitems + state->ncpus) * sizeof(int));
+
+	gettimeofday(&tv, NULL);
+
+	srand48(tv.tv_usec);
+
+	for (int i = 0; i < state->ncpus; i++)
+		state->items[state->nitems++] = i;
+}
+
+static void
+cpu_generator_reset(cpu_generator_state * state)
+{
+	state->nitems = 0;
+	cpu_generator_refill(state);
+
+	for (int i = 0; i < state->ncpus; i++)
+	{
+		state->nthreads[i] = 0;
+		state->nclients[i] = 0;
+	}
+}
+
+static int
+cpu_generator_thread(cpu_generator_state * state)
+{
+	if (state->nitems == 0)
+		cpu_generator_refill(state);
+
+	while (true)
+	{
+		int			idx = lrand48() % state->nitems;
+		int			cpu = state->items[idx];
+
+		state->items[idx] = state->items[state->nitems - 1];
+		state->nitems--;
+
+		state->nthreads[cpu]++;
+
+		return cpu;
+	}
+}
+
+static int
+cpu_generator_client(cpu_generator_state * state, int thread_cpu)
+{
+	int			min_clients;
+	bool		has_valid_cpus = false;
+
+	for (int i = 0; i < state->nitems; i++)
+	{
+		if (state->items[i] != thread_cpu)
+		{
+			has_valid_cpus = true;
+			break;
+		}
+	}
+
+	if (!has_valid_cpus)
+		cpu_generator_refill(state);
+
+	min_clients = INT_MAX;
+	for (int i = 0; i < state->nitems; i++)
+	{
+		if (state->items[i] == thread_cpu)
+			continue;
+
+		min_clients = Min(min_clients, state->nclients[state->items[i]]);
+	}
+
+	while (true)
+	{
+		int			idx = lrand48() % state->nitems;
+		int			cpu = state->items[idx];
+
+		if (cpu == thread_cpu)
+			continue;
+
+		if (state->nclients[cpu] != min_clients)
+			continue;
+
+		state->items[idx] = state->items[state->nitems - 1];
+		state->nitems--;
+
+		state->nclients[cpu]++;
+
+		return cpu;
+	}
+}
+
+static void
+cpu_generator_print(cpu_generator_state * state)
+{
+	for (int i = 0; i < state->ncpus; i++)
+	{
+		printf("CPU %d threads %d clients %d\n", i, state->nthreads[i], state->nclients[i]);
+	}
+}
+
+static bool
+cpu_generator_check(cpu_generator_state * state)
+{
+	int			min_count = INT_MAX,
+				max_count = 0;
+
+	for (int i = 0; i < state->ncpus; i++)
+	{
+		min_count = Min(min_count, state->nthreads[i] + state->nclients[i]);
+		max_count = Max(max_count, state->nthreads[i] + state->nclients[i]);
+	}
+
+	return (max_count - min_count <= 1);
+}
+
+static void
+reset_pinning(TState *threads, int nthreads)
+{
+	/* reset the CPU assignment, i.e. no pinning by default */
+	for (int i = 0; i < nthreads; i++)
+	{
+		TState	   *thread = &threads[i];
+
+		thread->cpu = -1;
+
+		for (int j = 0; j < thread->nstate; j++)
+		{
+			thread->state[j].cpu = -1;
+		}
+	}
+}
-- 
2.50.1

---

From: Tomas Vondra <[email-masked]>
Date: Thu, 7 Aug 2025 11:36:10 +0200

> Here's a slightly improved version of the patch series.

Ah, I made a mistake when generating the patches. The 0001 and 0002
patches are not part of the NUMA stuff, it's just something related to
benchmarking (addressing unrelated bottlenecks etc.). The actual NUMA
patches start with 0003.

Also, 0007, 0008 and 0009 should ultimately be collapsed into a single
patch. It's all about the clocksweep partitioning, I only kept those
separate to make it easier to see the changes and review.

regards

-- 
Tomas Vondra

---

From: Andres Freund <[email-masked]>
Date: Fri, 8 Aug 2025 20:25:40 -0400

Hi,

> assumes each core / piece of RAM is assigned to a particular NUMA node.

There are systems in which some NUMA nodes do *not* contain any CPUs. E.g. if
you attach memory via a CXL/PCIe add-in card, rather than via the CPUs memory
controller. In that case numactl -H (and obviously also the libnuma APIs) will
report that the numa node is not associated with any CPU.

I don't currently have live access to such a system, but this PR piece happens
to have numactl -H output:
https://lenovopress.lenovo.com/lp2184-implementing-cxl-memory-on-linux-on-thinksystem-v4-servers
> numactl -H
> available: 4 nodes (0-3)
> node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143
> node 0 size: 1031904 MB
> node 0 free: 1025554 MB
> node 1 cpus: 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191
> node 1 size: 1032105 MB
> node 1 free: 1024244 MB
> node 2 cpus:
> node 2 size: 262144 MB
> node 2 free: 262143 MB
> node 3 cpus:
> node 3 size: 262144 MB
> node 3 free: 262142 MB
> node distances:
> node   0   1   2   3
>   0:  10  21  14  24
>   1:  21  10  24  14
>   2:  14  24  10  26
>   3:  24  14  26  10

Note that node 2 & 3 don't have associated CPUs (and higher access costs).

I don't think this is common enough to worry about from a performance POV, but
we probably shouldn't crash if we encounter it...

> nodes.

FWIW, you can get a different version of this with AMD Epyc too, if "L3 LLC as
NUMA" is enabled.

> I'm not sure what to do about this (or how getcpu() or libnuma handle this).

I don't immediately see any libnuma functions that would care?

I also am somewhat curious about what getcpu() returns for the current node...

Greetings,

Andres Freund

---

From: Tomas Vondra <[email-masked]>
Date: Tue, 12 Aug 2025 13:04:07 +0200

> we probably shouldn't crash if we encounter it...

Right. I don't think the current patch would crash - I can't test it,
but I don't see why it would crash. In the worst case it'd end up with
partitions that are not ideal. The question is more what would an ideal
partitioning for buffers and PGPROC look like. Any opinions?

For PGPROC, it's simple - it doesn't make sense to allocate partitions
for nodes without CPUs.

For buffers, it probably does not really matter if a node does not have
any CPUs. If a node does not have any CPUs, that does not mean we should
not put any buffers on it. After all, CXL will never have any CPUs (at
least I think that's the case), and not using it for shared buffers
would be a bit strange. Although, it could still be used for page cache.

Maybe it should be "tiered" a bit more? The patch differentiates only
between partitions on "my" NUMA node vs. every other partition. Maybe it
should have more layers?

That'd make the "balancing" harder. But I wanted to make it a smarter
when handling cases with multiple partitions per NUMA node.

> I don't immediately see any libnuma functions that would care?

Not sure what "care" means here. I don't think it's necessarily broken,
it's more about the APIs not making the situation very clear (or
convenient).

How do you determine nodes for a CPU, for example? The closest thing I
see is numa_node_of_cpu(), but that only returns a single node. Or how
would you determine the number of nodes with CPUs (so that we create
PGPROC partitions only for those)? I suppose that requires literally
walking all the nodes.

> I also am somewhat curious about what getcpu() returns for the current node...

It seems it only returns node 0. The cpu changes, but the node does not.

regards

-- 
Tomas Vondra

---

From: Andres Freund <[email-masked]>
Date: Tue, 12 Aug 2025 10:24:15 -0400

Hi,

> would be a bit strange. Although, it could still be used for page cache.

For CXL memory to be really usable, I think we'd need nontrivial additional
work. CXL memory has considerably higher latency and lower throughput. You'd
*never* want things like BufferDescs or such on such nodes. And even the
buffered data itself, you'd want to make sure that frequently used data,
e.g. inner index pages, never end up on it.

Which leads to:

> Maybe it should be "tiered" a bit more?

Yes, for proper CXL support, we'd need a component that explicitly demotes and
promotes pages from "real" memory to CXL memory and the other way round. The
demotion is relatively easy, you'd probably just do it whenever you'd
otherwise throw out a victim buffer. When to promote back is harder...

> The patch differentiates only between partitions on "my" NUMA node vs. every
> other partition. Maybe it should have more layers?

Given the relative unavailability of CXL memory systems, I think just not
crashing is good enough for now...

> it's more about the APIs not making the situation very clear (or

What I mean is that I was looking through the libnuma functions and didn't see

> PGPROC partitions only for those)? I suppose that requires literally

I didn't think of numa_node_of_cpu().

As long as numa_node_of_cpu() returns *something* I think it may be good
enough. Nobody uses an RPi for high-throughput postgres workloads with a lot
of memory. Slightly sub-optimal mappings should really not matter.

I'm kinda wondering if we should deal with such fake numa systems by detecting
them and disabling our numa support.

Greetings,

Andres Freund

---

From: Tomas Vondra <[email-masked]>
Date: Tue, 12 Aug 2025 17:06:50 +0200

> e.g. inner index pages, never end up on it.

OK, let's keep that out of scope for these patches and assume we're
dealing only with local memory. CXL could still be used by the OS for
page cache, of whatever.

What does that mean for the patch, though. Does it need a way to
configure which nodes to use? I argued to leave this to the OS/numactl,
and we'd just use whatever is made available to Postgres. But maybe
we'll need something within Postgres after all?

FWIW there's work needed a actually inherit NUMA info from the OS. Right
now the patches just use all NUMA nodes, indexed by 0 ... (N-1) etc. I
like the "registry" concept I used for buffer/PGPROC partitions, it made
the patches much simpler. Maybe we should use something like that for
NUMA info too. That is, at startup build a record of the NUMA layout,
and use this as source of truth everywhere (instead of using libnuma
from all those places).

> otherwise throw out a victim buffer. When to promote back is harder...

Sounds very much like page cache (but that only works for buffered I/O).

> crashing is good enough for now...

The lowest of bars ;-)

> any that would be affected by having multiple "local" NUMA nodes. But:

My question is a bit of a "reverse" to this. That is, how do we even
find (with libnuma) there are multiple local nodes?

> I didn't think of numa_node_of_cpu().

Yeah. I think most of the libnuma API is designed for each CPU belonging
to single NUMA node. I suppose we'd need to use numa_node_to_cpus() to
build this kind of information ourselves.

> of memory. Slightly sub-optimal mappings should really not matter.

I'm not really concerned about rpi, or the performance on it. I only use
it as an example of system with "weird" NUMA layout.

> them and disabling our numa support.

That'd be an option too, if we can identify such systems. We could do
that while building the "NUMA registry" I mentioned earlier.

regards

-- 
Tomas Vondra

---

From: Andres Freund <[email-masked]>
Date: Wed, 13 Aug 2025 11:16:24 -0400

Hi,

> and then goes back to P1 in a round-robin way. The advantage is it can

I think that's a good plan.

A few comments about the clock sweep patch:

- It'd be easier to review if BgBufferSync() weren't basically re-indented
  wholesale. Maybe you could instead move the relevant code to a helper
  function that's called by BgBufferSync() for each clock?

- I think choosing a clock sweep partition in every tick would likely show up
  in workloads that do a lot of buffer replacement, particularly if buffers
  in the workload often have a high usagecount (and thus more ticks are used).
  Given that your balancing approach "sticks" with a partition for a while,
  could we perhaps only choose the partition after exhausting that budget?

- I don't really understand what

> +	pg_atomic_uint32 numRequestedAllocs;

  is intended for.

  Adding yet another atomic increment for every clock sweep tick seems rather
  expensive...

- I wonder if the balancing budgets being relatively low will be good
  enough. It's not too hard to imagine that this frequent "partition choosing"
  will be bad in buffer access heavy workloads. But it's probably the right
  approach until we've measured it being a problem.

- It'd be interesting to do some very simple evaluation like a single
  pg_prewarm() of a relation that's close to the size of shared buffers and
  verify that we don't end up evicting newly read in buffers.  I think your
  approach should work, but verifying that...

  I wonder if we could make some of this into tests somehow. It's pretty easy
  to break this kind of thing and not notice, as everything just continues to
  work, just a tad slower.

Greetings,

Andres Freund

---

From: Tomas Vondra <[email-masked]>
Date: Wed, 13 Aug 2025 18:36:17 +0200

>   function that's called by BgBufferSync() for each clock?

True, I'll rework it like that.

>   could we perhaps only choose the partition after exhausting that budget?

That should be possible, yes. By "exhausting budget" you mean going
through all the partitions, right?

>   Adding yet another atomic increment for every clock sweep tick seems rather

For the balancing (to calculate the budgets), we need to know the number
of allocation requests for each partition, before some of the requests
got redirected to other partitions. We can't use the number of "actual"
allocations. But it seems useful to have both - one to calculate the
budgets, the other to monitor how balanced the result is.

I haven't seen the extra atomic in profiles, even on workloads that do a
lot of buffer allocations (e.g. seqscan with datasets > shared buffers).
But if that happens, I think there are ways to mitigate that.

>   approach until we've measured it being a problem.

I don't follow. How would making the budgets higher change any of this?

Anyway, I think choosing the partitions less frequently - e.g. only
after consuming budget for the current partition, or going "full cycle",
would make this a non-issue.

>   approach should work, but verifying that...

Will try.

>   to break this kind of thing and not notice, as everything just continues to

Do you mean a test that'd be a part of make check, or a standalone test?
AFAICS any meaningful test would need to be fairly expensive, so
probably not a good fit for make check.

regards

-- 
Tomas Vondra

---

From: Tomas Vondra <[email-masked]>
Date: Thu, 11 Sep 2025 10:32:40 +0200

Hi,

Here's a fresh version of the NUMA patch series. There's a number of
substantial improvements:

1) Rebase to current master, particularly on top of 2c7894052759 which
removed the freelist. The patch that partitioned the freelist is gone.

2) A bunch of fixes, and it now passes CI workflows on github, and all
other testing I did. Of course, more testing is needed.

3) It builds with/without libnuma support, and so on.

4) The separate GUCs were replaced by a single list GUC, similar to what
we do for debug_io_direct. The GUC is called debug_numa, and accepts
"buffets" and "procs", to partition the two areas.

I'm considering adding a "clock-sweep" option in a future patch. I
didn't do that here, because the buffer partitioning is already enabled
by "buffers", and the clock-sweep just builds on that (partitioning the
same way, pretty much).

5) It also works with EXEC_BACKEND, but this turned out to be a bit more
challenging than expected. The trouble is some of the parameters (e.g.
memory page size) are used both in the "size" and "init" phases, and we
need to be extra careful to not make something "inconsistent".

For example, we may get confused about the memory page size. The "size"
happens before allocation, and at that point we don't know if we succeed
in getting enough huge pages. When "init" happens, we already know that,
so our "memory page size" could be different. We must be careful, e.g.
to not need more memory than we requested.

This is a general problem, but the EXEC_BACKEND makes it a bit trickier.
In regular fork() case we can simply set some global variables in "size"
and use them later in "init", but that doesn't work for EXEC_BACKEND.

The current approach simply does the calculations twice, in a way that
should end with the same results. But I'm not 100% happy with it, and I
suspect it just confirms we should store the results in shmem memory
(which is what I called "NUMA registry" before). That's still a TODO.

6) BufferDescPadded is 64B everywhere. Originally, the padding was
applied only on 64-bit platforms, and on 32-bit systems the struct was
left at 60B. But that is not compatible with buffer partitioning, which
relies on the memory page being a multiple of BufferDescPadded.

Perhaps it could be relaxed (so that a BufferDesc might span two memory
pages), but this seems like the cleanes solution. I don't expect it to
make any measurable difference.

7) I've moved some of the code to BgBufferSyncPartition, to make review
easier (without all the indentation changes).

8) I've realized some of the TAP tests occasionally fail with

    ERROR: no unpinned buffers

and I think I know why. Some of the tests set shared_buffers to a very
low value - like 1MB or even 128kB, and StrategyGetBuffer() may search
only a single partition (but not always). We may run out of unpinned
buffers in that one partition.

This apparently happens more easily on rpi5, due to the weird NUMA
layout (there are 8 nodes with memory, but getcpu() reports node 0 for
all cores).

I suspect the correct fix is to ensure StrategyGetBuffer() scans all
partitions, if there are no unpinned buffers in the current one. On
realistic setups this shouldn't happen very often, I think.

The other issue I just realized is that StrategyGetBuffer() recalculates
the partition index over and over, which seems unnecessary (and possibly
expensive, due to the modulo). And it also does too many loops, because
it used NBuffers instead of the partition size. I'll fix those later.

9) I'm keeping the cloc-sweep balancing patches separate for now. In the
end all the clock-sweep patches should be merged, but it keeps the
changes easier to review this way, I think.

10) There's not many changes in the PGPROC partitioning patch. I merely
fixed issues that broke it on EXEC_BACKEND, and did some smaller tweaks.

11) I'm not including the experimental patches to pin backends to CPUs
(or nodes), and so on. It's clear those are unlikely to go in, so it'd
be just a distraction.

12) What's the right / portable way to determine the current CPU for a
process? The clock-sweep / PGPROC patches need this to pick the right
partition, but it's not clear to me which API is the most portable. In
the end I used sched_getcpu(), and then numa_node_of_cpu(), but maybe
there's a better way.

regards

-- 
Tomas Vondra
From 318f2441cc67daf7fb2ba672900a2bd4dfdea170 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Mon, 28 Jul 2025 14:01:37 +0200
Subject: [PATCH v20250911 1/8] NUMA: shared buffers partitioning

The effect is similar to

     numactl --interleave=all

This allows partitioning clock-sweep in a later patch, with one clock
hand per partition.

With the feature disabled (GUC set to empty list), there'll be a single
partition for all the buffers (and it won't be mapped to a NUMA node).

Notes:

* The feature is enabled by debug_numa = buffers GUC (default: empty),
  which works similarly to debug_io_direct.

* This patch partitions just shared buffers, not the whole shared
  memory. A later patch will do that for PGPROC, but it's tricky and
  requires a different approach because of huge pages.
---
 contrib/pg_buffercache/Makefile               |   2 +-
 contrib/pg_buffercache/meson.build            |   1 +
 .../pg_buffercache--1.6--1.7.sql              |  26 +
 contrib/pg_buffercache/pg_buffercache.control |   2 +-
 contrib/pg_buffercache/pg_buffercache_pages.c |  92 +++
 src/backend/storage/buffer/buf_init.c         | 627 +++++++++++++++++-
 src/backend/utils/misc/guc_parameters.dat     |  10 +
 src/backend/utils/misc/guc_tables.c           |   1 +
 src/include/port/pg_numa.h                    |   6 +
 src/include/storage/buf_internals.h           |  16 +-
 src/include/storage/bufmgr.h                  |  23 +
 src/include/utils/guc_hooks.h                 |   3 +
 src/port/pg_numa.c                            |  64 ++
 src/tools/pgindent/typedefs.list              |   2 +
 14 files changed, 859 insertions(+), 16 deletions(-)
 create mode 100644 contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql

 REGRESS = pg_buffercache pg_buffercache_numa
diff --git a/contrib/pg_buffercache/meson.build b/contrib/pg_buffercache/meson.build
index 7cd039a1df9..7c31141881f 100644
--- a/contrib/pg_buffercache/meson.build
+++ b/contrib/pg_buffercache/meson.build
@@ -24,6 +24,7 @@ install_data(
   'pg_buffercache--1.3--1.4.sql',
   'pg_buffercache--1.4--1.5.sql',
   'pg_buffercache--1.5--1.6.sql',
+  'pg_buffercache--1.6--1.7.sql',
   'pg_buffercache.control',
   kwargs: contrib_data_args,
 )
diff --git a/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql b/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
new file mode 100644
index 00000000000..fb9003c011e
--- /dev/null
+++ b/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
@@ -0,0 +1,26 @@
+/* contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "ALTER EXTENSION pg_buffercache UPDATE TO '1.7'" to load this file. \quit
+
+-- Register the new functions.
+CREATE OR REPLACE FUNCTION pg_buffercache_partitions()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_partitions'
+LANGUAGE C PARALLEL SAFE;
+
+-- Create a view for convenient access.
+CREATE VIEW pg_buffercache_partitions AS
+	SELECT P.* FROM pg_buffercache_partitions() AS P
+	(partition integer,			-- partition index
+	 numa_node integer,			-- NUMA node of the partitioon
+	 num_buffers integer,		-- number of buffers in the partition
+	 first_buffer integer,		-- first buffer of partition
+	 last_buffer integer);		-- last buffer of partition
+
+-- Don't want these to be available to public.
+REVOKE ALL ON FUNCTION pg_buffercache_partitions() FROM PUBLIC;
+REVOKE ALL ON pg_buffercache_partitions FROM PUBLIC;
+
+GRANT EXECUTE ON FUNCTION pg_buffercache_partitions() TO pg_monitor;
+GRANT SELECT ON pg_buffercache_partitions TO pg_monitor;
diff --git a/contrib/pg_buffercache/pg_buffercache.control b/contrib/pg_buffercache/pg_buffercache.control
index b030ba3a6fa..11499550945 100644
--- a/contrib/pg_buffercache/pg_buffercache.control
+++ b/contrib/pg_buffercache/pg_buffercache.control
@@ -1,5 +1,5 @@
 # pg_buffercache extension
 comment = 'examine the shared buffer cache'
-default_version = '1.6'
+default_version = '1.7'
 module_pathname = '$libdir/pg_buffercache'
 relocatable = true
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 3df04c98959..8a0a4bd5cd6 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -27,6 +27,7 @@
 #define NUM_BUFFERCACHE_EVICT_ALL_ELEM 3

 PG_MODULE_MAGIC_EXT(

 /* Only need to touch memory once per backend process lifetime */
@@ -777,3 +779,93 @@ pg_buffercache_evict_all(PG_FUNCTION_ARGS)
 
 	PG_RETURN_DATUM(result);
 }
+
+/*
index 6fd3a6bbac5..f51c7db7855 100644
@@ -14,9 +14,20 @@
  */
 #include "postgres.h"
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
+#include "port/pg_numa.h"
 #include "storage/aio.h"
 #include "storage/buf_internals.h"
 #include "storage/bufmgr.h"
+#include "storage/pg_shmem.h"
+#include "storage/proc.h"
+#include "utils/guc.h"
+#include "utils/guc_hooks.h"
+#include "utils/varlena.h"
 
 BufferDescPadded *BufferDescriptors;
 char	   *BufferBlocks;
@@ -24,6 +35,23 @@ ConditionVariableMinimallyPadded *BufferIOCVArray;
 WritebackContext BackendWritebackContext;
 CkptSortItem *CkptBufferIds;
 
+/*
+ * Minimum number of buffer partitions, no matter the number of NUMA nodes.
+ */
+#define MIN_BUFFER_PARTITIONS	4
+
+/* Array of structs with information about buffer ranges */
+BufferPartitions *BufferPartitionsArray = NULL;
+
+static void buffer_partitions_prepare(void);
+static void buffer_partitions_init(void);
+
+/* number of NUMA nodes (as returned by numa_num_configured_nodes) */
+static int	numa_nodes = -1;	/* number of nodes when sizing */
+static Size numa_page_size = 0; /* page used to size partitions */
+static bool numa_can_partition = false; /* can map to NUMA nodes? */
+static int	numa_buffers_per_node = -1; /* buffers per node */
+static int	numa_partitions = 0;	/* total (multiple of nodes) */
 
 /*
  * Data Structures:
@@ -70,19 +98,87 @@ BufferManagerShmemInit(void)
 	bool		foundBufs,
 				foundDescs,
 				foundIOCV,
-				foundBufCkpt;
+				foundBufCkpt,
+				foundParts;
+	Size		buffer_align;
+
+	/*
+	 * Determine the memory page size used to partition shared buffers over
+	 * the available NUMA nodes.
+	 *
+	 * XXX We have to call prepare again, because with EXEC_BACKEND we may not
+	 * see the values already calculated in BufferManagerShmemSize().
+	 *
+	 * XXX We need to be careful to get the same value when calculating the
+	 * and then later when initializing the structs after allocation, or to not
+	 * depend on that value too much. Before the allocation we don't know if we
+	 * get huge pages, so we just have to assume we do.
+	 */
+	buffer_partitions_prepare();
+
+	/*
+	 * With NUMA we need to ensure the buffers are properly aligned not just
+	 * to PG_IO_ALIGN_SIZE, but also to memory page size. NUMA works on page
+	 * granularity, and we don't want a buffer to get split to multiple nodes
+	 * (when spanning multiple memory pages).
+	 *
+	 * We also don't want to interfere with other parts of shared memory,
+	 * which could easily happen with huge pages (e.g. with data stored before
+	 * buffers).
+	 *
+	 * We do this by aligning to the larger of the two values (we know both
+	 * are power-of-two values, so the larger value is automatically a
+	 * multiple of the lesser one).
+	 *
+	 * XXX Maybe there's a way to use less alignment?
+	 *
+	 * XXX Maybe with (numa_page_size > PG_IO_ALIGN_SIZE), we don't need to
+	 * align to numa_page_size? Especially for very large huge pages (e.g. 1GB)
+	 * that doesn't seem quite worth it. Maybe we should simply align to
+	 * BLCKSZ, so that buffers don't get split? Still, we might interfere with
+	 * other stuff stored in shared memory that we want to allocate on a
+	 * particular NUMA node (e.g. ProcArray).
+	 *
+	 * XXX Maybe with "too large" huge pages we should just not do this, or
+	 * maybe do this only for sufficiently large areas (e.g. shared buffers,
+	 * but not ProcArray).
+	 */
+	buffer_align = Max(numa_page_size, PG_IO_ALIGN_SIZE);
+
+	/* one page is a multiple of the other */
+	Assert(((numa_page_size % PG_IO_ALIGN_SIZE) == 0) ||
+		   ((PG_IO_ALIGN_SIZE % numa_page_size) == 0));

 
 	/* Align condition variables to cacheline boundary. */
@@ -112,6 +208,12 @@ BufferManagerShmemInit(void)
 	{
 		int			i;
 
+		/*
+		 * Initialize buffer partitions, including moving memory to different
+		 * NUMA nodes (if enabled by GUC).
+		 */
+		buffer_partitions_init();
+
 		/*
 		 * Initialize all the buffer headers.
 		 */
@@ -148,19 +250,26 @@ BufferManagerShmemInit(void)
  *
  * compute the size of shared memory for the buffer pool including
  * data pages, buffer descriptors, hash tables, etc.
+ *
+ * XXX Called before allocation, so we don't know if huge pages get used yet.
+ * So we need to assume huge pages get used, and use get_memory_page_size()
+ * to calculate the largest possible memory page.
  */
 Size
 BufferManagerShmemSize(void)
 {
 	Size		size = 0;

 
 	/* size of stuff controlled by freelist.c */
@@ -175,5 +284,505 @@ BufferManagerShmemSize(void)
 	/* size of checkpoint sort array in bufmgr.c */
 	size = add_size(size, mul_size(NBuffers, sizeof(CkptSortItem)));
 
+	/* not NUMA partitioning */
+#if USE_LIBNUMA
+	/* without NUMA, assume there's just one node */
+	/* we should never get here without at least one NUMA node */
+	numa_page_size = pg_numa_page_size();
+	elog(DEBUG1, "NUMA: buffers %d partitions %d num_nodes %d per_node %d buffers_per_node %d (min %d)",
+ * Sanity checks of buffers partitions - there must be no gaps, it must cover
+ * the whole range of buffers, etc.
+	Assert(BufferPartitionsArray->npartitions > 0);
+	for (int i = 0; i < BufferPartitionsArray->npartitions; i++)
+		if (i == (BufferPartitionsArray->npartitions - 1))
+ * buffer_partitions_init
+ *		Initialize array of buffer partitions.
+	BufferPartitionsArray->nnodes = numa_nodes;
+			if ((numa_flags & NUMA_BUFFERS) != 0)
+				part->numa_node = -1;
+			elog(DEBUG1, "NUMA: buffer %d node %d partition %d buffers %d first %d last %d", idx, n, p, num_buffers, buffer, buffer + (num_buffers - 1));
+	if (((numa_flags & NUMA_BUFFERS) == 0) || !numa_can_partition)
+		elog(DEBUG1, "NUMA: buffer_partitions_init: %d => %d buffers %d start %p end %p (size %zd)",
+		elog(DEBUG1, "NUMA: buffer_partitions_init: %d => %d descriptors %d start %p end %p (size %zd)",
+BufferPartitionNodes(void)
+	return BufferPartitionsArray->nnodes;
+/* XXX the GUC hooks should probably be somewhere else? */
+bool
+check_debug_numa(char **newval, void **extra, GucSource source)
+{
+	bool		result = true;
+	int			flags;
+
+#if USE_LIBNUMA == 0
+	if (strcmp(*newval, "") != 0)
+	{
+		GUC_check_errdetail("\"%s\" is not supported on this platform.",
+							"debug_numa");
+		result = false;
+	}
+	flags = 0;
+#else
+	List	   *elemlist;
+	ListCell   *l;
+	char	   *rawstring;
+
+	/* Need a modifiable copy of string */
+	rawstring = pstrdup(*newval);
+
+	if (!SplitGUCList(rawstring, ',', &elemlist))
+	{
+		GUC_check_errdetail("Invalid list syntax in parameter \"%s\".",
+							"debug_numa");
+		pfree(rawstring);
+		list_free(elemlist);
+		return false;
+	}
+
+	flags = 0;
+	foreach(l, elemlist)
+	{
+		char	   *item = (char *) lfirst(l);
+
+		if (pg_strcasecmp(item, "buffers") == 0)
+			flags |= NUMA_BUFFERS;
+		else
+		{
+			GUC_check_errdetail("Invalid option \"%s\".", item);
+			result = false;
+			break;
+		}
+	}
+
+	pfree(rawstring);
+	list_free(elemlist);
+#endif
+
+	if (!result)
+		return result;
+
+	/* Save the flags in *extra, for use by assign_debug_io_direct */
+	*extra = guc_malloc(LOG, sizeof(int));
+	if (!*extra)
+		return false;
+	*((int *) *extra) = flags;
+
+	return result;
+}
+
+void
+assign_debug_numa(const char *newval, void *extra)
+{
+	int		   *flags = (int *) extra;
+
+	numa_flags = *flags;
+}
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 0da01627cfe..d18666a94cb 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -899,6 +899,16 @@
   boot_val => 'true',
 },
 
+{ name => 'debug_numa', type => 'string', context => 'PGC_POSTMASTER', group => 'DEVELOPER_OPTIONS',
+  short_desc => 'NUMA-aware partitioning of shared memory.',
+  long_desc => 'An empty string disables NUMA-aware partitioning.',
+  flags => 'GUC_LIST_INPUT | GUC_NOT_IN_SAMPLE',
+  variable => 'debug_numa_string',
+  boot_val => '""',
+  check_hook => 'check_debug_numa',
+  assign_hook => 'assign_debug_numa',
+},
+
 { name => 'sync_replication_slots', type => 'bool', context => 'PGC_SIGHUP', group => 'REPLICATION_STANDBY',
   short_desc => 'Enables a physical standby to synchronize logical failover replication slots from the primary server.',
   variable => 'sync_replication_slots',
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 00c8376cf4d..d17ee9ca861 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -595,6 +595,7 @@ static char *server_version_string;
 static int	server_version_num;
 static char *debug_io_direct_string;
 static char *restrict_nonsystem_relation_kind_string;
+static char *debug_numa_string;
 
 #ifdef HAVE_SYSLOG
 #define	DEFAULT_SYSLOG_FACILITY LOG_LOCAL0
diff --git a/src/include/port/pg_numa.h b/src/include/port/pg_numa.h
index 9d1ea6d0db8..9734aa315ff 100644
--- a/src/include/port/pg_numa.h
+++ b/src/include/port/pg_numa.h
@@ -17,6 +17,12 @@
 extern PGDLLIMPORT int pg_numa_init(void);
 extern PGDLLIMPORT int pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status);
 extern PGDLLIMPORT int pg_numa_get_max_node(void);
+extern PGDLLIMPORT Size pg_numa_page_size(void);
+extern PGDLLIMPORT void pg_numa_move_to_node(char *startptr, char *endptr, int node);
+
+extern PGDLLIMPORT int numa_flags;
+
+#define		NUMA_BUFFERS		0x01
 
 #ifdef USE_LIBNUMA
 
diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h
index dfd614f7ca4..294188e21c5 100644
--- a/src/include/storage/buf_internals.h
+++ b/src/include/storage/buf_internals.h
@@ -275,10 +275,10 @@ typedef struct BufferDesc
  * line sized.
  *
  * XXX: As this is primarily matters in highly concurrent workloads which
- * probably all are 64bit these days, and the space wastage would be a bit
- * more noticeable on 32bit systems, we don't force the stride to be cache
- * line sized on those. If somebody does actual performance testing, we can
- * reevaluate.
+ * probably all are 64bit these days. We force the stride to be cache line
+ * sized even on 32bit systems, where the space wastage is be a bit more
+ * noticeable, to allow partitioning of shared buffers (which requires the
+ * memory page be a multiple of buffer descriptor).
  *
  * Note that local buffer descriptors aren't forced to be aligned - as there's
  * no concurrent access to those it's unlikely to be beneficial.
@@ -288,7 +288,7 @@ typedef struct BufferDesc
  * platform with either 32 or 128 byte line sizes, it's good to align to
  * boundaries and avoid false sharing.
  */
-#define BUFFERDESC_PAD_TO_SIZE	(SIZEOF_VOID_P == 8 ? 64 : 1)
+#define BUFFERDESC_PAD_TO_SIZE	64
 
 typedef union BufferDescPadded
 {
@@ -321,6 +321,7 @@ typedef struct WritebackContext
 
 /* in buf_init.c */
 
@@ -480,4 +481,9 @@ extern void DropRelationLocalBuffers(RelFileLocator rlocator,
 extern void DropRelationAllLocalBuffers(RelFileLocator rlocator);
 extern void AtEOXact_LocalBuffers(bool isCommit);
 
+extern int	BufferPartitionCount(void);
+extern int	BufferPartitionNodes(void);
+extern void BufferPartitionGet(int idx, int *node, int *num_buffers,
+							   int *first_buffer, int *last_buffer);
+
 #endif							/* BUFMGR_INTERNALS_H */
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index 41fdc1e7693..e52fca9e483 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -143,6 +143,28 @@ struct ReadBuffersOperation

+/*
+ * information about one partition of shared buffers
+ *
+ * numa_nod specifies node for this partition (-1 means allocated on any node)
+ * first/last buffer - the values are inclusive
+ */
+typedef struct BufferPartition
+{
+	int			numa_node;		/* NUMA node (-1 no node) */
+	int			num_buffers;	/* number of buffers */
+	int			first_buffer;	/* first buffer of partition */
+	int			last_buffer;	/* last buffer of partition */
+} BufferPartition;
+
+/* an array of information about all partitions */
+typedef struct BufferPartitions
+{
+	int			npartitions;	/* number of partitions */
+	int			nnodes;			/* number of NUMA nodes */
+	BufferPartition partitions[FLEXIBLE_ARRAY_MEMBER];
+} BufferPartitions;
+
 /* forward declared, to avoid having to expose buf_internals.h here */
 struct WritebackContext;
 
@@ -319,6 +341,7 @@ extern void EvictRelUnpinnedBuffers(Relation rel,
 /* in buf_init.c */
 extern void BufferManagerShmemInit(void);
 extern Size BufferManagerShmemSize(void);
+extern int	BufferGetNode(Buffer buffer);
 
 /* in localbuf.c */
 extern void AtProcExit_LocalBuffers(void);
diff --git a/src/include/utils/guc_hooks.h b/src/include/utils/guc_hooks.h
index 82ac8646a8d..15304df0de5 100644
--- a/src/include/utils/guc_hooks.h
+++ b/src/include/utils/guc_hooks.h
@@ -175,4 +175,7 @@ extern bool check_synchronized_standby_slots(char **newval, void **extra,
 											 GucSource source);
 extern void assign_synchronized_standby_slots(const char *newval, void *extra);
 
+extern bool check_debug_numa(char **newval, void **extra, GucSource source);
+extern void assign_debug_numa(const char *newval, void *extra);
+
 #endif							/* GUC_HOOKS_H */
diff --git a/src/port/pg_numa.c b/src/port/pg_numa.c
index 3368a43a338..8ee0e7d211c 100644
--- a/src/port/pg_numa.c
+++ b/src/port/pg_numa.c
@@ -18,6 +18,9 @@
 
 #include "miscadmin.h"
+int	numa_flags;
 
 /*
  * At this point we provide support only for Linux thanks to libnuma, but in
@@ -106,6 +109,36 @@ pg_numa_get_max_node(void)
 	return numa_max_node();
 }
 
+/*
+ * pg_numa_move_to_node
+ *		move memory to different NUMA nodes in larger chunks
+ *
+ * startptr - start of the region (should be aligned to page size)
+ * endptr - end of the region (doesn't need to be aligned)
+ * node - node to move the memory to
+ *
+ * The "startptr" is expected to be a multiple of system memory page size, as
+ * determined by pg_numa_page_size.
+ *
+ * XXX We only expect to do this during startup, when the shared memory is
+ * still being setup.
+ */
+void
+pg_numa_move_to_node(char *startptr, char *endptr, int node)
+{
+	Size		sz = (endptr - startptr);
+
+	Assert((int64) startptr % pg_numa_page_size() == 0);
+
+	/*
+	 * numa_tonode_memory does not actually cause a page fault, and thus does
+	 * not locate the memory on the node. So it's fast, at least compared to
+	 * pg_numa_query_pages, and does not make startup longer. But it also
+	 * means the expensive part happen later, on the first access.
+	 */
+	numa_tonode_memory(startptr, sz, node);
+}
+
 #else
 
 /* Empty wrappers */
@@ -128,4 +161,35 @@ pg_numa_get_max_node(void)
 	return 0;
 }
 
+void
+	/* we don't expect to ever get here in builds without libnuma */
+	Assert(false);
+Size
+pg_numa_page_size(void)
index a13e8162890..50195718294 100644
2.51.0

From 38ccd93b4a70d65e16da3303e9f94851c3a3fb5a Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Sun, 8 Jun 2025 18:53:12 +0200
Subject: [PATCH v20250911 2/8] NUMA: clockweep partitioning

Note: There's a problem with some tests running out of unpinned buffers,
due to (intentionally) setting shared buffers very low. That happens
because StrategyGetBuffer() only searches a single partition, and it
has a couple more issues.
---
 .../pg_buffercache--1.6--1.7.sql              |   8 +-
 contrib/pg_buffercache/pg_buffercache_pages.c |  32 +-
 src/backend/storage/buffer/buf_init.c         |  11 +
 src/backend/storage/buffer/bufmgr.c           | 186 +++++----
 src/backend/storage/buffer/freelist.c         | 353 ++++++++++++++++--
 src/include/storage/buf_internals.h           |   5 +-
 src/include/storage/bufmgr.h                  |   5 +
 src/test/recovery/t/027_stream_regress.pl     |   5 +
 src/tools/pgindent/typedefs.list              |   1 +
 9 files changed, 503 insertions(+), 103 deletions(-)

index fb9003c011e..6676e807034 100644
@@ -16,7 +16,13 @@ CREATE VIEW pg_buffercache_partitions AS
 
index 8a0a4bd5cd6..c9dfc8a1b82 100644
 
 #define NUM_BUFFERCACHE_NUMA_ELEM	3
-#define NUM_BUFFERCACHE_PARTITIONS_ELEM	5
+#define NUM_BUFFERCACHE_PARTITIONS_ELEM	9
 
 PG_MODULE_MAGIC_EXT(
 					.name = "pg_buffercache",
@@ -818,6 +818,14 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 						   INT4OID, -1, 0);
 		TupleDescInitEntry(tupledesc, (AttrNumber) 5, "last_buffer",
 						   INT4OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 6, "num_passes",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 7, "next_buffer",
+						   INT4OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 8, "total_allocs",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 9, "num_allocs",
+						   INT8OID, -1, 0);

@@ -839,12 +847,22 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 					first_buffer,
 					last_buffer;

 
+		ClockSweepPartitionGetInfo(i,
+								   &complete_passes, &next_victim_buffer,
+								   &buffer_total_allocs, &buffer_allocs);
+
 		values[0] = Int32GetDatum(i);
 		nulls[0] = false;
 
@@ -860,6 +878,18 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		values[4] = Int32GetDatum(last_buffer);
 		nulls[4] = false;
 
+		values[5] = Int64GetDatum(complete_passes);
+		nulls[5] = false;
+
+		values[6] = Int32GetDatum(next_victim_buffer);
+		nulls[6] = false;
+
+		values[7] = Int64GetDatum(buffer_total_allocs);
+		nulls[7] = false;
+
+		values[8] = Int64GetDatum(buffer_allocs);
+		nulls[8] = false;
+
 		/* Build and return the tuple. */
 		tuple = heap_form_tuple((TupleDesc) funcctx->user_fctx, values, nulls);
 		result = HeapTupleGetDatum(tuple);
diff --git a/src/backend/storage/buffer/buf_init.c b/src/backend/storage/buffer/buf_init.c
index f51c7db7855..dd9f51529b4 100644
--- a/src/backend/storage/buffer/buf_init.c
+++ b/src/backend/storage/buffer/buf_init.c
@@ -716,6 +716,17 @@ BufferPartitionGet(int idx, int *node, int *num_buffers,
 	elog(ERROR, "invalid partition index");
 }
 
+/* return parameters before the partitions are initialized (during sizing) */
+void
+BufferPartitionParams(int *num_partitions, int *num_nodes)
+{
+	if (num_partitions)
+		*num_partitions = numa_partitions;
+
+	if (num_nodes)
+		*num_nodes = numa_nodes;
+}
+
 /* XXX the GUC hooks should probably be somewhere else? */
 bool
 check_debug_numa(char **newval, void **extra, GucSource source)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index fe470de63f2..121134bb94c 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3580,33 +3580,29 @@ BufferSync(int flags)
 }
 
 /*
- * BgBufferSync -- Write out some dirty buffers in the pool.
+ * Information saved between calls so we can determine the strategy
+ * point's advance rate and avoid scanning already-cleaned buffers.
  *
- * This is called periodically by the background writer process.
+ * XXX One value per partition. We don't know how many partitions are
+ * there, so allocate 32, should be enough for the PoC patch.
  *
- * Returns true if it's appropriate for the bgwriter process to go into
- * low-power hibernation mode.  (This happens if the strategy clock-sweep
- * has been "lapped" and no buffer allocations have occurred recently,
- * or if the bgwriter has been effectively disabled by setting
- * bgwriter_lru_maxpages to 0.)
+ * XXX might be better to have a per-partition struct with all the info
  */
-bool
-BgBufferSync(WritebackContext *wb_context)
+#define MAX_CLOCKSWEEP_PARTITIONS 32
+static bool saved_info_valid = false;
+static int	prev_strategy_buf_id[MAX_CLOCKSWEEP_PARTITIONS];
+static uint32 prev_strategy_passes[MAX_CLOCKSWEEP_PARTITIONS];
+static int	next_to_clean[MAX_CLOCKSWEEP_PARTITIONS];
+static uint32 next_passes[MAX_CLOCKSWEEP_PARTITIONS];
+
+
+static bool
+BgBufferSyncPartition(WritebackContext *wb_context, int num_partitions,
+					  int partition, int recent_alloc_partition)
 {
 	/* info obtained from freelist.c */
 	int			strategy_buf_id;
 	uint32		strategy_passes;
-	uint32		recent_alloc;
-
-	/*
-	 * Information saved between calls so we can determine the strategy
-	 * point's advance rate and avoid scanning already-cleaned buffers.
-	 */
-	static bool saved_info_valid = false;
-	static int	prev_strategy_buf_id;
-	static uint32 prev_strategy_passes;
-	static int	next_to_clean;
-	static uint32 next_passes;
 
 	/* Moving averages of allocation rate and clean-buffer density */
 	static float smoothed_alloc = 0;
@@ -3634,25 +3630,16 @@ BgBufferSync(WritebackContext *wb_context)
 	long		new_strategy_delta;
 	uint32		new_recent_alloc;
 
+	/* buffer range for the clocksweep partition */
+	int			first_buffer;
+	int			num_buffers;
+
 	/*
 	 * Find out where the clock-sweep currently is, and how many buffer
 	 * allocations have happened since our last call.
 	 */
-	strategy_buf_id = StrategySyncStart(&strategy_passes, &recent_alloc);
-
-	/* Report buffer alloc counts to pgstat */
-	PendingBgWriterStats.buf_alloc += recent_alloc;
-
-	/*
-	 * If we're not running the LRU scan, just stop after doing the stats
-	 * stuff.  We mark the saved state invalid so that we can recover sanely
-	 * if LRU scan is turned back on later.
-	 */
-	if (bgwriter_lru_maxpages <= 0)
-	{
-		saved_info_valid = false;
-		return true;
-	}
+	strategy_buf_id = StrategySyncStart(partition, &strategy_passes,
+										&first_buffer, &num_buffers);
 
 	/*
 	 * Compute strategy_delta = how many buffers have been scanned by the
@@ -3664,17 +3651,17 @@ BgBufferSync(WritebackContext *wb_context)
 	 */
 	if (saved_info_valid)
 	{
-		int32		passes_delta = strategy_passes - prev_strategy_passes;
+		int32		passes_delta = strategy_passes - prev_strategy_passes[partition];

 		Assert(strategy_delta >= 0);
 
-		if ((int32) (next_passes - strategy_passes) > 0)
+		if ((int32) (next_passes[partition] - strategy_passes) > 0)
 		{
 			/* we're one pass ahead of the strategy point */
-			bufs_to_lap = strategy_buf_id - next_to_clean;
+			bufs_to_lap = strategy_buf_id - next_to_clean[partition];
 #ifdef BGW_DEBUG
 			elog(DEBUG2, "bgwriter ahead: bgw %u-%u strategy %u-%u delta=%ld lap=%d",
 				 next_passes, next_to_clean,
@@ -3682,11 +3669,11 @@ BgBufferSync(WritebackContext *wb_context)
 				 strategy_delta, bufs_to_lap);
 #endif
 		}
-		else if (next_passes == strategy_passes &&
-				 next_to_clean >= strategy_buf_id)
+		else if (next_passes[partition] == strategy_passes &&
+				 next_to_clean[partition] >= strategy_buf_id)
 		{
 			/* on same pass, but ahead or at least not behind */
-			bufs_to_lap = NBuffers - (next_to_clean - strategy_buf_id);
+			bufs_to_lap = num_buffers - (next_to_clean[partition] - strategy_buf_id);
 #ifdef BGW_DEBUG
 			elog(DEBUG2, "bgwriter ahead: bgw %u-%u strategy %u-%u delta=%ld lap=%d",
 				 next_passes, next_to_clean,
@@ -3706,9 +3693,9 @@ BgBufferSync(WritebackContext *wb_context)
 				 strategy_passes, strategy_buf_id,
 				 strategy_delta);
 #endif
-			next_to_clean = strategy_buf_id;
-			next_passes = strategy_passes;
-			bufs_to_lap = NBuffers;
+			next_to_clean[partition] = strategy_buf_id;
+			next_passes[partition] = strategy_passes;
+			bufs_to_lap = num_buffers;
 		}
 	}
 	else
@@ -3722,15 +3709,16 @@ BgBufferSync(WritebackContext *wb_context)
 			 strategy_passes, strategy_buf_id);
 #endif
 		strategy_delta = 0;
-		next_to_clean = strategy_buf_id;
-		next_passes = strategy_passes;
-		bufs_to_lap = NBuffers;
+		next_to_clean[partition] = strategy_buf_id;
+		next_passes[partition] = strategy_passes;
+		bufs_to_lap = num_buffers;
 	}
 
 	/* Update saved info for next time */
-	prev_strategy_buf_id = strategy_buf_id;
-	prev_strategy_passes = strategy_passes;
-	saved_info_valid = true;
+	prev_strategy_buf_id[partition] = strategy_buf_id;
+	prev_strategy_passes[partition] = strategy_passes;
+	/* XXX this needs to happen only after all partitions */
+	/* saved_info_valid = true; */
 
 	/*
 	 * Compute how many buffers had to be scanned for each new allocation, ie,
@@ -3738,9 +3726,9 @@ BgBufferSync(WritebackContext *wb_context)
 	 *
 	 * If the strategy point didn't move, we don't update the density estimate
 	 */
-	if (strategy_delta > 0 && recent_alloc > 0)
+	if (strategy_delta > 0 && recent_alloc_partition > 0)
 	{
-		scans_per_alloc = (float) strategy_delta / (float) recent_alloc;
+		scans_per_alloc = (float) strategy_delta / (float) recent_alloc_partition;
 		smoothed_density += (scans_per_alloc - smoothed_density) /
 			smoothing_samples;
 	}
@@ -3750,7 +3738,7 @@ BgBufferSync(WritebackContext *wb_context)
 	 * strategy point and where we've scanned ahead to, based on the smoothed
 	 * density estimate.
 	 */
-	bufs_ahead = NBuffers - bufs_to_lap;
+	bufs_ahead = num_buffers - bufs_to_lap;
 	reusable_buffers_est = (float) bufs_ahead / smoothed_density;
 
 	/*
@@ -3758,10 +3746,10 @@ BgBufferSync(WritebackContext *wb_context)
 	 * a true average we want a fast-attack, slow-decline behavior: we
 	 * immediately follow any increase.
 	 */
-	if (smoothed_alloc <= (float) recent_alloc)
-		smoothed_alloc = recent_alloc;
+	if (smoothed_alloc <= (float) recent_alloc_partition)
+		smoothed_alloc = recent_alloc_partition;
 	else
-		smoothed_alloc += ((float) recent_alloc - smoothed_alloc) /
+		smoothed_alloc += ((float) recent_alloc_partition - smoothed_alloc) /
 			smoothing_samples;
 
 	/* Scale the estimate by a GUC to allow more aggressive tuning. */
@@ -3788,7 +3776,7 @@ BgBufferSync(WritebackContext *wb_context)
 	 * the BGW will be called during the scan_whole_pool time; slice the
 	 * buffer pool into that many sections.
 	 */
-	min_scan_buffers = (int) (NBuffers / (scan_whole_pool_milliseconds / BgWriterDelay));
+	min_scan_buffers = (int) (num_buffers / (scan_whole_pool_milliseconds / BgWriterDelay));
 
 	if (upcoming_alloc_est < (min_scan_buffers + reusable_buffers_est))
 	{
@@ -3813,20 +3801,20 @@ BgBufferSync(WritebackContext *wb_context)
 	/* Execute the LRU scan */
 	while (num_to_scan > 0 && reusable_buffers < upcoming_alloc_est)
 	{
-		int			sync_state = SyncOneBuffer(next_to_clean, true,
+		int			sync_state = SyncOneBuffer(next_to_clean[partition], true,
 											   wb_context);
 
 		num_to_scan--;
 
 		if (sync_state & BUF_WRITTEN)
 		{
 			reusable_buffers++;
-			if (++num_written >= bgwriter_lru_maxpages)
+			if (++num_written >= (bgwriter_lru_maxpages / num_partitions))
 			{
 				PendingBgWriterStats.maxwritten_clean++;
 				break;
@@ -3840,7 +3828,7 @@ BgBufferSync(WritebackContext *wb_context)
 
 #ifdef BGW_DEBUG
 	elog(DEBUG1, "bgwriter: recent_alloc=%u smoothed=%.2f delta=%ld ahead=%d density=%.2f reusable_est=%d upcoming_est=%d scanned=%d wrote=%d reusable=%d",
-		 recent_alloc, smoothed_alloc, strategy_delta, bufs_ahead,
+		 recent_alloc_partition, smoothed_alloc, strategy_delta, bufs_ahead,
 		 smoothed_density, reusable_buffers_est, upcoming_alloc_est,
 		 bufs_to_lap - num_to_scan,
 		 num_written,
@@ -3870,8 +3858,74 @@ BgBufferSync(WritebackContext *wb_context)
 #endif
 	}
 
+	/* can this partition hibernate */
+	return (bufs_to_lap == 0 && recent_alloc_partition == 0);
+}
+
+/*
+ * BgBufferSync -- Write out some dirty buffers in the pool.
+ *
+ * This is called periodically by the background writer process.
+ *
+ * Returns true if it's appropriate for the bgwriter process to go into
+ * low-power hibernation mode.  (This happens if the strategy clock-sweep
+ * has been "lapped" and no buffer allocations have occurred recently,
+ * or if the bgwriter has been effectively disabled by setting
+ * bgwriter_lru_maxpages to 0.)
+ */
+bool
+BgBufferSync(WritebackContext *wb_context)
+{
+	/* info obtained from freelist.c */
+	uint32		recent_alloc;
+	uint32		recent_alloc_partition;
+	int			num_partitions;
+
+	/* assume we can hibernate, any partition can set to false */
+	bool		hibernate = true;
+
+	/* get the number of clocksweep partitions, and total alloc count */
+	StrategySyncPrepare(&num_partitions, &recent_alloc);
+
+	Assert(num_partitions <= MAX_CLOCKSWEEP_PARTITIONS);
+
+	/* Report buffer alloc counts to pgstat */
+	PendingBgWriterStats.buf_alloc += recent_alloc;
+
+	/* average alloc buffers per partition */
+	recent_alloc_partition = (recent_alloc / num_partitions);
+
+	/*
+	 * If we're not running the LRU scan, just stop after doing the stats
+	 * stuff.  We mark the saved state invalid so that we can recover sanely
+	 * if LRU scan is turned back on later.
+	 */
+	if (bgwriter_lru_maxpages <= 0)
+	{
+		saved_info_valid = false;
+		return true;
+	}
+
+	/*
+	 * now process the clocksweep partitions, one by one, using the same
+	 * cleanup that we used for all buffers
+	 *
+	 * XXX Maybe we should randomize the order of partitions a bit, so that we
+	 * don't start from partition 0 all the time? Perhaps not entirely, but at
+	 * least pick a random starting point?
+	 */
+	for (int partition = 0; partition < num_partitions; partition++)
+	{
+		/* hibernate if all partitions can hibernate */
+		hibernate &= BgBufferSyncPartition(wb_context, num_partitions,
+										   partition, recent_alloc_partition);
+	}
+
+	/* now that we've scanned all partitions, mark the cached info as valid */
+	saved_info_valid = true;
+
 	/* Return true if OK to hibernate */
-	return (bufs_to_lap == 0 && recent_alloc == 0);
+	return hibernate;
 }
 
 /*
diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index 7d59a92bd1a..d5f8f28f562 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -15,27 +15,47 @@
  */
 #include "postgres.h"
 
+#ifdef USE_LIBNUMA
+#include <sched.h>
+#endif
+
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
 #include "pgstat.h"

 /*
 
 	/*
 	 * clock-sweep hand: index of next buffer to consider grabbing. Note that
 
@@ -46,11 +66,34 @@ typedef struct
 	uint32		completePasses; /* Complete cycles of the clock-sweep */
 	pg_atomic_uint32 numBufferAllocs;	/* Buffers allocated since last reset */
 
+	/* running total of allocs */
+	pg_atomic_uint64 numTotalAllocs;
+
+} ClockSweep;
+
+/*
+ * The shared freelist control information.
+ */
+typedef struct
+{
+	/* Spinlock: protects the values below */
+	slock_t		buffer_strategy_lock;
+
 	/*
 	 * Bgworker process to be notified upon activity or -1 if none. See
 	 * StrategyNotifyBgWriter.
 	 */
 	int			bgwprocno;
+	// the _attribute_ does not work on Windows, it seems
+	//int			__attribute__((aligned(64))) bgwprocno;
+
+	/* info about freelist partitioning */
+	int			num_nodes;		/* effectively number of NUMA nodes */
+	int			num_partitions;
+	int			num_partitions_per_node;
+
+	/* clocksweep partitions */
+	ClockSweep	sweeps[FLEXIBLE_ARRAY_MEMBER];
 } BufferStrategyControl;
 
 /* Pointers to shared state */
@@ -89,6 +132,7 @@ static BufferDesc *GetBufferFromRing(BufferAccessStrategy strategy,
 									 uint32 *buf_state);
 static void AddBufferToRing(BufferAccessStrategy strategy,
 							BufferDesc *buf);
+static ClockSweep *ChooseClockSweep(void);
 
 /*
  * ClockSweepTick - Helper routine for StrategyGetBuffer()
@@ -100,6 +144,7 @@ static inline uint32
 ClockSweepTick(void)
 {
 	uint32		victim;
+	ClockSweep *sweep = ChooseClockSweep();
 
 	/*
 	 * Atomically move hand ahead one buffer - if there's several processes
@@ -107,14 +152,14 @@ ClockSweepTick(void)
 	 * apparent order.
 	 */
 	victim =
-		pg_atomic_fetch_add_u32(&StrategyControl->nextVictimBuffer, 1);
+		pg_atomic_fetch_add_u32(&sweep->nextVictimBuffer, 1);

 
 		/*
@@ -140,19 +185,117 @@ ClockSweepTick(void)

-				success = pg_atomic_compare_exchange_u32(&StrategyControl->nextVictimBuffer,
+				success = pg_atomic_compare_exchange_u32(&sweep->nextVictimBuffer,
 														 &expected, wrapped);
 				if (success)
-					StrategyControl->completePasses++;
-				SpinLockRelease(&StrategyControl->buffer_strategy_lock);
+					sweep->completePasses++;
+				SpinLockRelease(&sweep->clock_sweep_lock);
 			}
 		}
 	}
-	return victim;
+
+	/* XXX buffer IDs are 1-based, we're calculating 0-based indexes */
+	Assert(BufferIsValid(1 + sweep->firstBuffer + (victim % sweep->numBuffers)));
+
+	return sweep->firstBuffer + victim;
+}
+
+/*
+ * calculate_partition_index
+ *		calculate the buffer / clock-sweep partition to use
+ *
+ * With libnuma, use the NUMA node and CPU to pick the partition. Otherwise
+ * use just PID instead of CPU (we assume everything is a single NUMA node).
+ */
+static int
+calculate_partition_index(void)
+{
+	int		cpu,
+			node,
+			index;
+
+	/*
+	 * The buffers are partitioned, so determine the CPU/NUMA node, and pick a
+	 * partition based on that.
+	 *
+	 * Without NUMA assume everything is a single NUMA node, and we pick the
+	 * partition based on PID (we may not have sched_getcpu).
+	 */
+#ifdef USE_LIBNUMA
+	cpu = sched_getcpu();
+
+	if (cpu < 0)
+		elog(ERROR, "sched_getcpu failed: %m");
+
+	node = numa_node_of_cpu(cpu);
+#else
+	cpu = MyProcPid;
+	node = 0;
+#endif
+
+	Assert(StrategyControl->num_partitions ==
+		   (StrategyControl->num_nodes * StrategyControl->num_partitions_per_node));
+
+	/*
+	 * XXX We should't get nodes that we haven't considered while building the
+	 * partitions. Maybe if we allow this (e.g. due to support adjusting the
+	 * NUMA stuff at runtime), we should just do our best to minimize the
+	 * conflicts somehow. But it'll make the mapping harder, so for now we
+	 * ignore it.
+	 */
+	if (node > StrategyControl->num_nodes)
+		elog(ERROR, "node out of range: %d > %u", cpu, StrategyControl->num_nodes);
+
+	/*
+	 * Find the partition. If we have a single partition per node, we can
+	 * calculate the index directly from node. Otherwise we need to do two
+	 * steps, using node and then cpu.
+	 */
+	if (StrategyControl->num_partitions_per_node == 1)
+	{
+		/* fast-path */
+		index = (node % StrategyControl->num_partitions);
+	}
+	else
+	{
+		int			index_group,
+					index_part;
+
+		/* two steps - calculate group from node, partition from cpu */
+		index_group = (node % StrategyControl->num_nodes);
+		index_part = (cpu % StrategyControl->num_partitions_per_node);
+
+		index = (index_group * StrategyControl->num_partitions_per_node)
+			+ index_part;
+	}
+
+	return index;
+}
+
+/*
+ * ChooseClockSweep
+ *		pick a clocksweep partition based on NUMA node and CPU
+ *
+ * The number of clocksweep partitions may not match the number of NUMA
+ * nodes, but it should not be lower. Each partition should be mapped to
+ * a single NUMA node, but a node may have multiple partitions. If there
+ * are multiple partitions per node (all nodes have the same number of
+ * partitions), we pick the partition using CPU.
+ *
+ * XXX Maybe we should do both the total and "per group" counts a power of
+ * two? That'd allow using shifts instead of divisions in the calculation,
+ * and that's cheaper. But how would that deal with odd number of nodes?
+ */
+static ClockSweep *
+ChooseClockSweep(void)
+{
+	int			index = calculate_partition_index();
+
+	return &StrategyControl->sweeps[index];
 }
 
 /*
@@ -222,9 +365,35 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 
-	/* Use the "clock sweep" algorithm to find a free buffer */
+	/*
+	 * Use the "clock sweep" algorithm to find a free buffer
+	 *
+	 * XXX Note that ClockSweepTick() is NUMA-aware, i.e. it only looks at
+	 * buffers from a single partition, aligned with the NUMA node. That means
+	 * it only accesses buffers from the same NUMA node.
+	 *
+	 * XXX That also means each process "sweeps" only a fraction of buffers,
+	 * even if the other buffers are better candidates for eviction. Maybe
+	 * there should be some logic to "steal" buffers from other freelists or
+	 * other nodes?
+	 *
+	 * XXX Would that also mean we'd have multiple bgwriters, one for each
+	 * node, or would one bgwriter handle all of that?
+	 *
+	 * XXX This only searches a single partition, which can result in "no
+	 * unpinned buffers available" even if there are buffers in other
+	 * partitions. Should be fixed by falling back to other partitions if
+	 * needed.
+	 *
+	 * XXX Also, the trycounter should not be set to NBuffers, but to buffer
+	 * count for that one partition. In fact, this should not call ClockSweepTick
+	 * for every iteration. The call is likely quite expensive (does a lot
+	 * of stuff), and also may return a different partition on each call.
+	 * We should just do it once, and then do the for(;;) loop. And then
+	 * maybe advance to the next partition, until we scan through all of them.
+	 */
 	trycounter = NBuffers;
 	for (;;)
 	{
@@ -269,6 +438,46 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 	}
 }
 
+/*
+ * StrategySyncPrepare -- prepare for sync of all partitions
+ *
@@ -276,37 +485,44 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r

 	if (complete_passes)
 	{
 
 		/*
 
-	if (num_buf_alloc)
-	{
 
 /*
@@ -343,6 +559,9 @@ Size
 StrategyShmemSize(void)
 {
 	Size		size = 0;
+	int			num_partitions;
+
+	BufferPartitionParams(&num_partitions, NULL);
 
 	/* size of lookup hash table ... see comment in StrategyInitialize */
 	size = add_size(size, BufTableShmemSize(NBuffers + NUM_BUFFER_PARTITIONS));
@@ -350,6 +569,10 @@ StrategyShmemSize(void)
 	/* size of the shared replacement strategy control block */
 	size = add_size(size, MAXALIGN(sizeof(BufferStrategyControl)));

@@ -365,6 +588,18 @@ StrategyInitialize(bool init)
 {
 	bool		found;
 
+	int			num_nodes;
+	int			num_partitions;
+	int			num_partitions_per_node;
+
+	num_partitions = BufferPartitionCount();
+	num_nodes = BufferPartitionNodes();
+
+	/* always a multiple of NUMA nodes */
+	Assert(num_partitions % num_nodes == 0);
+
+	num_partitions_per_node = (num_partitions / num_nodes);
+
 	/*
 	 * Initialize the shared buffer lookup hashtable.
 	 *
@@ -382,7 +617,8 @@ StrategyInitialize(bool init)
 	 */
 	StrategyControl = (BufferStrategyControl *)
 		ShmemInitStruct("Buffer Strategy Status",
-						sizeof(BufferStrategyControl),
+						MAXALIGN(offsetof(BufferStrategyControl, sweeps)) +
+						MAXALIGN(sizeof(ClockSweep) * num_partitions),
 						&found);
 
 	if (!found)
@@ -394,15 +630,44 @@ StrategyInitialize(bool init)

-		/* Initialize the clock-sweep pointer */
 
-		/* Clear statistics */
 
@@ -739,3 +1004,23 @@ StrategyRejectBuffer(BufferAccessStrategy strategy, BufferDesc *buf, bool from_r
 
 	return true;
 }
+
+void
+ClockSweepPartitionGetInfo(int idx,
+						   uint32 *complete_passes, uint32 *next_victim_buffer,
+						   uint64 *buffer_total_allocs, uint32 *buffer_allocs)
+{
+	ClockSweep *sweep = &StrategyControl->sweeps[idx];
+
+	Assert((idx >= 0) && (idx < StrategyControl->num_partitions));
+
+	/* get the clocksweep stats */
+	*complete_passes = sweep->completePasses;
+	*next_victim_buffer = pg_atomic_read_u32(&sweep->nextVictimBuffer);
+
+	*buffer_allocs = pg_atomic_read_u32(&sweep->numBufferAllocs);
+	*buffer_total_allocs = pg_atomic_read_u64(&sweep->numTotalAllocs);
+
+	/* calculate the actual buffer ID */
+	*next_victim_buffer = sweep->firstBuffer + (*next_victim_buffer % sweep->numBuffers);
+}
diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h
index 294188e21c5..5cce690933b 100644
--- a/src/include/storage/buf_internals.h
+++ b/src/include/storage/buf_internals.h
@@ -439,7 +439,9 @@ extern BufferDesc *StrategyGetBuffer(BufferAccessStrategy strategy,
 extern bool StrategyRejectBuffer(BufferAccessStrategy strategy,
 								 BufferDesc *buf, bool from_ring);

 extern Size StrategyShmemSize(void);
@@ -485,5 +487,6 @@ extern int	BufferPartitionCount(void);
 extern int	BufferPartitionNodes(void);
 extern void BufferPartitionGet(int idx, int *node, int *num_buffers,
 							   int *first_buffer, int *last_buffer);
+extern void BufferPartitionParams(int *num_partitions, int *num_nodes);
 
 #endif							/* BUFMGR_INTERNALS_H */
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index e52fca9e483..9ade69e53b5 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -355,6 +355,11 @@ extern int	GetAccessStrategyBufferCount(BufferAccessStrategy strategy);
 extern int	GetAccessStrategyPinLimit(BufferAccessStrategy strategy);
 
+extern void ClockSweepPartitionGetInfo(int idx,

 /* inline functions */
diff --git a/src/test/recovery/t/027_stream_regress.pl b/src/test/recovery/t/027_stream_regress.pl
index 589c79d97d3..98b146ed4b7 100644
--- a/src/test/recovery/t/027_stream_regress.pl
+++ b/src/test/recovery/t/027_stream_regress.pl
@@ -18,6 +18,11 @@ $node_primary->adjust_conf('postgresql.conf', 'max_connections', '25');
 $node_primary->append_conf('postgresql.conf',
 	'max_prepared_transactions = 10');
 
+# The default is 1MB, which is not enough with clock-sweep partitioning.
+# Increase to 32MB, so that we don't get "no unpinned buffers".
+$node_primary->append_conf('postgresql.conf',
+	'shared_buffers = 32MB');
+
 # Enable pg_stat_statements to force tests to do query jumbling.
 # pg_stat_statements.max should be large enough to hold all the entries
 # of the regression database.
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 50195718294..b68f75b7f31 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -427,6 +427,7 @@ ClientCertName
 ClientConnectionInfo
 ClientData
 ClientSocket
+ClockSweep
 ClonePtrType
 ClosePortalStmt
 ClosePtrType
-- 
2.51.0

From 6a4444f319394ac88abe27faf4ccba22ea1a0dbe Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 10 Sep 2025 18:56:28 +0200
Subject: [PATCH v20250911 3/8] NUMA: clocksweep allocation balancing

This is very simple/cheap, and over many allocations it has the same
effect. For periods of low activity it may diverge, but that does not
matter much (we care about high-activity periods much more).
---
 .../pg_buffercache--1.6--1.7.sql              |   5 +-
 contrib/pg_buffercache/pg_buffercache_pages.c |  43 +-
 src/backend/storage/buffer/bufmgr.c           |   3 +
 src/backend/storage/buffer/freelist.c         | 377 +++++++++++++++++-
 src/include/storage/buf_internals.h           |   1 +
 src/include/storage/bufmgr.h                  |  12 +-
 6 files changed, 419 insertions(+), 22 deletions(-)

index 6676e807034..dc2ce019283 100644
@@ -22,7 +22,10 @@ CREATE VIEW pg_buffercache_partitions AS
 
index c9dfc8a1b82..f6831f60b9e 100644

@@ -27,7 +29,7 @@
 
 #define NUM_BUFFERCACHE_NUMA_ELEM	3
-#define NUM_BUFFERCACHE_PARTITIONS_ELEM	9
+#define NUM_BUFFERCACHE_PARTITIONS_ELEM	12
 
 PG_MODULE_MAGIC_EXT(
 					.name = "pg_buffercache",
@@ -795,6 +797,8 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 
 	if (SRF_IS_FIRSTCALL())
 	{
 
 		/* Switch context when allocating stuff to be used in later calls */
@@ -826,6 +830,12 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 						   INT8OID, -1, 0);
 		TupleDescInitEntry(tupledesc, (AttrNumber) 9, "num_allocs",
 						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 10, "total_req_allocs",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 11, "num_req_allocs",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 12, "weigths",
+						   typentry->typarray, -1, 0);

@@ -847,11 +857,17 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 					first_buffer,
 					last_buffer;

 		uint32		complete_passes,
 					next_victim_buffer,
-					buffer_allocs;
+					buffer_allocs,
+					buffer_req_allocs;
+
+		int		   *weights;
+		Datum	   *dweights;
+		ArrayType  *array;
 
 		Datum		values[NUM_BUFFERCACHE_PARTITIONS_ELEM];
 		bool		nulls[NUM_BUFFERCACHE_PARTITIONS_ELEM];
@@ -860,8 +876,16 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 						   &first_buffer, &last_buffer);
 
 		ClockSweepPartitionGetInfo(i,
-								   &complete_passes, &next_victim_buffer,
 
 		values[0] = Int32GetDatum(i);
 		nulls[0] = false;
@@ -890,6 +914,15 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		values[8] = Int64GetDatum(buffer_allocs);
 		nulls[8] = false;
 
+		values[9] = Int64GetDatum(buffer_total_req_allocs);
+		nulls[9] = false;
+
+		values[10] = Int64GetDatum(buffer_req_allocs);
+		nulls[10] = false;
+
+		values[11] = PointerGetDatum(array);
+		nulls[11] = false;
+
 		/* Build and return the tuple. */
 		tuple = heap_form_tuple((TupleDesc) funcctx->user_fctx, values, nulls);
 		result = HeapTupleGetDatum(tuple);
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 121134bb94c..8315105394d 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3884,6 +3884,9 @@ BgBufferSync(WritebackContext *wb_context)
 	/* assume we can hibernate, any partition can set to false */
 	bool		hibernate = true;

index d5f8f28f562..349b626db3b 100644
@@ -34,6 +34,23 @@

+/*
+ * XXX needed for make ClockSweep fixed-size, should be tied to the number
+ * of buffer partitions (bufmgr.c already has MAX_CLOCKSWEEP_PARTITIONS, so
+ * at least set it to the same value).
+ */
+#define MAX_BUFFER_PARTITIONS		32
+
+/*
+ * Coefficient used to combine the old and new balance coefficients, using
+ * weighted average. The higher the value, the more the old value affects the
+ * result.
+ *
+ * XXX Doesn't this invalidate the interpretation as a probability to allocate
+ * from a given partition? Does it still sum to 100%?
+ */
+#define CLOCKSWEEP_HISTORY_COEFF	0.5
+
 /*
  * Information about one partition of the ClockSweep (on a subset of buffers).
  *
@@ -66,9 +83,28 @@ typedef struct
 	uint32		completePasses; /* Complete cycles of the clock-sweep */
 	pg_atomic_uint32 numBufferAllocs;	/* Buffers allocated since last reset */
 
+	/*
 
+	/*
 
 /*
@@ -132,7 +168,33 @@ static BufferDesc *GetBufferFromRing(BufferAccessStrategy strategy,
 									 uint32 *buf_state);
 
 /*
  * ClockSweepTick - Helper routine for StrategyGetBuffer()
@@ -144,7 +206,7 @@ static inline uint32
 ClockSweepTick(void)
 {
 	uint32		victim;
-	ClockSweep *sweep = ChooseClockSweep();
+	ClockSweep *sweep = ChooseClockSweep(true);
 
 	/*
@@ -291,11 +353,59 @@ calculate_partition_index(void)

 /*
@@ -365,7 +475,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 
 	/*
 	 * Use the "clock sweep" algorithm to find a free buffer
@@ -438,6 +548,224 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 	}
 }
 
+/*
+ * StrategySyncBalance
  * StrategySyncPrepare -- prepare for sync of all partitions
@@ -464,6 +792,7 @@ StrategySyncPrepare(int *num_parts, uint32 *num_buf_alloc)
 
@@ -658,7 +987,21 @@ StrategyInitialize(bool init)
 
 		/* No pending notification */
@@ -1007,8 +1350,10 @@ StrategyRejectBuffer(BufferAccessStrategy strategy, BufferDesc *buf, bool from_r
 
 void
 ClockSweepPartitionGetInfo(int idx,
-						   uint32 *complete_passes, uint32 *next_victim_buffer,
-						   uint64 *buffer_total_allocs, uint32 *buffer_allocs)
+						 uint32 *complete_passes, uint32 *next_victim_buffer,
+						 uint64 *buffer_total_allocs, uint32 *buffer_allocs,
+						 uint64 *buffer_total_req_allocs, uint32 *buffer_req_allocs,
+						 int **weights)
 {
 	ClockSweep *sweep = &StrategyControl->sweeps[idx];
 
@@ -1016,11 +1361,21 @@ ClockSweepPartitionGetInfo(int idx,
 
 	/* get the clocksweep stats */

index 5cce690933b..1b4dae180e0 100644
@@ -439,6 +439,7 @@ extern BufferDesc *StrategyGetBuffer(BufferAccessStrategy strategy,
 
+extern void StrategySyncBalance(void);
 extern void StrategySyncPrepare(int *num_parts, uint32 *num_buf_alloc);
 extern int	StrategySyncStart(int partition, uint32 *complete_passes,
 							  int *first_buffer, int *num_buffers);
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index 9ade69e53b5..fb4162eb764 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -356,11 +356,13 @@ extern int	GetAccessStrategyPinLimit(BufferAccessStrategy strategy);
 
 extern void FreeAccessStrategy(BufferAccessStrategy strategy);
 extern void ClockSweepPartitionGetInfo(int idx,
-									   uint32 *complete_passes,
-									   uint32 *next_victim_buffer,
-									   uint64 *buffer_total_allocs,
-									   uint32 *buffer_allocs);
-
+									 uint32 *complete_passes,
+									 uint32 *next_victim_buffer,
+									 uint64 *buffer_total_allocs,
+									 uint32 *buffer_allocs,
+									 uint64 *buffer_total_req_allocs,
+									 uint32 *buffer_req_allocs,
+									 int **weights);
 
 /* inline functions */
 
-- 
2.51.0

From 391a25e8cbe88e9a018a5123b3d40ffec607891e Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 6 Aug 2025 01:09:57 +0200
Subject: [PATCH v20250911 4/8] NUMA: weighted clocksweep balancing

diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index 349b626db3b..5cf9a565914 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -605,6 +605,20 @@ StrategySyncBalance(void)
 			avg_allocs,			/* average allocations (per partition) */
 			delta_allocs = 0;	/* sum of allocs above average */
 
+	/*
@@ -631,16 +645,27 @@ StrategySyncBalance(void)

 	/*
 
 	/*
@@ -648,8 +673,14 @@ StrategySyncBalance(void)
 
 	/*
@@ -712,6 +743,10 @@ StrategySyncBalance(void)
 		ClockSweep *sweep = &StrategyControl->sweeps[i];
 		uint8		balance[MAX_BUFFER_PARTITIONS];

@@ -719,7 +754,7 @@ StrategySyncBalance(void)
 		memset(balance, 0, sizeof(uint8) * MAX_BUFFER_PARTITIONS);
 
@@ -733,22 +768,30 @@ StrategySyncBalance(void)

 

2.51.0

From 4cdfff63d1a903902ce182b7e723ff6d2d09a84e Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Mon, 8 Sep 2025 13:11:02 +0200
Subject: [PATCH v20250911 5/8] NUMA: partition PGPROC

 - PGPROC array / node #0
 - PGPROC array / node #1

Note: There's some challenges in making this work on EXEC_BACKEND, even
if we don't support NUMA on platforms that require this.
---
 .../pg_buffercache--1.6--1.7.sql              |  19 +
 contrib/pg_buffercache/pg_buffercache_pages.c |  94 ++++
 src/backend/access/transam/clog.c             |   4 +-
 src/backend/access/transam/twophase.c         |   3 +-
 src/backend/postmaster/launch_backend.c       |   4 +-
 src/backend/postmaster/pgarch.c               |   2 +-
 src/backend/postmaster/walsummarizer.c        |   2 +-
 src/backend/storage/buffer/buf_init.c         |   2 +
 src/backend/storage/buffer/freelist.c         |   2 +-
 src/backend/storage/ipc/procarray.c           |  85 +--
 src/backend/storage/lmgr/lock.c               |   6 +-
 src/backend/storage/lmgr/proc.c               | 532 +++++++++++++++++-
 src/include/port/pg_numa.h                    |   1 +
 src/include/storage/proc.h                    |  18 +-
 src/tools/pgindent/typedefs.list              |   1 +
 15 files changed, 705 insertions(+), 70 deletions(-)

diff --git a/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql b/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
index dc2ce019283..306063e159e 100644
--- a/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
+++ b/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
@@ -33,3 +33,22 @@ REVOKE ALL ON pg_buffercache_partitions FROM PUBLIC;
 
index f6831f60b9e..a859962f5f8 100644
 
 #define NUM_BUFFERCACHE_NUMA_ELEM	3
 #define NUM_BUFFERCACHE_PARTITIONS_ELEM	12
+#define NUM_BUFFERCACHE_PGPROC_ELEM	5
 
 PG_MODULE_MAGIC_EXT(

@@ -932,3 +935,94 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 
 		/*
 
 		wakeidx = pg_atomic_read_u32(&wakeproc->clogGroupNext);
 		pg_atomic_write_u32(&wakeproc->clogGroupNext, INVALID_PROC_NUMBER);
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index d8e2fce2c99..7745a197470 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -282,7 +282,7 @@ TwoPhaseShmemInit(void)
 			TwoPhaseState->freeGXacts = &gxacts[i];
 
 			/* associate it with a PGPROC assigned by InitProcGlobal */
-			gxacts[i].pgprocno = GetNumberFromPGProc(&PreparedXactProcs[i]);
+			gxacts[i].pgprocno = GetNumberFromPGProc(PreparedXactProcs[i]);
 		}
 	}
 	else
@@ -447,6 +447,7 @@ MarkAsPreparingGuts(GlobalTransaction gxact, FullTransactionId fxid,
 
 	/* Initialize the PGPROC entry */
 	MemSet(proc, 0, sizeof(PGPROC));
+	proc->procnumber = gxact->pgprocno;
 	dlist_node_init(&proc->links);
 	proc->waitStatus = PROC_WAIT_STATUS_OK;
 	if (LocalTransactionIdIsValid(MyProc->vxid.lxid))
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index a38979c50e4..01356365eaa 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -106,8 +106,8 @@ typedef struct
 	LWLockPadded *MainLWLockArray;
 	slock_t    *ProcStructLock;
 	PROC_HDR   *ProcGlobal;
-	PGPROC	   *AuxiliaryProcs;
-	PGPROC	   *PreparedXactProcs;
+	PGPROC	   **AuxiliaryProcs;
+	PGPROC	   **PreparedXactProcs;
 	volatile PMSignalData *PMSignalState;
 	ProcSignalHeader *ProcSignal;
 	pid_t		PostmasterPid;
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 78e39e5f866..e28e0f7d3bd 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -289,7 +289,7 @@ PgArchWakeup(void)
 	 * be relaunched shortly and will start archiving.
 	 */
 	if (arch_pgprocno != INVALID_PROC_NUMBER)
-		SetLatch(&ProcGlobal->allProcs[arch_pgprocno].procLatch);
+		SetLatch(&ProcGlobal->allProcs[arch_pgprocno]->procLatch);
 }

index e1f142f20c7..011fecfc58b 100644

 /*
diff --git a/src/backend/storage/buffer/buf_init.c b/src/backend/storage/buffer/buf_init.c
index dd9f51529b4..2fd7f937ffb 100644
--- a/src/backend/storage/buffer/buf_init.c
+++ b/src/backend/storage/buffer/buf_init.c
@@ -766,6 +766,8 @@ check_debug_numa(char **newval, void **extra, GucSource source)
 
 		if (pg_strcasecmp(item, "buffers") == 0)
 			flags |= NUMA_BUFFERS;
+		else if (pg_strcasecmp(item, "procs") == 0)
+			flags |= NUMA_PROCS;
 		else
 		{
 			GUC_check_errdetail("Invalid option \"%s\".", item);
diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index 5cf9a565914..a6657c0fc13 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -467,7 +467,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 		 * actually fine because procLatch isn't ever freed, so we just can
 		 * potentially set the wrong process' (or no process') latch.
 		 */
-		SetLatch(&ProcGlobal->allProcs[bgwprocno].procLatch);
+		SetLatch(&ProcGlobal->allProcs[bgwprocno]->procLatch);
 	}
 
 	/*
index 200f72c6e25..7e28fbdfea3 100644

 
 /*
  * Cache to reduce overhead of repeated calls to TransactionIdIsInProgress()
@@ -369,6 +369,8 @@ static inline FullTransactionId FullXidRelativeTo(FullTransactionId rel,
 												  TransactionId xid);
 static void GlobalVisUpdateApply(ComputeXidHorizonsResult *horizons);
 
+static void AssertCheckAllProcs(void);
+
 /*
  * Report shared-memory space needed by ProcArrayShmemInit
  */
@@ -476,6 +478,8 @@ ProcArrayAdd(PGPROC *proc)
 	LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE);
 	LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
 
+	AssertCheckAllProcs();
+
 	if (arrayP->numProcs >= arrayP->maxProcs)
 	{
 		/*
@@ -502,7 +506,7 @@ ProcArrayAdd(PGPROC *proc)
 		int			this_procno = arrayP->pgprocnos[index];

 		/* If we have found our right position in the array, break */
 		if (this_procno > pgprocno)
@@ -538,11 +542,13 @@ ProcArrayAdd(PGPROC *proc)
 		int			procno = arrayP->pgprocnos[index];

 
+	AssertCheckAllProcs();
+
 	/*
 	 * Release in reversed acquisition order, to reduce frequency of having to
 	 * wait for XidGenLock while holding ProcArrayLock.
@@ -578,10 +584,12 @@ ProcArrayRemove(PGPROC *proc, TransactionId latestXid)
 	LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE);
 	LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
 
+	AssertCheckAllProcs();
+
 	myoff = proc->pgxactoff;

 	if (TransactionIdIsValid(latestXid))
 	{
@@ -636,11 +644,13 @@ ProcArrayRemove(PGPROC *proc, TransactionId latestXid)
 		int			procno = arrayP->pgprocnos[index];

 
+	AssertCheckAllProcs();
+
 	/*
 	 * Release in reversed acquisition order, to reduce frequency of having to
 	 * wait for XidGenLock while holding ProcArrayLock.
@@ -860,7 +870,7 @@ ProcArrayGroupClearXid(PGPROC *proc, TransactionId latestXid)
 	/* Walk the list and clear all XIDs. */
 	while (nextidx != INVALID_PROC_NUMBER)
 	{
-		PGPROC	   *nextproc = &allProcs[nextidx];
+		PGPROC	   *nextproc = allProcs[nextidx];

@@ -880,7 +890,7 @@ ProcArrayGroupClearXid(PGPROC *proc, TransactionId latestXid)
 	 */
 	while (wakeidx != INVALID_PROC_NUMBER)
 	{
-		PGPROC	   *nextproc = &allProcs[wakeidx];
+		PGPROC	   *nextproc = allProcs[wakeidx];
 
@@ -1526,7 +1536,7 @@ TransactionIdIsInProgress(TransactionId xid)
@@ -1622,7 +1632,6 @@ TransactionIdIsInProgress(TransactionId xid)
 	return false;
 }
 
-
 /*
  * Determine XID horizons.
  *
@@ -1740,7 +1749,7 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h)
 	for (int index = 0; index < arrayP->numProcs; index++)
 	{
 		int			pgprocno = arrayP->pgprocnos[index];
-		PGPROC	   *proc = &allProcs[pgprocno];
+		PGPROC	   *proc = allProcs[pgprocno];
 		int8		statusFlags = ProcGlobal->statusFlags[index];
 		TransactionId xid;
 		TransactionId xmin;
@@ -2224,7 +2233,7 @@ GetSnapshotData(Snapshot snapshot)
 			TransactionId xid = UINT32_ACCESS_ONCE(other_xids[pgxactoff]);
 			uint8		statusFlags;

 			/*
@@ -2298,7 +2307,7 @@ GetSnapshotData(Snapshot snapshot)

@@ -2499,7 +2508,7 @@ ProcArrayInstallImportedXmin(TransactionId xmin,
 
@@ -2725,7 +2734,7 @@ GetRunningTransactionData(void)
 
@@ -2756,7 +2765,7 @@ GetRunningTransactionData(void)
 
 			/*
@@ -2858,7 +2867,7 @@ GetOldestActiveTransactionId(bool inCommitOnly, bool allDbs)
 	{
 		TransactionId xid;
 		int			pgprocno = arrayP->pgprocnos[index];
-		PGPROC	   *proc = &allProcs[pgprocno];
+		PGPROC	   *proc = allProcs[pgprocno];
 
@@ -3020,7 +3029,7 @@ GetVirtualXIDsDelayingChkpt(int *nvxids, int type)
 
@@ -3061,7 +3070,7 @@ HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids, int type)
 
 		GET_VXID_FROM_PGPROC(vxid, *proc);
@@ -3189,7 +3198,7 @@ BackendPidGetProcWithLock(int pid)

 		if (proc->pid == pid)
 		{
@@ -3232,7 +3241,7 @@ BackendXidGetPid(TransactionId xid)
 
 			result = proc->pid;
 			break;
@@ -3301,7 +3310,7 @@ GetCurrentVirtualXIDs(TransactionId limitXmin, bool excludeXmin0,
 
 		if (proc == MyProc)
@@ -3403,7 +3412,7 @@ GetConflictingVirtualXIDs(TransactionId limitXmin, Oid dbOid)
 
@@ -3468,7 +3477,7 @@ SignalVirtualTransaction(VirtualTransactionId vxid, ProcSignalReason sigmode,
 
@@ -3523,7 +3532,7 @@ MinimumActiveBackends(int min)
 
 		/*
@@ -3569,7 +3578,7 @@ CountDBBackends(Oid databaseid)
 
 		if (proc->pid == 0)
@@ -3598,7 +3607,7 @@ CountDBConnections(Oid databaseid)
 
 		if (proc->pid == 0)
@@ -3629,7 +3638,7 @@ CancelDBBackends(Oid databaseid, ProcSignalReason sigmode, bool conflictPending)
 
@@ -3670,7 +3679,7 @@ CountUserBackends(Oid roleid)
 
 		if (proc->pid == 0)
@@ -3733,7 +3742,7 @@ CountOtherDBBackends(Oid databaseId, int *nbackends, int *nprepared)
 
@@ -3799,7 +3808,7 @@ TerminateOtherDBBackends(Oid databaseId)
 
 		if (proc->databaseId != databaseId)
 			continue;
@@ -5227,3 +5236,15 @@ KnownAssignedXidsReset(void)
 
 	LWLockRelease(ProcArrayLock);
 }
+
+static void
+AssertCheckAllProcs(void)
+{
+	ProcArrayStruct *arrayP = procArray;
+	int		numProcs = arrayP->numProcs;
+
+	for (int pgxactoff = 0; pgxactoff < numProcs; pgxactoff++)
+	{
+		Assert(allProcs[arrayP->pgprocnos[pgxactoff]]->pgxactoff == pgxactoff);
+	}
+}
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index 4cc7f645c31..d01f486876d 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -2876,7 +2876,7 @@ FastPathTransferRelationLocks(LockMethod lockMethodTable, const LOCKTAG *locktag
 	 */
 	for (i = 0; i < ProcGlobal->allProcCount; i++)
 	{
-		PGPROC	   *proc = &ProcGlobal->allProcs[i];
+		PGPROC	   *proc = ProcGlobal->allProcs[i];
 		uint32		j;
 
@@ -3135,7 +3135,7 @@ GetLockConflicts(const LOCKTAG *locktag, LOCKMODE lockmode, int *countp)
 
@@ -3822,7 +3822,7 @@ GetLockStatusData(void)
 
index e9ef0fbfe32..08d1900fb59 100644
@@ -29,21 +29,32 @@
  */
 #include "postgres.h"
 
+#ifdef USE_LIBNUMA
+#include <sched.h>
+#endif
+
 #include <signal.h>
 #include <unistd.h>
 #include <sys/time.h>
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
@@ -76,8 +87,8 @@ NON_EXEC_STATIC slock_t *ProcStructLock = NULL;
 
 /* Pointers to shared-memory structures */
 PROC_HDR   *ProcGlobal = NULL;
-NON_EXEC_STATIC PGPROC *AuxiliaryProcs = NULL;
-PGPROC	   *PreparedXactProcs = NULL;
+NON_EXEC_STATIC PGPROC **AuxiliaryProcs = NULL;
+PGPROC	   **PreparedXactProcs = NULL;
 
 static DeadLockState deadlock_state = DS_NOT_YET_CHECKED;
 
@@ -90,6 +101,29 @@ static void AuxiliaryProcKill(int code, Datum arg);
 static void CheckDeadLock(void);

@@ -100,11 +134,36 @@ PGProcShmemSize(void)
 
+	size = add_size(size, CACHELINEALIGN(mul_size(TotalProcs, sizeof(PGPROC *))));
 
+	/*
 
@@ -129,6 +188,60 @@ FastPathLockShmemSize(void)

+	/*
+	if (((numa_flags & NUMA_PROCS) != 0) && numa_can_partition)
+	return size;
+PGProcPartitionsShmemSize(void)
+	Size		size = 0;
+	if (((numa_flags & NUMA_PROCS) != 0) && numa_can_partition)
 
@@ -140,12 +253,16 @@ ProcGlobalShmemSize(void)
 {
 	Size		size = 0;

 	size = add_size(size, PGProcShmemSize());
 	size = add_size(size, FastPathLockShmemSize());
+	size = add_size(size, PGProcPartitionsShmemSize());
 
 	return size;
 }
@@ -191,7 +308,7 @@ ProcGlobalSemas(void)
 void
 InitProcGlobal(void)
 {
-	PGPROC	   *procs;
+	PGPROC	  **procs;
 	int			i,
 				j;
 	bool		found;
@@ -210,6 +327,9 @@ InitProcGlobal(void)
 		ShmemInitStruct("Proc Header", sizeof(PROC_HDR), &found);
 	Assert(!found);
 
+	/* XXX call again, EXEC_BACKEND may not see the already computed value */
+	pgproc_partitions_prepare();
+
 	/*
 	 * Initialize the data structures.
 	 */
@@ -224,6 +344,15 @@ InitProcGlobal(void)
 	pg_atomic_init_u32(&ProcGlobal->procArrayGroupFirst, INVALID_PROC_NUMBER);
 	pg_atomic_init_u32(&ProcGlobal->clogGroupFirst, INVALID_PROC_NUMBER);
 
+	/* PGPROC partition registry */
+	requestSize = PGProcPartitionsShmemSize();
+
+	ptr = ShmemInitStruct("PGPROC partitions",
+						  requestSize,
+						  &found);
+
+	partitions = (PGProcPartition *) ptr;
+
 	/*
 	 * Create and initialize all the PGPROC structures we'll need.  There are
 	 * six separate consumers: (1) normal backends, (2) autovacuum workers and
@@ -241,19 +370,104 @@ InitProcGlobal(void)

+	ptr = (char *) ptr + CACHELINEALIGN(TotalProcs * sizeof(PGPROC *));

+	/*
+	if (((numa_flags & NUMA_PROCS) != 0) && numa_can_partition)
+		 * Now initialize the PGPROC partition registry with one partition
+		 * per NUMA node (and then one extra partition for auxiliary procs).
@@ -286,24 +500,92 @@ InitProcGlobal(void)
 
+	if (((numa_flags & NUMA_PROCS) != 0) && numa_can_partition)

 		/*

 
@@ -366,9 +648,6 @@ InitProcGlobal(void)
 
@@ -435,7 +714,51 @@ InitProcess(void)
 
 	if (!dlist_is_empty(procgloballist))
 	{
-		MyProc = dlist_container(PGPROC, links, dlist_pop_head_node(procgloballist));
+		/*
+		 * With numa interleaving of PGPROC, try to get a PROC entry from the
+		 * right NUMA node (when the process starts).
+		 *
+		 * XXX The process may move to a different NUMA node later, but
+		 * there's not much we can do about that.
+		 */
+		if ((numa_flags & NUMA_PROCS) != 0)
+		{
+			dlist_mutable_iter iter;
+			int		node;
+
+#ifdef USE_LIBNUMA
+			int	cpu = sched_getcpu();
+
+			if (cpu < 0)
+				elog(ERROR, "getcpu failed: %m");
+
+			node = numa_node_of_cpu(cpu);
+#else
+			/* FIXME is defaulting to 0 correct? */
+			node = 0;
+#endif
+
+			MyProc = NULL;
+
+			dlist_foreach_modify(iter, procgloballist)
+			{
+				PGPROC	   *proc;
+
+				proc = dlist_container(PGPROC, links, iter.cur);
+
+				if (proc->numa_node == node)
+				{
+					MyProc = proc;
+					dlist_delete(iter.cur);
+					break;
+				}
+			}
+		}
+
+		/* didn't find PGPROC from the correct NUMA node, pick any free one */
+		if (MyProc == NULL)
+			MyProc = dlist_container(PGPROC, links, dlist_pop_head_node(procgloballist));
+
 		SpinLockRelease(ProcStructLock);
 	}
 	else
@@ -646,7 +969,7 @@ InitAuxiliaryProcess(void)
 	 */
 	for (proctype = 0; proctype < NUM_AUXILIARY_PROCS; proctype++)
 	{
-		auxproc = &AuxiliaryProcs[proctype];
+		auxproc = AuxiliaryProcs[proctype];
 		if (auxproc->pid == 0)
 			break;
 	}
@@ -1049,7 +1372,7 @@ AuxiliaryProcKill(int code, Datum arg)
 	if (MyProc->pid != (int) getpid())
 		elog(PANIC, "AuxiliaryProcKill() called in child process");
 
-	auxproc = &AuxiliaryProcs[proctype];
+	auxproc = AuxiliaryProcs[proctype];
 
 	Assert(MyProc == auxproc);
 
@@ -1098,7 +1421,7 @@ AuxiliaryPidGetProc(int pid)
 
 	for (index = 0; index < NUM_AUXILIARY_PROCS; index++)
 	{
-		PGPROC	   *proc = &AuxiliaryProcs[index];
+		PGPROC	   *proc = AuxiliaryProcs[index];
 
 		if (proc->pid == pid)
 		{
@@ -1988,7 +2311,7 @@ ProcSendSignal(ProcNumber procNumber)
 	if (procNumber < 0 || procNumber >= ProcGlobal->allProcCount)
 		elog(ERROR, "procNumber out of range");

 /*
@@ -2063,3 +2386,168 @@ BecomeLockGroupMember(PGPROC *leader, int pid)
 
 	return ok;
 }
+
+/*
+	// Assert(!IsUnderPostmaster);
+	numa_page_size = pg_numa_page_size();
+	elog(DEBUG1, "NUMA: pgproc backends %d num_nodes %d per_node %d",
+	elog(DEBUG1, "NUMA: pgproc_init_partition procs %p endptr %p num_procs %d node %d",
+	if (((numa_flags & NUMA_PROCS) != 0) && numa_can_partition)
diff --git a/src/include/port/pg_numa.h b/src/include/port/pg_numa.h
index 9734aa315ff..aa524f6f7f3 100644
--- a/src/include/port/pg_numa.h
+++ b/src/include/port/pg_numa.h
@@ -23,6 +23,7 @@ extern PGDLLIMPORT void pg_numa_move_to_node(char *startptr, char *endptr, int n
 extern PGDLLIMPORT int numa_flags;
 
 #define		NUMA_BUFFERS		0x01
+#define		NUMA_PROCS			0x02
 
 #ifdef USE_LIBNUMA
 
index c6f5ebceefd..21f2619fd40 100644

 
 
 	/* Array mirroring PGPROC.xid for each PGPROC currently in the procarray */
 	TransactionId *xids;
@@ -438,13 +443,13 @@ typedef struct PROC_HDR
 
 extern PGDLLIMPORT PROC_HDR *ProcGlobal;
 
-extern PGDLLIMPORT PGPROC *PreparedXactProcs;
+extern PGDLLIMPORT PGPROC **PreparedXactProcs;
 
 /*
 
 /*
  * We set aside some extra PGPROC structures for "special worker" processes,
@@ -480,7 +485,7 @@ extern PGDLLIMPORT bool log_lock_waits;
 
 #ifdef EXEC_BACKEND
 extern PGDLLIMPORT slock_t *ProcStructLock;
-extern PGDLLIMPORT PGPROC *AuxiliaryProcs;
+extern PGDLLIMPORT PGPROC **AuxiliaryProcs;
 #endif

index b68f75b7f31..6a970998e5c 100644
2.51.0

---

From: Tomas Vondra <[email-masked]>
Date: Thu, 11 Sep 2025 17:41:23 +0200

> to not need more memory than we requested.

I forgot to mention the other issue with huge pages on NUMA. I already
reported [1] it's trivial to crash with a SIGBUS, because

(1) huge pages get reserved on all NUMA nodes (evenly)

(2) the decision whether to use huge pages is done by mmap(), which only
needs to check if there are enough huge pages in total

(3) numa_tonode_memory is called later, and does not verify if the
target node has enough free pages (I'm not sure it should / can)

(4) we only partition (and locate to NUMA nodes) some of the memory, and
the rest (which is much smaller, but still sizeable) is likely causing
"imbalance" - it gets placed on one (random) node, and it then does not
have enough space for the stuff we explicitly placed there

(5) then at some point we try accessing one of the shared buffers, that
triggers page fault, tries to get a huge page on the NUMA node, realizes
there are no free huge pages, and crashes with SIGBUS

It clearly is not an option to just let it crash, but I still don't have
a great idea how to address it. The only idea I have is to manually
interleave the whole shared memory (when using huge pages), page by
page, so that this imbalance does not happen.

But it's harder than it looks, because we don't necessarily partition
everything evenly. For example, one node can get a smaller chunk of
shared buffers, because we try to partition buffers and buffers
descriptors in a "nice" way. The PGPROC stuff is also not distributed
quite evenly (e.g. aux/2pc entries are not mapped to any node).

A different approach would be to calculate how many per-node huge pages
we'll need (for the stuff we partition explicitly - buffers and PGPROC),
and then the rest of the memory that can get placed on any node. And
require the "maximum" number of pages that can get placed on any node.
But that's annoying wasteful, because every other node will end up with
unusable memory.

regards

[1]
https://www.postgresql.org/message-id/71a46484-053c-4b81-ba32-ddac050a8b5d%40vondra.me

-- 
Tomas Vondra

---

From: Tomas Vondra <[email-masked]>
Date: Thu, 18 Sep 2025 23:04:45 +0200

> it used NBuffers instead of the partition size. I'll fix those later.

Here's a version fixing this issue (in the 0006 part). It modifies
StrategyGetBuffer() to walk through all the partitions, in a round-robin
manner. The way it steps to the next partition is a bit ugly, but it
works and I'll think about some better way.

I haven't done anything about the other issue (the one with huge pages
reserved on NUMA nodes, and SIGBUS).

regards

-- 
Tomas Vondra

From 9141bb89bce485873978e1ae3988b43e675ee047 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 17 Sep 2025 23:04:29 +0200
Subject: [PATCH v20250918 1/6] NUMA: shared buffers partitioning

The effect is similar to

     numactl --interleave=all

Notes:

 

 PG_MODULE_MAGIC_EXT(

 	PG_RETURN_DATUM(result);
 }
+
+/*
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+

+/*
 
 /*
  * Data Structures:

 
 
+		/*

 
 
index 6bc6be13d2a..65bef2b3c1b 100644

 #ifdef HAVE_SYSLOG
 
 #ifdef USE_LIBNUMA

 
 /* in buf_init.c */

index 47360a3d3d8..1618caf1c2c 100644
@@ -146,6 +146,28 @@ struct ReadBuffersOperation

+/*
 /* to avoid having to expose buf_internals.h here */
 typedef struct WritebackContext WritebackContext;

 /* in localbuf.c */

 #include "miscadmin.h"
 
 /*
 
+/*
+ * pg_numa_move_to_node
 
 /* Empty wrappers */
 
+void
index e90af5b2ad3..b3f0504008e 100644

From e7145b7e771487012aa0c723309353723db93172 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Sun, 8 Jun 2025 18:53:12 +0200
Subject: [PATCH v20250918 2/6] NUMA: clockweep partitioning

 

 PG_MODULE_MAGIC_EXT(

 

 

 
 /*

 
 	/*

 

 	/*
 
 	/*

 

 #ifdef BGW_DEBUG

 /*
 
+#ifdef USE_LIBNUMA
+#include <sched.h>
+#endif
+
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
 #include "pgstat.h"

 /*
 
 	/*

 
 
 /*
 
 	/*

 
 		/*

 
 /*

+/*

 	if (complete_passes)
 	{
 
 		/*
 
-	if (num_buf_alloc)
-	{
 
 /*
@@ -343,6 +559,9 @@ Size
 StrategyShmemSize(void)
 {
 	Size		size = 0;
+	int			num_partitions;
+

 
 
+	int			num_nodes;
+	int			num_partitions;
 
 	if (!found)

 
-		/* Clear statistics */

 	return true;
 }
+
+void

 
index 1618caf1c2c..7d66e8276cd 100644

 /* inline functions */
 
index b3f0504008e..2b7ce2d0b9f 100644

From 1707ae15be8f0311219c4ba84b15a41d9bde6c69 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 10 Sep 2025 18:56:28 +0200
Subject: [PATCH v20250918 3/6] NUMA: clocksweep allocation balancing

 

@@ -27,7 +29,7 @@

 PG_MODULE_MAGIC_EXT(
 
 	if (SRF_IS_FIRSTCALL())
 	{

 

 		uint32		complete_passes,
 					next_victim_buffer,
-					buffer_allocs;
+					buffer_allocs,
+					buffer_req_allocs;
+
+		int		   *weights;
+		Datum	   *dweights;
+		ArrayType  *array;

 

 

+/*
 
+	/*
 
+	/*
 
 /*
 
 /*
 
 	/*

 /*
 
 	/*
 
+/*
+ * StrategySyncBalance

 
 void

 

index 7d66e8276cd..1ca0d4c375e 100644

 /* inline functions */
 
-- 
2.51.0

From 5631cd15f07c368153014464c4ce58ab0838a317 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 6 Aug 2025 01:09:57 +0200
Subject: [PATCH v20250918 4/6] NUMA: weighted clocksweep balancing

 
+	/*

 	/*
 
 	/*
 
 	/*

 

 

From 5247bf1c147ec9593e72168fec3579910b4f262c Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Mon, 8 Sep 2025 13:11:02 +0200
Subject: [PATCH v20250918 5/6] NUMA: partition PGPROC

 - PGPROC array / node #0
 - PGPROC array / node #1

 

 PG_MODULE_MAGIC_EXT(

 		/*

 
index 79708e59259..92db67fa5f9 100644
@@ -107,8 +107,8 @@ typedef struct

 
 /*

 	/*

 
 /*

+	AssertCheckAllProcs();
+

 

+	AssertCheckAllProcs();
+
 	/*
 
+	AssertCheckAllProcs();
+
 	myoff = proc->pgxactoff;

 

+	AssertCheckAllProcs();
+
 	/*

 
 
-
 /*

 			/*

 

 			/*

 

 		if (proc->pid == pid)
 		{
 
 			result = proc->pid;
 			break;
 
 		if (proc == MyProc)

 
 		/*
 
 		if (proc->pid == 0)
 
 		if (proc->pid == 0)

 		if (proc->pid == 0)

 

 
index 96f29aafc39..70ccfebef55 100644
 
+#ifdef USE_LIBNUMA
+#include <sched.h>
+#endif
+
 #include <signal.h>
 #include <unistd.h>
 #include <sys/time.h>
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+

 

 
+	/*

 
+	/*

 
 
 	return size;
 }

 

 
+	/*

 
 		/*

 

 

 
 
 		if (proc->pid == pid)
 		{

 /*
 
 	return ok;
 }
+
+/*

 #ifdef USE_LIBNUMA

 

 

 /*
 
 /*
 
 #ifdef EXEC_BACKEND

index 2b7ce2d0b9f..912dd4ac100 100644

From e78f37f4769225522ec4ce828fbc3a3e46bcdf6e Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 17 Sep 2025 23:42:34 +0200
Subject: [PATCH v20250918 6/6] fixup: StrategyGetBuffer

StrategyGetBuffer is expected to scan all clock-sweep partitions, not
just the "local" one. So start at the optimal one (as calculated by
ChooseClockSweep), and then advance to the next one in a round-robin
way, until we find a free / unpinned buffer.
---
 src/backend/storage/buffer/freelist.c     | 94 ++++++++++++++++-------
 src/test/recovery/t/027_stream_regress.pl |  5 --
 2 files changed, 65 insertions(+), 34 deletions(-)

diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index a6657c0fc13..d911986d7ce 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -169,6 +169,9 @@ static BufferDesc *GetBufferFromRing(BufferAccessStrategy strategy,
 static void AddBufferToRing(BufferAccessStrategy strategy,
 							BufferDesc *buf);
 static ClockSweep *ChooseClockSweep(bool balance);
+static BufferDesc *StrategyGetBufferPartition(ClockSweep *sweep,
+											  BufferAccessStrategy strategy,
+											  uint32 *buf_state);
 
 /*
  * clocksweep allocation balancing
@@ -203,10 +206,9 @@ static int clocksweep_count = 0;
  * id of the buffer now under the hand.
  */
 static inline uint32
-ClockSweepTick(void)
+ClockSweepTick(ClockSweep *sweep)
 {
 	uint32		victim;
-	ClockSweep *sweep = ChooseClockSweep(true);
 
 	/*
 	 * Atomically move hand ahead one buffer - if there's several processes
@@ -425,8 +427,8 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 {
 	BufferDesc *buf;
 	int			bgwprocno;
-	int			trycounter;
-	uint32		local_buf_state;	/* to avoid repeated (de-)referencing */
+	ClockSweep *sweep,
+			   *sweep_start;		/* starting clock-sweep partition */
 
 	*from_ring = false;
 
@@ -480,34 +482,68 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 	/*
 	 * Use the "clock sweep" algorithm to find a free buffer
 	 *
-	 * XXX Note that ClockSweepTick() is NUMA-aware, i.e. it only looks at
-	 * buffers from a single partition, aligned with the NUMA node. That means
-	 * it only accesses buffers from the same NUMA node.
-	 *
-	 * XXX That also means each process "sweeps" only a fraction of buffers,
-	 * even if the other buffers are better candidates for eviction. Maybe
-	 * there should be some logic to "steal" buffers from other freelists or
-	 * other nodes?
-	 *
-	 * XXX Would that also mean we'd have multiple bgwriters, one for each
-	 * node, or would one bgwriter handle all of that?
+	 * Start with the "preferred" partition, and then proceed in a round-robin
+	 * manner. If we cycle back to the starting partition, it means none of the
+	 * partitions has unpinned buffers.
 	 *
-	 * XXX This only searches a single partition, which can result in "no
-	 * unpinned buffers available" even if there are buffers in other
-	 * partitions. Should be fixed by falling back to other partitions if
-	 * needed.
+	 * XXX Does this need to do similar balancing "balancing" as for bgwriter
+	 * in StrategySyncBalance? Maybe it's be enough to simply pick the initial
+	 * partition that way? We'd only getting a single buffer, so not much chance
+	 * to balance over many allocations.
 	 *
-	 * XXX Also, the trycounter should not be set to NBuffers, but to buffer
-	 * count for that one partition. In fact, this should not call ClockSweepTick
-	 * for every iteration. The call is likely quite expensive (does a lot
-	 * of stuff), and also may return a different partition on each call.
-	 * We should just do it once, and then do the for(;;) loop. And then
-	 * maybe advance to the next partition, until we scan through all of them.
+	 * XXX But actually, we're calling ChooseClockSweep() with balance=true, so
+	 * maybe it already does balancing?
 	 */
-	trycounter = NBuffers;
+	sweep = ChooseClockSweep(true);
+	sweep_start = sweep;
+
 	for (;;)
 	{
-		buf = GetBufferDescriptor(ClockSweepTick());
+		buf = StrategyGetBufferPartition(sweep, strategy, buf_state);
+
+		/* found a buffer in the "sweep" partition, we're done */
+		if (buf != NULL)
+			return buf;
+
+		/*
+		 * Try advancing to the next partition, round-robin (if last partition,
+		 * wrap around to the beginning).
+		 *
+		 * XXX This is a bit ugly, there must be a better way to advance to the
+		 * next partition.
+		 */
+		if (sweep == &StrategyControl->sweeps[StrategyControl->num_partitions - 1])
+			sweep = StrategyControl->sweeps;
+		else
+			sweep++;
+
+		/* we've scanned all partitions */
+		if (sweep == sweep_start)
+			break;
+	}
+
+	/* we shouldn't get here if there are unpinned buffers */
+	elog(ERROR, "no unpinned buffers available");
+}
+
+/*
+ * StrategyGetBufferPartition
+ *		get a free buffer from a single clock-sweep partition
+ *
+ * Returns NULL if there are no free (unpinned) buffers in the partition.
+*/
+static BufferDesc *
+StrategyGetBufferPartition(ClockSweep *sweep, BufferAccessStrategy strategy,
+						   uint32 *buf_state)
+{
+	BufferDesc *buf;
+	int			trycounter;
+	uint32		local_buf_state;	/* to avoid repeated (de-)referencing */
+
+	trycounter = sweep->numBuffers;
+	for (;;)
+	{
+		buf = GetBufferDescriptor(ClockSweepTick(sweep));
 
 		/*
 		 * If the buffer is pinned or has a nonzero usage_count, we cannot use
@@ -521,7 +557,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 			{
 				local_buf_state -= BUF_USAGECOUNT_ONE;
 
-				trycounter = NBuffers;
+				trycounter = sweep->numBuffers;
 			}
 			else
 			{
@@ -542,7 +578,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 			 * infinite loop.
 			 */
 			UnlockBufHdr(buf, local_buf_state);
-			elog(ERROR, "no unpinned buffers available");
+			return NULL;
 		}
 		UnlockBufHdr(buf, local_buf_state);
 	}
diff --git a/src/test/recovery/t/027_stream_regress.pl b/src/test/recovery/t/027_stream_regress.pl
index 98b146ed4b7..589c79d97d3 100644
--- a/src/test/recovery/t/027_stream_regress.pl
+++ b/src/test/recovery/t/027_stream_regress.pl
@@ -18,11 +18,6 @@ $node_primary->adjust_conf('postgresql.conf', 'max_connections', '25');
 $node_primary->append_conf('postgresql.conf',
 	'max_prepared_transactions = 10');
 
-# The default is 1MB, which is not enough with clock-sweep partitioning.
-# Increase to 32MB, so that we don't get "no unpinned buffers".
-$node_primary->append_conf('postgresql.conf',
-	'shared_buffers = 32MB');
-
 # Enable pg_stat_statements to force tests to do query jumbling.
 # pg_stat_statements.max should be large enough to hold all the entries
 # of the regression database.
-- 
2.51.0

---

From: Alexey Makhmutov <[email-masked]>
Date: Mon, 13 Oct 2025 02:58:38 +0300

Hi Tomas,

Thank you very much for working on this problem and the entire line of 
patches prepared! I've tried to play with these patches a little and 
here are some my observations and suggestions.

In the current implementation we try to use all available NUMA nodes on 
the machine, however it's often useful to limit the database only to a 
set of specific nodes, so that other nodes can be used for other 
processes. In my testing I was trying to use one node out of four for 
the client program, so I'd liked to limit the database to the remaining 
nodes. I use a systemd service with AllowedMemoryNodes/AllowedCPUs to 
start the cluster, so the obvious choice for me was to use the 
'numa_get_membind' function instead of 'numa_num_configured_nodes' to 
get the list of usable nodes. However, it is much easier to work with 
logical nodes in the [0; n] range inside the PG code, so I've decided to 
add mapping between 'logical nodes' (0-n in PG) to a set of physical 
nodes actually returned by 'numa_get_membind'. We may need to map number 
in both directions, so two translation tables are allocated and filled 
at the first usage of 'pg_numa' functions. It also seems to be a good 
idea to isolate all 'libnuma' calls inside 'pg_numa.c', so to keep all 
'numa_...' calls in it and this also allows us to hide this mapping in 
static functions. Here is the patch, which I've used to test this idea: 
https://github.com/Lerm/postgres/commit/9ec625c2bf564f5432375ec1d7ad02e4b2559161. 
This idea probably could be extended by adding some view to expose this 
mapping to the user (at least for testing purposes) and allow to 
explicitly override this mapping with a GUC setting. With such GUC 
setting we would be able to control PG memory usage on NUMA nodes 
without the need for systemd resource control or numactl parameters.

Next, I've noticed some problems related to the size alignment for 
'numa_tonode_memory' call in 'pg_numa_move_to_node' function. The 
documentation for the 'numa_tonode_memory' says that 'The  size 
argument  will  be rounded up to a multiple of the system page size'. 
However this does not work well with huge pages as alignment is 
performed for the default kernel page size (i.e. 4K in most cases). If 
addr + size value (rounded to the default page size) does not cover the 
entire huge page, then such invocation seems to be processed incorrectly 
and allocation policy is not applied for next pages access in such 
segment. At least this was the behavior I've observed on Debian 12 / 
6.1.40 kernel (i.e. '/proc/<pid>/numa_maps' shows that the segment 
contains pages from wrong nodes).

There are two location at which we could face such situation in current 
patches. First is related to buffers partitions mapping. With current 
code we basically ensure that combined size of all partitions for a 
single node is aligned to (huge) page size (as size is bound to the 
number of descriptors on one page), but individual partition is not 
explicitly aligned to this size. So, we could get the situation in which 
single page is split between adjacent partitions (e.g. 32GB buffers 
split by 3 nodes). With current code we will try to map each partition 
independently, which will results in unaligned calls to 
'numa_tonode_memory', so resulting mapping will be incorrect. We could 
either try to choose size for individual partition to align it to the 
desired page size or map all the partitions for a single node using a 
single 'pg_numa_move_to_node' invocation. During testing I've used the 
second approach, so here is the change to implement such logic: 
https://github.com/Lerm/postgres/commit/ee8b3603afd6d89e67b755dadc8e4c25ffba88be.

The second location which could expose the same problem is related to 
the mapping of PGPROC arrays in 'pgproc_partition_init': here we need to 
align pointer to the end PGPROC partition. There seems to be also two 
additional problems with PGPROC partitioning: we need to account 
additional padding pages in 'PGProcShmemSize' (using the same logic as 
with fplocks) and we should not call 'MemSet(ptr, 0, ...)' prior to 
partitions mapping call (otherwise it will be mapped to current node). 
Here is a potential change, which tries to address these problems: 
https://github.com/Lerm/postgres/commit/eaf12776f59ff150735d0f187595fc8ce3f0a872.

There are also some potential problems with buffers distribution between 
nodes. I have a feeling that current logic in 
'buffer_partitions_prepare' does not work correctly if number of buffers 
is enough to cover just a single partition per node, but total number of 
nodes is below MIN_BUFFER_PARTITIONS (i.e. 2 or 3). In this case we will 
set 'numa_can_partition' to 'true', but will allocate 2 partitions per 
node (so, 4 or 6 partitions in total), while we can fill just 2 or 3 
partition and leaving remaining partitions empty. This should violate 
the last assert check, as last partition will get zero buffers in this 
case. Another issue is related to the usage of 1GB pages, as minimal 
size for buffers partitions is limited by the minimal number of buffer 
descriptors in a single page. For 2MB pages this gives 2097152 / 64 * 8K 
= 256M as minimal size for partition, but for 1GB page the minimal size 
is equal to 1GB / 64 * 8K  = 128GB. So, if we assume 4 as minimal number 
of partitions, then for 2MB pages we need just 1GB for shared_buffers to 
enable partitioning (which seems a perfectly fine minimal limit for most 
cases), but for 1GB pages we need to allocate at least 512GB to allow 
buffers partitioning. Certainly, 1GB pages are usually used on large 
machines with large number of buffers allocated, but still it may be 
useful to allow configurations with 32GB or 64GB buffer cache to use 
both 1GB pages and buffers partitioning at the same time. However, I 
don't see an easy way to achieve this with the current logic. We either 
need to allow usage of different page sizes here (i.e. 2MB for 
descriptors and 1GB for buffers) or combine both buffers and its 
descriptors in a single object (i.e. 'buffer chunk', which cover enough 
buffers and their descriptors to fit into one or several memory pages), 
effectively replacing both buffers and descriptors arrays with an array 
of such 'chunks'. The latter solution may also help with dynamic buffer 
cache resizing (as we may just add additional 'chunks' in this case) and 
also increase TLB-hits with 1GB page (as both descriptor and its buffer 
will be likely located in the same page). However, both these changes 
seems to be quite large.

I've tried also to run some benchmarks on my server: I've got some 
improvements in 'pgbench/tpcb-like'results - about 8%, but only with 
backends pinning to NUMA node (i.e. adjusting your previous pinning 
patch to 'debug_numa' GUC: 
https://github.com/Lerm/postgres/commit/5942a3e12c7c501aa9febb63972a039e7ce00c20). 
For 'select-only' scenario the gain is more substantial (about 15%), but 
these tests are tricky, as they are more sensitive to other server 
settings and specific functions layout in compiled code, so they need 
more checks.

Thank you again for sharing these patches!

Thanks,
Alexey

---

From: Tomas Vondra <[email-masked]>
Date: Mon, 13 Oct 2025 13:09:20 +0200

On 10/13/25 01:58, Alexey Makhmutov wrote:
> https://github.com/Lerm/postgres/
> commit/9ec625c2bf564f5432375ec1d7ad02e4b2559161. This idea probably
> could be extended by adding some view to expose this mapping to the user
> (at least for testing purposes) and allow to explicitly override this
> mapping with a GUC setting. With such GUC setting we would be able to
> control PG memory usage on NUMA nodes without the need for systemd
> resource control or numactl parameters.
> 

I've argued to keep this out of scope for v1, to keep it smaller and
simpler. I'm not against adding that feature, though. If someone writes
a patch to support this. I suppose the commit you linked is a step in
that direction.

I agree we should isolate libnuma calls to pg_numa.{c,h}. I wasn't quite
consistent when doing that.

> contains pages from wrong nodes).

I'm not sure I understand. Are you suggesting there's a bug in the
patch, the kernel, or somewhere else?

There's definitely a possibility of confusion with huge pages, no doubt
about that. The default "system page size" is 4KB, but we need to
process whole huge pages.

But this is exactly why (with hugepages) the code aligns everything to
huge page boundary, and sizes everything as a multiple of huge page. At
least I think so. Maybe I remember wrong?

> second approach, so here is the change to implement such logic: https://
> github.com/Lerm/postgres/commit/ee8b3603afd6d89e67b755dadc8e4c25ffba88be.
> 

Can you actually demonstrate this? The code does these two things:

* calculate min_node_buffers so that buffers/descriptors are a multiple
  of page size (either 4K or huge page)

* align buffers and descriptors to memory page

  TYPEALIGN(buffer_align, ...)

I believe this is sufficient to ensure nothing gets split / mapped
incorrectly. Maybe this fails sometimes?

> https://github.com/Lerm/postgres/commit/
> eaf12776f59ff150735d0f187595fc8ce3f0a872.
> 

So you're saying pgproc_partition_init() should not do just this

    ptr = (char *) ptr + num_procs * sizeof(PGPROC);

but align the pointer to numa_page_size too? Sounds reasonable.

Yeah, PGProcShmemSize() should have added huge pages for each partition,
just like FastPathLockShmemSize(). Seems like a bug.

I don't think the memset() is a problem. Yes, it might map it to the
current node, but so what - the numa_tonode_memory() will just move it
to the correct one.

> will be likely located in the same page). However, both these changes

I'll look at handling the case with shared_buffers being too small to
allow partitioning. There well might be a bug. We should simply disable
partitioning in such cases.

As for 1GB huge pages, I don't see a good way to support configurations
with small buffers in these cases. To me it seems acceptable to say that
if you want 1GB huge pages, you should have a lot of memory and shared
buffers large enough.

I'm not against supporting such systems, if we can come up with a good
partitioning scheme. When I tried to come up with a scheme like that, it
always came with a substantial complexity & cost. The main challenge was
that it forced splitting the array of buffer descriptors, similarly to
what the PGPROC partitioning does. And that made buffer access so much
more complex / expensive it seemed not worth it. I was worried about
impact on systems without NUMA partitioning.

> patch to 'debug_numa' GUC: https://github.com/Lerm/postgres/
> commit/5942a3e12c7c501aa9febb63972a039e7ce00c20). For 'select-only'
> scenario the gain is more substantial (about 15%), but these tests are
> tricky, as they are more sensitive to other server settings and specific
> functions layout in compiled code, so they need more checks.
> 

What kind of hardware was that? What/how many cpus, NUMA nodes, how much
memory, what storage?

FWIW the main purpose of these patches was not so much throughput
improvement, but making the behavior more stable / consistent.

regards

-- 
Tomas Vondra

---

From: Alexey Makhmutov <[email-masked]>
Date: Mon, 13 Oct 2025 21:34:39 +0300

We need to ensure that both addr and (addr + size) are aligned to the 
page size of the target mapping during 'numa_tonode_memory' invocation, 
otherwise it may produce unexpected results.

 > But this is exactly why (with hugepages) the code aligns everything 
to huge page boundary, and sizes everything as a multiple of huge page. 
At least I think so. Maybe I remember wrong?

I assume that there are places in the current patch, which could perform 
such unaligned mapping. See below for samples.

 > Can you actually demonstrate this?

This issue is related to the calculation of partition size for buffer 
descriptors in case we have multiple partitions per node. Currently we 
ensure that each node gets number of buffers, which fits into whole 
memory pages, but if we have several partitions per node, then there is 
no guarantee that partition size will be properly aligned for 
descriptors. We could observe this problem only if we have multiple 
partitions per node and with MIN_BUFFER_PARTITIONS equal to 4, this 
issue can potentially affect only configurations with 2 or 3 nodes.

Two examples here: first, let's assume we want to have shared_buffers 
set to 32GB with 3 NUMA nodes and 2MB pages. The NBuffers will be 
4,194,304,  min_node_buffers will be 32,768 and num_partitions_per_node 
will be 2 (so, 6 partitions in total). NBuffers/min_node_buffers = 128, 
so the nearest multiplier for min_node_buffers which allow us to cover 
all buffers with 3 nodes is 43 (42*3 = 126, 43*3 = 129). The 
num_buffers_per_node is 43*min_node_buffers and it is aligned to page 
size, but we need to split it between two partitions, so each gets 
41.5*min_node_buffers buffers. This still allow us to split buffers 
itself by page boundary, but descriptor partitions will be split just in 
the middle of the page. Here is the log for such configuration:
NUMA: buffers 4194304 partitions 6 num_nodes 3 per_node 2 
buffers_per_node 1409024 (min 32768)
NUMA: buffer 0 node 0 partition 0 buffers 704512 first 0 last 704511
NUMA: buffer 1 node 0 partition 1 buffers 704512 first 704512 last 1409023
NUMA: buffer 2 node 1 partition 0 buffers 704512 first 1409024 last 2113535
NUMA: buffer 3 node 1 partition 1 buffers 704512 first 2113536 last 2818047
NUMA: buffer 4 node 2 partition 0 buffers 688128 first 2818048 last 3506175
NUMA: buffer 5 node 2 partition 1 buffers 688128 first 3506176 last 4194303
NUMA: buffer_partitions_init: 0 => 0 buffers 704512 start 0x7ff7c8c00000 
end 0x7ff920c00000 (size 5771362304)
NUMA: buffer_partitions_init: 0 => 0 descriptors 704512 start 
0x7ff7b8a00000 end 0x7ff7bb500000 (size 45088768)
mbind: Invalid argument
NUMA: buffer_partitions_init: 1 => 0 buffers 704512 start 0x7ff920c00000 
end 0x7ffa78c00000 (size 5771362304)
NUMA: buffer_partitions_init: 1 => 0 descriptors 704512 start 
0x7ff7bb500000 end 0x7ff7be000000 (size 45088768)
mbind: Invalid argument
NUMA: buffer_partitions_init: 2 => 1 buffers 704512 start 0x7ffa78c00000 
end 0x7ffbd0c00000 (size 5771362304)
NUMA: buffer_partitions_init: 2 => 1 descriptors 704512 start 
0x7ff7be000000 end 0x7ff7c0b00000 (size 45088768)
mbind: Invalid argument
NUMA: buffer_partitions_init: 3 => 1 buffers 704512 start 0x7ffbd0c00000 
end 0x7ffd28c00000 (size 5771362304)
NUMA: buffer_partitions_init: 3 => 1 descriptors 704512 start 
0x7ff7c0b00000 end 0x7ff7c3600000 (size 45088768)
mbind: Invalid argument
NUMA: buffer_partitions_init: 4 => 2 buffers 688128 start 0x7ffd28c00000 
end 0x7ffe78c00000 (size 5637144576)
NUMA: buffer_partitions_init: 4 => 2 descriptors 688128 start 
0x7ff7c3600000 end 0x7ff7c6000000 (size 44040192)
NUMA: buffer_partitions_init: 5 => 2 buffers 688128 start 0x7ffe78c00000 
end 0x7fffc8c00000 (size 5637144576)
NUMA: buffer_partitions_init: 5 => 2 descriptors 688128 start 
0x7ff7c6000000 end 0x7ff7c8a00000 (size 44040192)

Another example: 2 nodes and 15872MB shared_buffers. Again, 
NBuffers/min_node_buffers=62, so num_buffers_per_node is 
31*min_node_buffers, which gives each partition 15.5*min_node_buffers. 
Here is the log output:
NUMA: buffers 2031616 partitions 4 num_nodes 2 per_node 2 
buffers_per_node 1015808 (min 32768)
NUMA: buffer 0 node 0 partition 0 buffers 507904 first 0 last 507903
NUMA: buffer 1 node 0 partition 1 buffers 507904 first 507904 last 1015807
NUMA: buffer 2 node 1 partition 0 buffers 507904 first 1015808 last 1523711
NUMA: buffer 3 node 1 partition 1 buffers 507904 first 1523712 last 2031615
NUMA: buffer_partitions_init: 0 => 0 buffers 507904 start 0x7ffbf9c00000 
end 0x7ffcf1c00000 (size 4160749568)
NUMA: buffer_partitions_init: 0 => 0 descriptors 507904 start 
0x7ffbf1e00000 end 0x7ffbf3d00000 (size 32505856)
mbind: Invalid argument
NUMA: buffer_partitions_init: 1 => 0 buffers 507904 start 0x7ffcf1c00000 
end 0x7ffde9c00000 (size 4160749568)
NUMA: buffer_partitions_init: 1 => 0 descriptors 507904 start 
0x7ffbf3d00000 end 0x7ffbf5c00000 (size 32505856)
mbind: Invalid argument
NUMA: buffer_partitions_init: 2 => 1 buffers 507904 start 0x7ffde9c00000 
end 0x7ffee1c00000 (size 4160749568)
NUMA: buffer_partitions_init: 2 => 1 descriptors 507904 start 
0x7ffbf5c00000 end 0x7ffbf7b00000 (size 32505856)
mbind: Invalid argument
NUMA: buffer_partitions_init: 3 => 1 buffers 507904 start 0x7ffee1c00000 
end 0x7fffd9c00000 (size 4160749568)
NUMA: buffer_partitions_init: 3 => 1 descriptors 507904 start 
0x7ffbf7b00000 end 0x7ffbf9a00000 (size 32505856)
mbind: Invalid argument

 > but align the pointer to numa_page_size too? Sounds reasonable.

Yes, that's exactly my point, otherwise we could violate the alignment 
rule for 'numa_tonode_memory'. Here is an extraction from the log for 
system with 2 nodes, 2000 max_connections and 2MB pages:
NUMA: pgproc backends 2056 num_nodes 2 per_node 1028
NUMA: pgproc_init_partition procs 0x7fffe7800000 endptr 0x7fffe78d2d20 
num_procs 1028 node 0
mbind: Invalid argument
NUMA: pgproc_init_partition procs 0x7fffe7a00000 endptr 0x7fffe7ad2d20 
num_procs 1028 node 1
mbind: Invalid argument
NUMA: pgproc_init_partition procs 0x7fffe7c00000 endptr 0x7fffe7c07cb0 
num_procs 38 node -1
mbind: Invalid argument
mbind: Invalid argument

Well, the 'numa_tonode_memory' call does not move pages to the target 
node. It just sets the policy for mapping, so system will actually try 
to provide page from the correct node once we touch it. However, if the 
page is already faulted, then it won't be affected by this mapping, so 
that's why it works faster compared to 'numa_move_pages'. As stated in 
libnuma documentation:
* numa_tonode_memory() put memory on a specific node. The constraints 
described for numa_interleave_memory() apply here too.
* numa_interleave_memory()  interleaves  size  bytes of memory page by 
page from start on nodes specified in nodemask. <...> This is a lower 
level function to interleave allocated but not yet faulted in memory. 
Not yet faulted in means the memory is allocated using mmap(2) or 
shmat(2), but has not been accessed by  the current process yet. <...> 
If the numa_set_strict() flag is true then the operation will cause a 
numa_error if there were already pages in the mapping that do not follow 
the policy.

I assume, that for the regular page kernel may rebalance memory in the 
future (not immediately), but not for hugepages. So, we really don't 
want to touch the memory area before we call the 'numa_tonode_memory'.

This can be easily tested with the simple program:
#include <stdio.h>
#include <numa.h>
#include <sys/mman.h>
#include <linux/mman.h>

#define MAP_SIZE 2*1024*1024

int main(int argc, char** argv) {
   void* ptr1 = mmap(NULL, MAP_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED 
| MAP_ANONYMOUS | MAP_HUGETLB | MAP_HUGE_2MB, -1, 0);
   void* ptr2 = mmap(NULL, MAP_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED 
| MAP_ANONYMOUS | MAP_HUGETLB | MAP_HUGE_2MB, -1, 0);

   /* Fault first page */
   memset(ptr1, 1, MAP_SIZE);
   /* Move to node 1 */
   numa_tonode_memory(ptr1, MAP_SIZE, 1);
   numa_tonode_memory(ptr2, MAP_SIZE, 1);
   /* Fault second page */
   memset(ptr2, 1, MAP_SIZE);

   /* Wait */
   printf("ptr1=%lx\nptr2=\%lx\nPress Enter to continue...\n",ptr1,ptr2);
   getchar();
   munmap(ptr2, MAP_SIZE);
   munmap(ptr1, MAP_SIZE);
   return 0;
}

Running it on the first node:
# gcc -o test_mem test_mem.c -lnuma
# taskset -c 0 ./test_mem
ptr1=7ffff7a00000
ptr2=7ffff7800000
Press Enter to continue...

 From another terminal:
# grep huge /proc/`pgrep test_mem`/numa_maps
7ffff7800000 bind:1 file=/anon_hugepage\040(deleted) huge dirty=1 N1=1 
kernelpagesize_kB=2048
7ffff7a00000 bind:1 file=/anon_hugepage\040(deleted) huge dirty=1 N0=1 
kernelpagesize_kB=2048

So, while policy (bind:1) is set for both mappings, but only the second 
one (which was not touched before the 'numa_tonode_memory' invocation) 
is actualy located on node 1 rather than 0.

 > What kind of hardware was that? What/how many cpus, NUMA nodes, how 
much memory, what storage?

Of course, that's valid question. I probably should not have commented 
on performance side without providing full data, while I was still 
trying to measure it and it was just preliminary runs. Sorry for that.

Thanks,
Alexey

---

From: Tomas Vondra <[email-masked]>
Date: Wed, 15 Oct 2025 19:02:38 +0200

Hi,

Here's an updated patch series, addressing (some) of the issues. I've
kept the changes in separate patches, to make the changes easier to
review and discuss.

> otherwise it may produce unexpected results.

Hmm. The libnuma docs about numa_intereave_memory says:

    ... The size argument will be rounded up to a multiple of the system
    page size. ...

Which I interpreted that it does all the necessary rounding. But if this
ignores huge pages (i.e. "system page size" is 4K, not a HP size), then
aligning the size explicitly is would be needed.

This would be pretty annoying, though. It'd mean we can't rely on any
rounding done by libnuma, at least for code that might use huge pages.

Nevertheless, the updated patches should address both cases ...

> buffers_per_node 1409024 (min 32768)

I see. I was really puzzled how could a node get chunk of buffers that's
not a multiple of page size, because min_node_buffers was meant to
guarantee that. But now I realize it's not about per-node buffers, it's
about individual partitions.

Initially I thought the right way to fix this is to use min_node_buffers
for each partitions, not for nodes. But that would increase the amount
of memory needed for NUMA partitioning to work. I practice that wouldn't
be an issue, because it'd still be only ~1GB (with 2MB huge pages), and
the relevant systems will have way more.

But then I realized it's we don't need to map the partitions one by one.
We can simply map all partitions for the whole NUMA node at once, and
then we don't have this problem at all.

The attached 0007 patch does this to fix the issue. And I just noticed
this is pretty much exactly how you fixed this in your commit ee8b360.

The last partition may still not have the size aligned, though, because
may not be a multiple of min_node_buffers.

> 
> Another example: 2 nodes and 15872MB shared_buffers. Again, NBuffers/
> min_node_buffers=62, so num_buffers_per_node is 31*min_node_buffers,
> which gives each partition 15.5*min_node_buffers. Here is the log output:

Should be fixed by 0010 by explicitly aligning the size like this. It's
a bit more extensive than your eaf1277.

BTW what's the mbind failures about? Is that something we check, at
least in memory

> numa_error if there were already pages in the mapping that do not follow

Point taken. The 0009 fixes this by moving the MemSet() to after the
partitioning. At that point the policy is already set.

There's a couple more fixes. 0008 improves handling of cases that don't
allow NUMA partitioning (like when shared_buffers are too small). 0011
adds the missing padding to PGProcShmemSize, which you also fixed in one
of your commits.

0012 reduces logging in clock-sweep balancing, which on idle systems was
annoyingly verbose.

I keps 0006 separate for now. It got broken by 5e89985928, and the
conflicts were fairly extensive. Better keep it separate a bit longer.

regards

-- 
Tomas Vondra
From 81749da92eb8e66fce0f3a94059b9a0dafa8bca6 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 17 Sep 2025 23:04:29 +0200
Subject: [PATCH v20251015 01/12] NUMA: shared buffers partitioning

The effect is similar to

     numactl --interleave=all

Notes:

 

 PG_MODULE_MAGIC_EXT(

 	PG_RETURN_DATUM(result);
 }
+
+/*
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+

+/*
 
 /*
  * Data Structures:

 
 
+		/*

 
 
index b176d5130e4..82acead2e47 100644
@@ -906,6 +906,16 @@
   boot_val => 'true',
 },

 #ifdef HAVE_SYSLOG
 
 #ifdef USE_LIBNUMA
 
index c1206a46aba..80b9d8d012d 100644

 /* in buf_init.c */
 
@@ -484,4 +485,9 @@ extern void DropRelationLocalBuffers(RelFileLocator rlocator,
 extern void DropRelationAllLocalBuffers(RelFileLocator rlocator);
 extern void AtEOXact_LocalBuffers(bool isCommit);
 
index 3f37b294af6..43ec9f4cf0f 100644

+/*
 
@@ -320,6 +342,7 @@ extern void EvictRelUnpinnedBuffers(Relation rel,
 /* in buf_init.c */
 extern void BufferManagerShmemInit(void);
 extern Size BufferManagerShmemSize(void);
+extern int	BufferGetNode(Buffer buffer);
 
 /* in localbuf.c */

 #include "miscadmin.h"
 
 /*
 
+/*
+ * pg_numa_move_to_node
 
 /* Empty wrappers */
 
+void
index 5290b91e83e..b02d117c374 100644
@@ -347,6 +347,8 @@ BufferDescPadded
 BufferHeapTupleTableSlot
 BufferLookupEnt
 BufferManagerRelation
+BufferPartition
+BufferPartitions
 BufferStrategyControl
 BufferTag
 BufferUsage
-- 
2.51.0

From dc30d0de0c048e3ca572614c2278ce57158a4a1a Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Sun, 8 Jun 2025 18:53:12 +0200
Subject: [PATCH v20251015 02/12] NUMA: clockweep partitioning

 

 PG_MODULE_MAGIC_EXT(

 

 

index edf17ce3ea1..aa209eddaab 100644
@@ -3614,33 +3614,29 @@ BufferSync(int flags)
 }
 
 /*
 
 	/* Moving averages of allocation rate and clean-buffer density */
 	static float smoothed_alloc = 0;
@@ -3668,25 +3664,16 @@ BgBufferSync(WritebackContext *wb_context)
 	long		new_strategy_delta;
 	uint32		new_recent_alloc;

 	/*
 	 * Compute strategy_delta = how many buffers have been scanned by the
@@ -3698,17 +3685,17 @@ BgBufferSync(WritebackContext *wb_context)
 	 */
 	if (saved_info_valid)
 	{
-		int32		passes_delta = strategy_passes - prev_strategy_passes;
+		int32		passes_delta = strategy_passes - prev_strategy_passes[partition];

 
@@ -3716,11 +3703,11 @@ BgBufferSync(WritebackContext *wb_context)
@@ -3740,9 +3727,9 @@ BgBufferSync(WritebackContext *wb_context)
@@ -3756,15 +3743,16 @@ BgBufferSync(WritebackContext *wb_context)

 	/*
@@ -3772,9 +3760,9 @@ BgBufferSync(WritebackContext *wb_context)
@@ -3784,7 +3772,7 @@ BgBufferSync(WritebackContext *wb_context)
 
 	/*
@@ -3792,10 +3780,10 @@ BgBufferSync(WritebackContext *wb_context)
 
@@ -3822,7 +3810,7 @@ BgBufferSync(WritebackContext *wb_context)
 
@@ -3847,20 +3835,20 @@ BgBufferSync(WritebackContext *wb_context)

@@ -3874,7 +3862,7 @@ BgBufferSync(WritebackContext *wb_context)
 
 #ifdef BGW_DEBUG
@@ -3904,8 +3892,74 @@ BgBufferSync(WritebackContext *wb_context)
 #endif
 	}

 /*
diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index 7fe34d3ef4c..952440fd9e5 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -15,27 +15,47 @@
  */
 #include "postgres.h"
 
+#ifdef USE_LIBNUMA
+#include <sched.h>
+#endif
+
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
 #include "pgstat.h"

 /*
 
 	/*

 
 
 /*
 
 	/*

 
 		/*

 
 /*
@@ -224,9 +367,35 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 
@@ -306,6 +475,46 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 	}
 }
 
+/*
@@ -313,37 +522,44 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r

 	if (complete_passes)
 	{
 
 		/*
 
-	if (num_buf_alloc)
-	{
 
 /*
@@ -380,6 +596,9 @@ Size
 StrategyShmemSize(void)
 {
 	Size		size = 0;
+	int			num_partitions;
+
 
@@ -387,6 +606,10 @@ StrategyShmemSize(void)

@@ -402,6 +625,18 @@ StrategyInitialize(bool init)
 {
 	bool		found;
 
+	int			num_nodes;
+	int			num_partitions;
@@ -419,7 +654,8 @@ StrategyInitialize(bool init)
 
 	if (!found)
@@ -431,15 +667,44 @@ StrategyInitialize(bool init)

 
-		/* Clear statistics */
 
@@ -802,3 +1067,23 @@ StrategyRejectBuffer(BufferAccessStrategy strategy, BufferDesc *buf, bool from_r
 
 	return true;
 }
+
+void
index 80b9d8d012d..67f2afc623d 100644
@@ -443,7 +443,9 @@ extern BufferDesc *StrategyGetBuffer(BufferAccessStrategy strategy,

@@ -489,5 +491,6 @@ extern int	BufferPartitionCount(void);
 
 #endif							/* BUFMGR_INTERNALS_H */
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index 43ec9f4cf0f..67f07d10e48 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -356,6 +356,11 @@ extern int	GetAccessStrategyBufferCount(BufferAccessStrategy strategy);
 extern int	GetAccessStrategyPinLimit(BufferAccessStrategy strategy);

 /* inline functions */
 
index b02d117c374..33a9721cef3 100644

From 38072e13f35662a953f9439c8093a7c72937564d Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 10 Sep 2025 18:56:28 +0200
Subject: [PATCH v20251015 03/12] NUMA: clocksweep allocation balancing

 

@@ -27,7 +29,7 @@

 PG_MODULE_MAGIC_EXT(
 
 	if (SRF_IS_FIRSTCALL())
 	{

 

 		uint32		complete_passes,
 					next_victim_buffer,
-					buffer_allocs;
+					buffer_allocs,
+					buffer_req_allocs;
+
+		int		   *weights;
+		Datum	   *dweights;
+		ArrayType  *array;

 
 
index aa209eddaab..13258200526 100644
@@ -3918,6 +3918,9 @@ BgBufferSync(WritebackContext *wb_context)

index 952440fd9e5..73e57e13ba1 100644

+/*
 
+	/*
 
+	/*
 
 /*
 
 /*
 
 	/*

 /*
@@ -367,7 +477,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 
 	/*
 	 * Use the "clock sweep" algorithm to find a free buffer
@@ -475,6 +585,224 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 	}
 }
 
+/*
+ * StrategySyncBalance
@@ -501,6 +829,7 @@ StrategySyncPrepare(int *num_parts, uint32 *num_buf_alloc)
 
@@ -695,7 +1024,21 @@ StrategyInitialize(bool init)
 
 		/* No pending notification */
@@ -1070,8 +1413,10 @@ StrategyRejectBuffer(BufferAccessStrategy strategy, BufferDesc *buf, bool from_r
 
 void
 
@@ -1079,11 +1424,21 @@ ClockSweepPartitionGetInfo(int idx,

 
index 67f2afc623d..4ae0c32dfe5 100644
@@ -443,6 +443,7 @@ extern BufferDesc *StrategyGetBuffer(BufferAccessStrategy strategy,
 
+extern void StrategySyncBalance(void);
 extern void StrategySyncPrepare(int *num_parts, uint32 *num_buf_alloc);
 extern int	StrategySyncStart(int partition, uint32 *complete_passes,
 							  int *first_buffer, int *num_buffers);
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index 67f07d10e48..1971f0d9731 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -357,11 +357,13 @@ extern int	GetAccessStrategyPinLimit(BufferAccessStrategy strategy);

 /* inline functions */
 
-- 
2.51.0

From 366202b656fb63400e0a1ccde29137ddab846198 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 6 Aug 2025 01:09:57 +0200
Subject: [PATCH v20251015 04/12] NUMA: weighted clocksweep balancing

diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index 73e57e13ba1..2b4c853745d 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -642,6 +642,20 @@ StrategySyncBalance(void)
 			avg_allocs,			/* average allocations (per partition) */
 			delta_allocs = 0;	/* sum of allocs above average */
 
+	/*
@@ -668,16 +682,27 @@ StrategySyncBalance(void)

 	/*
 
 	/*
@@ -685,8 +710,14 @@ StrategySyncBalance(void)
 
 	/*
@@ -749,6 +780,10 @@ StrategySyncBalance(void)
 		ClockSweep *sweep = &StrategyControl->sweeps[i];
 		uint8		balance[MAX_BUFFER_PARTITIONS];

@@ -756,7 +791,7 @@ StrategySyncBalance(void)
 		memset(balance, 0, sizeof(uint8) * MAX_BUFFER_PARTITIONS);
 
@@ -770,22 +805,30 @@ StrategySyncBalance(void)

 

From 8ac90c354e3305e7ce714077bf8c5ac179a5013c Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Mon, 8 Sep 2025 13:11:02 +0200
Subject: [PATCH v20251015 05/12] NUMA: partition PGPROC

 - PGPROC array / node #0
 - PGPROC array / node #1

 

 PG_MODULE_MAGIC_EXT(

 		/*
 
index 33369fbe23a..afa74466006 100644

index 976638a58ac..5e7b0ac8850 100644

 
 /*
 
index 2b4c853745d..314ccbe4f93 100644
@@ -469,7 +469,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 
 	/*

 
 /*

+	AssertCheckAllProcs();
+

 

+	AssertCheckAllProcs();
+
 	/*
 
+	AssertCheckAllProcs();
+
 	myoff = proc->pgxactoff;

 

+	AssertCheckAllProcs();
+
 	/*

 
 
-
 /*

 			/*

 

 			/*

 

 		if (proc->pid == pid)
 		{
 
 			result = proc->pid;
 			break;
 
 		if (proc == MyProc)

 
 		/*
 
 		if (proc->pid == 0)
 
 		if (proc->pid == 0)

 		if (proc->pid == 0)

 

 
 
+#ifdef USE_LIBNUMA
+#include <sched.h>
+#endif
+
 #include <signal.h>
 #include <unistd.h>
 #include <sys/time.h>
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+

 

 
+	/*

 
+	/*

 
 
 	return size;
 }

 

 
+	/*

 
 		/*

 

 

 
 
 		if (proc->pid == pid)
 		{

 /*
 
 	return ok;
 }
+
+/*

 #ifdef USE_LIBNUMA

 

 

 /*
 
 /*
 
 #ifdef EXEC_BACKEND

+extern int	ProcPartitionCount(void);
+extern void ProcPartitionGet(int idx, int *node, int *nprocs, void **procsptr, void **fpptr);
+
 #endif							/* _PROC_H_ */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 33a9721cef3..8b0d091c6c0 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1878,6 +1878,7 @@ PGP_MPI
 PGP_PubKey
 PGP_S2K
 PGPing
+PGProcPartition
 PGQueryClass
 PGRUsage
 PGSemaphore
-- 
2.51.0

From 2fd75d9fcd58cab66cf75b1b39eb3b477ba9b311 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 15 Oct 2025 13:59:29 +0200
Subject: [PATCH v20251015 06/12] fix: StrategyGetBuffer

StrategyGetBuffer is expected to scan all clock-sweep partitions, not
just the "local" one. So start at the optimal one (as calculated by
ChooseClockSweep), and then advance to the next one in a round-robin
way, until we find a free / unpinned buffer.
---
 src/backend/storage/buffer/freelist.c     | 91 ++++++++++++++++-------
 src/test/recovery/t/027_stream_regress.pl |  5 --
 2 files changed, 63 insertions(+), 33 deletions(-)

index 314ccbe4f93..7f241301dcb 100644
 
 /*
 
 	/*
 	 * Atomically move hand ahead one buffer - if there's several processes
@@ -428,7 +430,8 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 {
 	BufferDesc *buf;
 	int			bgwprocno;
-	int			trycounter;
+	ClockSweep *sweep,
+			   *sweep_start;		/* starting clock-sweep partition */
 
 	*from_ring = false;
 
@@ -482,37 +485,69 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 	/*
 		uint32		old_buf_state;
 		uint32		local_buf_state;

 		/*
 		 * Check whether the buffer can be used and pin it if so. Do this
@@ -540,7 +575,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 					 * one eventually, but it's probably better to fail than
 					 * to risk getting stuck in an infinite loop.
 					 */
-					elog(ERROR, "no unpinned buffers available");
+					return NULL;
 				}
 				break;
 			}
@@ -558,7 +593,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 				if (pg_atomic_compare_exchange_u32(&buf->state, &old_buf_state,
 												   local_buf_state))
 				{
-					trycounter = NBuffers;
+					trycounter = sweep->numBuffers;
 					break;
 				}
 			}
diff --git a/src/test/recovery/t/027_stream_regress.pl b/src/test/recovery/t/027_stream_regress.pl
index 98b146ed4b7..589c79d97d3 100644
--- a/src/test/recovery/t/027_stream_regress.pl
+++ b/src/test/recovery/t/027_stream_regress.pl
@@ -18,11 +18,6 @@ $node_primary->adjust_conf('postgresql.conf', 'max_connections', '25');
 $node_primary->append_conf('postgresql.conf',
 	'max_prepared_transactions = 10');

From d6bc867374a1195d2426de629b0ffd1f22b040a9 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 15 Oct 2025 15:03:48 +0200
Subject: [PATCH v20251015 07/12] fix: map all buffer partitions for NUMA node
 at once

Don't map individual partitions, but all partitions for the whole NUMA
node at once. This means we don't need to worry about memory pages that
span two partitions (min_node_buffers applies to the whole node). That
might be causing problems with huge pages, as reported by Alexey
Makhmutov.

Discussion: [email-masked]
---
 src/backend/storage/buffer/buf_init.c | 42 ++++++++++++++++++---------
 1 file changed, 29 insertions(+), 13 deletions(-)

diff --git a/src/backend/storage/buffer/buf_init.c b/src/backend/storage/buffer/buf_init.c
index 2fd7f937ffb..65f45346bd1 100644
--- a/src/backend/storage/buffer/buf_init.c
+++ b/src/backend/storage/buffer/buf_init.c
@@ -460,7 +460,7 @@ buffer_partitions_prepare(void)
 	numa_can_partition = true;	/* assume we can allocate to nodes */
 	if (numa_nodes > max_nodes)
 	{
-		elog(WARNING, "shared buffers too small for %d nodes (max nodes %d)",
+		elog(LOG, "shared buffers too small for %d nodes (max nodes %d)",
 			 numa_nodes, max_nodes);
 		numa_can_partition = false;
 	}
@@ -630,8 +630,8 @@ buffer_partitions_init(void)
 	 * memory pages (smaller than BLCKSZ) we'd split all buffers to multiple
 	 * NUMA nodes. And we don't want that.
 	 *
-	 * But even with huge pages it seems like a good idea to not have mapping
-	 * for each page.
+	 * But even with huge pages it seems like a good idea to not map pages
+	 * one by one.
 	 *
 	 * So we always assign a larger contiguous chunk of buffers to the same
 	 * NUMA node, as calculated by choose_chunk_buffers(). We try to keep the
@@ -649,35 +649,51 @@ buffer_partitions_init(void)
 	 * We need to account for partitions being of different length, when the
 	 * NBuffers is not nicely divisible. To do that we keep track of the start
 	 * of the next partition.
+	 *
+	 * We always map all partitions for the same node at once, so that we
+	 * don't need to worry about alignment of memory pages that get split
+	 * between partitions (we only worry about min_node_buffers for whole
+	 * NUMA nodes, not for individual partitions).
 	 */
 	buffers_ptr = BufferBlocks;
 	descriptors_ptr = (char *) BufferDescriptors;
 
-	for (int i = 0; i < numa_partitions; i++)
+	for (int n = 0; n < numa_nodes; n++)
 	{
-		BufferPartition *part = &BufferPartitionsArray->partitions[i];
 		char	   *startptr,
 				   *endptr;
+		int			num_buffers = 0;
+
+		/* sum buffers in all partitions for this node */
+		for (int p = 0; p < parts_per_node; p++)
+		{
+			int		pidx = (n * parts_per_node + p);
+			BufferPartition *part = &BufferPartitionsArray->partitions[pidx];
+
+			Assert(part->numa_node == n);
+
+			num_buffers += part->num_buffers;
+		}
 
 		/* first map buffers */
 		startptr = buffers_ptr;
-		endptr = startptr + ((Size) part->num_buffers * BLCKSZ);
+		endptr = startptr + ((Size) num_buffers * BLCKSZ);
 		buffers_ptr = endptr;	/* start of the next partition */
 
-		elog(DEBUG1, "NUMA: buffer_partitions_init: %d => %d buffers %d start %p end %p (size %zd)",
-			 i, part->numa_node, part->num_buffers, startptr, endptr, (endptr - startptr));
+		elog(DEBUG1, "NUMA: buffer_partitions_init: %d => buffers %d start %p end %p (size %zd)",
+			 n, num_buffers, startptr, endptr, (endptr - startptr));
 
-		pg_numa_move_to_node(startptr, endptr, part->numa_node);
+		pg_numa_move_to_node(startptr, endptr, n);
 
 		/* now do the same for buffer descriptors */
 		startptr = descriptors_ptr;
-		endptr = startptr + ((Size) part->num_buffers * sizeof(BufferDescPadded));
+		endptr = startptr + ((Size) num_buffers * sizeof(BufferDescPadded));
 		descriptors_ptr = endptr;
 
-		elog(DEBUG1, "NUMA: buffer_partitions_init: %d => %d descriptors %d start %p end %p (size %zd)",
-			 i, part->numa_node, part->num_buffers, startptr, endptr, (endptr - startptr));
+		elog(DEBUG1, "NUMA: buffer_partitions_init: %d => descriptors %d start %p end %p (size %zd)",
+			 n, num_buffers, startptr, endptr, (endptr - startptr));
 
-		pg_numa_move_to_node(startptr, endptr, part->numa_node);
+		pg_numa_move_to_node(startptr, endptr, n);
 	}
 
 	/* we should have consumed the arrays exactly */
-- 
2.51.0

From 963e48dadc3ecc6941ee8e70d0f6d84d13ceb276 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 15 Oct 2025 17:42:23 +0200
Subject: [PATCH v20251015 08/12] fix: handle disabled NUMA-partitioning of
 buffers

We still partition the buffers, because we want to partition the
clock-sweep (and that relies on partitions). But we don't map the
partitions to NUMA nodes.

The NUMA partitioning may be disabled for a number of reasons. The build
may not have libnuma enabled, debug_numa may not include "buffers"
and/or the shared buffers are too small (especially with huge pages).
---
 src/backend/storage/buffer/buf_init.c | 25 +++++++++++++++++++++----
 1 file changed, 21 insertions(+), 4 deletions(-)

diff --git a/src/backend/storage/buffer/buf_init.c b/src/backend/storage/buffer/buf_init.c
index 65f45346bd1..d0efa102d82 100644
--- a/src/backend/storage/buffer/buf_init.c
+++ b/src/backend/storage/buffer/buf_init.c
@@ -301,6 +301,10 @@ BufferGetNode(Buffer buffer)
 	if (numa_buffers_per_node == -1)
 		return 0;
 
+	/* no NUMA-aware partitioning */
+	if ((numa_flags & NUMA_BUFFERS) == 0)
+		return 0;
+
 	return (buffer / numa_buffers_per_node);
 }
 
@@ -460,10 +464,15 @@ buffer_partitions_prepare(void)
 	numa_can_partition = true;	/* assume we can allocate to nodes */
 	if (numa_nodes > max_nodes)
 	{
-		elog(LOG, "shared buffers too small for %d nodes (max nodes %d)",
+		elog(NOTICE, "shared buffers too small for %d nodes (max nodes %d)",
 			 numa_nodes, max_nodes);
 		numa_can_partition = false;
 	}
+	else if ((numa_flags & NUMA_BUFFERS) == 0)
+	{
+		elog(NOTICE, "NUMA-partitioning of buffers disabled");
+		numa_can_partition = false;
+	}
 
 	/*
 	 * We know we can partition to the desired number of nodes, now it's time
@@ -483,8 +492,16 @@ buffer_partitions_prepare(void)
 
 	/*
 	 * Finally, calculate how many buffers we'll assign to a single NUMA node.
-	 * If we have only a single node, or can't map to that many nodes, just
-	 * take a "fair share" of buffers.
+	 * If we have only a single node, or when we can't partition for some
+	 * reason, just take a "fair share" of buffers. This can happen for a
+	 * number of reasons - missing NUMA support, partitioning of buffers not
+	 * enabled, or not enough buffers for this many nodes.
+	 *
+	 * We still build partitions, because we want to allow partitioning of
+	 * the clock-sweep later.
+	 *
+	 * The number of buffers for each partition is calculated later, once we
+	 * have allocated the shared memory (because that's where we store it).
 	 *
 	 * XXX In both cases the last node can get fewer buffers.
 	 */
@@ -599,7 +616,7 @@ buffer_partitions_init(void)
 			Assert((num_buffers > 0) && (num_buffers <= part_buffers));
 
 			/* XXX we should get the actual node ID from the mask */
-			if ((numa_flags & NUMA_BUFFERS) != 0)
+			if (numa_can_partition)
 				part->numa_node = n;
 			else
 				part->numa_node = -1;
-- 
2.51.0

From 30a104a5b3e9567f2410b75d5fefb57d2144e606 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 15 Oct 2025 16:33:55 +0200
Subject: [PATCH v20251015 09/12] fix: move memset after PGPROC partitioning

The memset faults the pages into memory, which interferes with the NUMA
(see e.g. the requirements for numa_interleave_memory). Reported by
Alexey Makhmutov.

Discussion: [email-masked]
---
 src/backend/storage/lmgr/proc.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index 70ccfebef55..56812a05860 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -368,8 +368,6 @@ InitProcGlobal(void)
 						  requestSize,
 						  &found);
 
-	MemSet(ptr, 0, requestSize);
-
 	/* allprocs (array of pointers to PGPROC entries) */
 	procs = (PGPROC **) ptr;
 	ptr = (char *) ptr + CACHELINEALIGN(TotalProcs * sizeof(PGPROC *));
@@ -458,6 +456,12 @@ InitProcGlobal(void)
 		Assert((ptr > (char *) procs) && (ptr <= (char *) procs + requestSize));
 	}
 
+	/*
+	 * Don't memset the memory before locating it to NUMA nodes (which requires
+	 * the pages to be allocated but not yet faulted in memory).
+	 */
+	MemSet(ptr, 0, requestSize);
+
 	/*
 	 * Allocate arrays mirroring PGPROC fields in a dense manner. See
 	 * PROC_HDR.
-- 
2.51.0

From e8f69161eb459f89c8befd2da052f7a6bc0598ea Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 15 Oct 2025 16:52:53 +0200
Subject: [PATCH v20251015 10/12] fix: pgproc partitioning - align size

I'm not sure this is actually required. The libnuma docs say:

  The size argument will be rounded up to a multiple of the system page size.

so why should it be up to us to round it like that?

Discussion: [email-masked]
---
 src/backend/storage/lmgr/proc.c | 25 +++++++++++++++----------
 1 file changed, 15 insertions(+), 10 deletions(-)

diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index 56812a05860..fc5e1969c36 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -392,6 +392,9 @@ InitProcGlobal(void)
 		Assert(numa_procs_per_node > 0);
 		Assert(numa_nodes > 0);
 
+		/* make sure to align the PGPROC array to memory page */
+		ptr = (char *) TYPEALIGN(numa_page_size, ptr);
+
 		/*
 		 * Now initialize the PGPROC partition registry with one partition
 		 * per NUMA node (and then one extra partition for auxiliary procs).
@@ -401,9 +404,6 @@ InitProcGlobal(void)
 			/* the last NUMA node may get fewer PGPROC entries, but meh */
 			node_procs = Min(numa_procs_per_node, MaxBackends - total_procs);
 
-			/* make sure to align the PGPROC array to memory page */
-			ptr = (char *) TYPEALIGN(numa_page_size, ptr);
-
 			/* fill in the partition info */
 			partitions[i].num_procs = node_procs;
 			partitions[i].numa_node = i;
@@ -411,6 +411,9 @@ InitProcGlobal(void)
 
 			ptr = pgproc_partition_init(ptr, node_procs, total_procs, i);
 
+			/* should have been aligned */
+			Assert(ptr == (char *) TYPEALIGN(numa_page_size, ptr));
+
 			total_procs += node_procs;
 
 			/* don't underflow/overflow the allocation */
@@ -425,9 +428,6 @@ InitProcGlobal(void)
 		 */
 		node_procs = (NUM_AUXILIARY_PROCS + max_prepared_xacts);
 
-		/* make sure to align the PGPROC array to memory page */
-		ptr = (char *) TYPEALIGN(numa_page_size, ptr);
-
 		/* fill in the partition info */
 		partitions[numa_nodes].num_procs = node_procs;
 		partitions[numa_nodes].numa_node = -1;
@@ -2452,7 +2452,7 @@ pgproc_partitions_prepare(void)
 }
 
 /*
- * doesn't do alignment
+ *
  */
 static char *
 pgproc_partition_init(char *ptr, int num_procs, int allprocs_index, int node)
@@ -2465,15 +2465,20 @@ pgproc_partition_init(char *ptr, int num_procs, int allprocs_index, int node)
 	/* pointer right after this array */
 	ptr = (char *) ptr + num_procs * sizeof(PGPROC);
 
-	elog(DEBUG1, "NUMA: pgproc_init_partition procs %p endptr %p num_procs %d node %d",
-		 procs_node, ptr, num_procs, node);
-
 	/*
 	 * if node specified, move to node - do this before we start touching the
 	 * memory, to make sure it's not mapped to any node yet
 	 */
 	if (node != -1)
+	{
+		/* align the pointer to the next page */
+		ptr = (char *) TYPEALIGN(numa_page_size, ptr);
+
 		pg_numa_move_to_node((char *) procs_node, ptr, node);
+	}
+
+	elog(DEBUG1, "NUMA: pgproc_init_partition procs %p endptr %p num_procs %d node %d",
+		 procs_node, ptr, num_procs, node);
 
 	/* add pointers to the PGPROC entries to allProcs */
 	for (int i = 0; i < num_procs; i++)
-- 
2.51.0

From 21ec8c1801857206947c6b08701493a05e5c82cb Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 15 Oct 2025 17:32:29 +0200
Subject: [PATCH v20251015 11/12] fix: add padding for pgproc partitions

same as for fast-path locks
---
 src/backend/storage/lmgr/proc.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index fc5e1969c36..cd938adbc33 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -163,6 +163,11 @@ PGProcShmemSize(void)
 	 *
 	 * XXX It might be more painful with very large huge pages (e.g. 1GB).
 	 */
+	if (((numa_flags & NUMA_PROCS) != 0) && numa_can_partition)
+	{
+		Assert(numa_nodes > 0);
+		size = add_size(size, mul_size((numa_nodes + 1), numa_page_size));
+	}
 
 	return size;
 }
-- 
2.51.0

From 1b0238f4778789cb1f8fed5be1aa0fe0099edce3 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 15 Oct 2025 17:59:26 +0200
Subject: [PATCH v20251015 12/12] fix: clock-sweep log level

---
 src/backend/storage/buffer/freelist.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index 7f241301dcb..2de5cd4439e 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -769,7 +769,7 @@ StrategySyncBalance(void)
 	 */
 	if (avg_allocs < 100)
 	{
-		elog(LOG, "rebalance skipped: not enough allocations (allocs: %u)",
+		elog(INFO, "rebalance skipped: not enough allocations (allocs: %u)",
 			 avg_allocs);
 		return;
 	}
@@ -783,7 +783,7 @@ StrategySyncBalance(void)
 	 */
 	if (delta_allocs < (avg_allocs * 0.1))
 	{
-		elog(LOG, "rebalance skipped: delta within limit (delta: %u, threshold: %u)",
+		elog(INFO, "rebalance skipped: delta within limit (delta: %u, threshold: %u)",
 			 delta_allocs, (uint32) (avg_allocs * 0.1));
 		return;
 	}
-- 
2.51.0

---

From: Tomas Vondra <[email-masked]>
Date: Wed, 15 Oct 2025 19:15:47 +0200

> a patch to support this. I suppose the commit you linked is a step in
On second thought, I probably spoke too soon ...

What I wanted to keep out of scope for v1 is ability to pick NUMA nodes
from Postgres, e.g. setting a GUC to limit which NUMA nodes to use, etc.

But that's not what you proposed here, clearly. You're saying we should
find which NUMA nodes the process is allowed to run, and use those.
Instead of just using all *configured* nodes. And I agree with that.

I'll take a look at your commit 9ec625c. I'm not sure it's a good idea
to have our internal "logical" node ID, and a mapping to external node
ID values (exposed by the libnuma). I was thinking maybe we should use
just the external IDs, but it's true that'd be tricky when iterating
through nodes, etc. So maybe having such mapping is a good approach.

Another thing I wasn't sure about is checking for memory-only nodes. For
example rpi5 has a NUMA node for each 1GB of memory, and each CPU is
mapped to all those nodes. For buffers this probably does not matter,
but we probably should not use those NUMA nodes for PGPROC partitioning.

regards

-- 
Tomas Vondra

---

From: Tomas Vondra <[email-masked]>
Date: Fri, 31 Oct 2025 12:57:33 +0100

Hi,

here's a significantly reworked version of this patch series.

I had a couple discussions about these patches at pgconf.eu last week,
and one interesting suggestion was that maybe it'd be easier to the
clock-sweep partitioning first, in a NUMA-oblivious way. And then add
the NUMA stuff later.

The logic is that this way we could ignore some of the hard stuff (e.g.
handling huge page reservation), while still reducing clocksweep
contention. Which we speculated might be the main benefit anyway.

The attached patches do this.

0001 - Introduces a simplified version of the "buffer partition
registry" (think array in shmem, storing info about ranges of shared
buffer). The partitions are calculated as simple fraction of shared
buffers. There's no need to align the partitions to memory pages etc.

0002-0005 - Does the clock-sweep partitioning. I chose to keep this
split into smaller increments, to keep the patches easier to review.

0006 - Make the partitioning NUMA-aware. This used to be part of 0001,
but now it's moved on top of the clock-sweep stuff. It ensures the
partitions are properly aligned to memory pages, and all that.

0007 - PGPROC partitioning.

This made the 0001 patch much simpler/smaller - it used to be ~50kB, now
it's 15kB (and most of the complexity is in 0006).

The question however is how this performs, or how much of the benefit
was due to NUMA-awareness and how much was due to just partitioning
clock-sweep. I repeated the benchmark from [1], doing concurrent
sequential scans to put significant pressure on buffer replacements, and
I got this:

    hp   clients |  master  | sweep   sweep-16  |  numa  numa-16
    =============|==========|===================|===============
    off       16 |      24  |    46         46  |    33       40
              32 |      33  |    53         51  |    45       51
              48 |      38  |    51         61  |    46       56
              64 |      41  |    56         75  |    47       65
              80 |      47  |    53         77  |    48       71
              96 |      45  |    54         80  |    47       66
             112 |      45  |    52         83  |    44       65
             128 |      43  |    55         81  |    39       48
    -------------|----------|-------------------|---------------
    on        16 |      26  |    47         47  |    35       42
              32 |      33  |    49         52  |    40       49
              48 |      39  |    52         63  |    43       57
              64 |      42  |    53         72  |    43       66
              80 |      43  |    54         81  |    46       71
              96 |      48  |    58         80  |    49       73
             112 |      51  |    58         78  |    51       76
             128 |      55  |    60         83  |    52       76

"hp" means huge pages, the compared branches are:

- master - current master
- sweep - patches up to 0005, default number of partitions (4)
- sweep-16 - patches up to 0005, 16 partitions
- numa - patches up to 0006, default number of partitions (4)
- numa-16 - patches up to 0006, 16 partitions

Compared to master, the results look like this:

    hp   clients  |  sweep   sweep-16  |  numa   numa-16
    ==============|====================|================
    off       16  |   192%       192%  |  138%      167%
              32  |   161%       155%  |  136%      155%
              48  |   132%       160%  |  121%      145%
              64  |   137%       183%  |  115%      159%
              80  |   113%       164%  |  102%      151%
              96  |   120%       177%  |  104%      146%
             112  |   116%       184%  |   98%      144%
             128  |   128%       186%  |   90%      110%
    --------------|--------------------|----------------
    on        16  |   181%       181%  |  135%      162%
              32  |   148%       158%  |  121%      148%
              48  |   133%       161%  |  110%      144%
              64  |   126%       171%  |  102%      157%
              80  |   126%       188%  |  107%      165%
              96  |   121%       167%  |  102%      152%
             112  |   114%       153%  |  100%      149%
             128  |   109%       151%  |   95%      138%

The attached PDF has more results for runs with somewhat modified
parameters, but the overall it's very similar to these numbers.

I think this confirms most of the benefit really comes from just
partitioning clock-sweep, and it's mostly independent of the NUMA stuff.
In fact, the NUMA partitioning is often slower. Some of this may be due
to inefficiencies in the patch (e.g. division in formula calculating the
partition index, etc.).

So I think this looks quite promising ...

There are a couple unsolved issues, though. While running the tests, I
ran into a bunch of weird issues. I saw two types of failures:

1) Bad address
-----------------------------------------------------------------------
2025-10-30 15:24:21.195 UTC [2038558] LOG:  could not read blocks
114543..114558 in file "base/16384/16588": Bad address
2025-10-30 15:24:21.195 UTC [2038558] STATEMENT:  SELECT * FROM t_41
OFFSET 1000000000

2025-10-30 15:24:21.195 UTC [2038523] LOG:  could not read blocks
119981..119996 in file "base/16384/16869": Bad address
2025-10-30 15:24:21.195 UTC [2038523] CONTEXT:  completing I/O on behalf
of process 2038464
2025-10-30 15:24:21.195 UTC [2038523] STATEMENT:  SELECT * FROM t_96
OFFSET 1000000000

2025-10-30 15:24:21.195 UTC [2038492] LOG:  could not read blocks
118226..118232 in file "base/16384/16478": Bad address
2025-10-30 15:24:21.195 UTC [2038492] STATEMENT:  SELECT * FROM t_19
OFFSET 1000000000

2025-10-30 15:24:21.196 UTC [2038477] LOG:  could not read blocks
120515..120517 in file "base/16384/16945": Bad address
2025-10-30 15:24:21.196 UTC [2038477] CONTEXT:  completing I/O on behalf
of process 2038545
2025-10-30 15:24:21.196 UTC [2038477] STATEMENT:  SELECT * FROM t_111
OFFSET 1000000000
-----------------------------------------------------------------------

2) Operation canceled
-----------------------------------------------------------------------
2025-10-31 10:57:21.742 UTC [2685933] LOG:  could not read blocks
159..174 in file "base/16384/16398": Operation canceled
2025-10-31 10:57:21.742 UTC [2685933] STATEMENT:  SELECT * FROM t_3
OFFSET 1000000000

2025-10-31 10:57:21.742 UTC [2685933] LOG:  could not read blocks
143..158 in file "base/16384/16398": Operation canceled
2025-10-31 10:57:21.742 UTC [2685933] STATEMENT:  SELECT * FROM t_3
OFFSET 1000000000

2025-10-31 10:57:21.781 UTC [2685933] ERROR:  could not read blocks
143..158 in file "base/16384/16398": Operation canceled
2025-10-31 10:57:21.781 UTC [2685933] STATEMENT:  SELECT * FROM t_3
OFFSET 1000000000
-----------------------------------------------------------------------

I'm still not sure what's causing these, and it's happening rarely and
randomly, so it's hard to catch and reproduce. I'd welcome suggestions
what to look for / what might be the issue.

I did run the whole test under valgrind to make sure there's nothing
obviously broken, but that found no issues. Of course, it's much slower
under valgrind, so maybe it just didn't hit the issue.

I suspect the "bad address" might be just a different symptom of the
issues with reserving huge pages I already mentioned [2]. I assume
io_uring might try using huge pages internally, and then it fails
because postgres also reserves huge pages.

I have no idea what "operation canceled" might be about.

I'm not entirely sure if this affect all patches, or just the patches
with NUMA partitioning. Or if this happens with huge pages. I'll do more
runs to test specifically this.

But it does seem to be specific to io_uring - or at least the canceled
issue. I haven't seen it after switching to "worker".

[1]
https://www.postgresql.org/message-id/51e51832-7f47-412a-a1a6-b972101cc8cb%40vondra.me

[2]
https://www.postgresql.org/message-id/1d57d68d-b178-415a-ba11-be0c3714638e%40vondra.me

regards

-- 
Tomas Vondra
From 5ecfc8c4f042219b4691e5f0770c72f55c4830ac Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Mon, 8 Sep 2025 13:11:02 +0200
Subject: [PATCH v20251101 7/7] NUMA: partition PGPROC

 - PGPROC array / node #0
 - PGPROC array / node #1

Note: There's some challenges in making this work on EXEC_BACKEND, even
if we don't support NUMA on platforms that require this.
---
 .../pg_buffercache--1.6--1.7.sql              |  19 +
 contrib/pg_buffercache/pg_buffercache_pages.c |  96 ++-
 src/backend/access/transam/clog.c             |   4 +-
 src/backend/access/transam/twophase.c         |   3 +-
 src/backend/postmaster/launch_backend.c       |   4 +-
 src/backend/postmaster/pgarch.c               |   2 +-
 src/backend/postmaster/walsummarizer.c        |   2 +-
 src/backend/storage/buffer/buf_init.c         |   2 +
 src/backend/storage/buffer/freelist.c         |   2 +-
 src/backend/storage/ipc/procarray.c           |  85 ++-
 src/backend/storage/lmgr/lock.c               |   6 +-
 src/backend/storage/lmgr/proc.c               | 550 +++++++++++++++++-
 src/include/port/pg_numa.h                    |   1 +
 src/include/storage/proc.h                    |  18 +-
 src/tools/pgindent/typedefs.list              |   1 +
 15 files changed, 722 insertions(+), 73 deletions(-)

index 61207357e53..a859962f5f8 100644
@@ -29,7 +30,8 @@
 
 #define NUM_BUFFERCACHE_NUMA_ELEM	3
-#define NUM_BUFFERCACHE_PARTITIONS_ELEM	11
+#define NUM_BUFFERCACHE_PARTITIONS_ELEM	12
+#define NUM_BUFFERCACHE_PGPROC_ELEM	5
 
 PG_MODULE_MAGIC_EXT(

 		/*

 
index ce6b5299324..3288900bb6f 100644
@@ -292,7 +292,7 @@ PgArchWakeup(void)

 
 /*
diff --git a/src/backend/storage/buffer/buf_init.c b/src/backend/storage/buffer/buf_init.c
index f8314a0e299..d0efa102d82 100644
--- a/src/backend/storage/buffer/buf_init.c
+++ b/src/backend/storage/buffer/buf_init.c
@@ -799,6 +799,8 @@ check_debug_numa(char **newval, void **extra, GucSource source)
 
index 341eaf55577..2de5cd4439e 100644
@@ -472,7 +472,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 
 	/*

 
 /*

+	AssertCheckAllProcs();
+

 

+	AssertCheckAllProcs();
+
 	/*
 
+	AssertCheckAllProcs();
+
 	myoff = proc->pgxactoff;

 

+	AssertCheckAllProcs();
+
 	/*

 
 
-
 /*

 			/*

 

 			/*

 

 		if (proc->pid == pid)
 		{
 
 			result = proc->pid;
 			break;
 
 		if (proc == MyProc)

 
 		/*
 
 		if (proc->pid == 0)
 
 		if (proc->pid == 0)

 		if (proc->pid == 0)

 

 
index 96f29aafc39..cd938adbc33 100644
 
+#ifdef USE_LIBNUMA
+#include <sched.h>
+#endif
+
 #include <signal.h>
 #include <unistd.h>
 #include <sys/time.h>
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+

 

@@ -100,11 +134,41 @@ PGProcShmemSize(void)

+	/*
 
@@ -129,6 +193,60 @@ FastPathLockShmemSize(void)

+	/*
 
@@ -140,12 +258,16 @@ ProcGlobalShmemSize(void)
 {
 	Size		size = 0;

 
 	return size;
 }
@@ -191,7 +313,7 @@ ProcGlobalSemas(void)
 void
 InitProcGlobal(void)
 {
-	PGPROC	   *procs;
+	PGPROC	  **procs;
 	int			i,
 				j;
 	bool		found;
@@ -210,6 +332,9 @@ InitProcGlobal(void)
 		ShmemInitStruct("Proc Header", sizeof(PROC_HDR), &found);
 	Assert(!found);
 
@@ -224,6 +349,15 @@ InitProcGlobal(void)
 
@@ -239,21 +373,110 @@ InitProcGlobal(void)
 						  requestSize,
 						  &found);

 
+	/*
@@ -286,24 +509,92 @@ InitProcGlobal(void)

 
 		/*

 
@@ -366,9 +657,6 @@ InitProcGlobal(void)
 
@@ -435,7 +723,51 @@ InitProcess(void)
 
@@ -646,7 +978,7 @@ InitAuxiliaryProcess(void)
@@ -1049,7 +1381,7 @@ AuxiliaryProcKill(int code, Datum arg)

 
@@ -1098,7 +1430,7 @@ AuxiliaryPidGetProc(int pid)

 		if (proc->pid == pid)
 		{
@@ -1988,7 +2320,7 @@ ProcSendSignal(ProcNumber procNumber)
 	if (procNumber < 0 || procNumber >= ProcGlobal->allProcCount)
 		elog(ERROR, "procNumber out of range");

 /*
@@ -2063,3 +2395,173 @@ BecomeLockGroupMember(PGPROC *leader, int pid)
 
 	return ok;
 }
+
+/*

 #ifdef USE_LIBNUMA

 

 

 /*
 
 /*
 
 #ifdef EXEC_BACKEND

index f7730ece976..be4a6334fb7 100644

From cff7ec32ca1256227d4c2ea51c61986b10c0f28d Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 29 Oct 2025 21:41:26 +0100
Subject: [PATCH v20251101 6/7] NUMA: shared buffers partitioning

The effect is similar to

     numactl --interleave=all

The "buffer partitions" may not be 1:1 with NUMA nodes. There's a
minimal number of partitions (default: 4) that will be created even with
fewer NUMA nodes, or no NUMA at all. Each node gets the same number of
partitions, to keep things simple. For example, with 2 nodes there'll be
4 partitions, with each node getting 2 of them. With 3 nodes there'll be
6 partitions (again, 2 per node).
---
 .../pg_buffercache--1.6--1.7.sql              |   1 +
 contrib/pg_buffercache/pg_buffercache_pages.c |  44 +-
 src/backend/storage/buffer/buf_init.c         | 569 +++++++++++++++++-
 src/backend/storage/buffer/freelist.c         |  88 ++-
 src/backend/utils/misc/guc_parameters.dat     |  10 +
 src/backend/utils/misc/guc_tables.c           |   1 +
 src/include/port/pg_numa.h                    |   6 +
 src/include/storage/buf_internals.h           |  14 +-
 src/include/storage/bufmgr.h                  |   4 +
 src/include/utils/guc_hooks.h                 |   3 +
 src/port/pg_numa.c                            |  64 ++
 11 files changed, 736 insertions(+), 68 deletions(-)

diff --git a/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql b/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
index 2c4d560514d..dc2ce019283 100644
--- a/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
+++ b/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
@@ -13,6 +13,7 @@ LANGUAGE C PARALLEL SAFE;
 CREATE VIEW pg_buffercache_partitions AS
 	SELECT P.* FROM pg_buffercache_partitions() AS P
 	(partition integer,			-- partition index
+	 numa_node integer,			-- NUMA node of the partitioon
 	 num_buffers integer,		-- number of buffers in the partition
 	 first_buffer integer,		-- first buffer of partition
 	 last_buffer integer,		-- last buffer of partition
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index b0b9112fdba..61207357e53 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -814,19 +814,21 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
 		TupleDescInitEntry(tupledesc, (AttrNumber) 1, "partition",
 						   INT4OID, -1, 0);
-		TupleDescInitEntry(tupledesc, (AttrNumber) 2, "num_buffers",
+		TupleDescInitEntry(tupledesc, (AttrNumber) 2, "numa_node",
 						   INT4OID, -1, 0);
-		TupleDescInitEntry(tupledesc, (AttrNumber) 3, "first_buffer",
+		TupleDescInitEntry(tupledesc, (AttrNumber) 3, "num_buffers",
 						   INT4OID, -1, 0);
-		TupleDescInitEntry(tupledesc, (AttrNumber) 4, "last_buffer",
+		TupleDescInitEntry(tupledesc, (AttrNumber) 4, "first_buffer",
 						   INT4OID, -1, 0);
-		TupleDescInitEntry(tupledesc, (AttrNumber) 5, "num_passes",
+		TupleDescInitEntry(tupledesc, (AttrNumber) 5, "last_buffer",
+						   INT4OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 6, "num_passes",
 						   INT8OID, -1, 0);
-		TupleDescInitEntry(tupledesc, (AttrNumber) 6, "next_buffer",
+		TupleDescInitEntry(tupledesc, (AttrNumber) 7, "next_buffer",
 						   INT4OID, -1, 0);
-		TupleDescInitEntry(tupledesc, (AttrNumber) 7, "total_allocs",
+		TupleDescInitEntry(tupledesc, (AttrNumber) 8, "total_allocs",
 						   INT8OID, -1, 0);
-		TupleDescInitEntry(tupledesc, (AttrNumber) 8, "num_allocs",
+		TupleDescInitEntry(tupledesc, (AttrNumber) 9, "num_allocs",
 						   INT8OID, -1, 0);
 		TupleDescInitEntry(tupledesc, (AttrNumber) 10, "total_req_allocs",
 						   INT8OID, -1, 0);
@@ -850,7 +852,8 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 	{
 		uint32		i = funcctx->call_cntr;
 
-		int			num_buffers,
+		int			numa_node,
+					num_buffers,
 					first_buffer,
 					last_buffer;
 
@@ -869,7 +872,7 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		Datum		values[NUM_BUFFERCACHE_PARTITIONS_ELEM];
 		bool		nulls[NUM_BUFFERCACHE_PARTITIONS_ELEM];
 
-		BufferPartitionGet(i, &num_buffers,
+		BufferPartitionGet(i, &numa_node, &num_buffers,
 						   &first_buffer, &last_buffer);
 
 		ClockSweepPartitionGetInfo(i,
@@ -887,36 +890,39 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		values[0] = Int32GetDatum(i);
 		nulls[0] = false;
 
-		values[1] = Int32GetDatum(num_buffers);
+		values[1] = Int32GetDatum(numa_node);
 		nulls[1] = false;
 
-		values[2] = Int32GetDatum(first_buffer);
+		values[2] = Int32GetDatum(num_buffers);
 		nulls[2] = false;
 
-		values[3] = Int32GetDatum(last_buffer);
+		values[3] = Int32GetDatum(first_buffer);
 		nulls[3] = false;
 
-		values[4] = Int64GetDatum(complete_passes);
+		values[4] = Int32GetDatum(last_buffer);
 		nulls[4] = false;
 
-		values[5] = Int32GetDatum(next_victim_buffer);
+		values[5] = Int64GetDatum(complete_passes);
 		nulls[5] = false;
 
-		values[6] = Int64GetDatum(buffer_total_allocs);
+		values[6] = Int32GetDatum(next_victim_buffer);
 		nulls[6] = false;
 
-		values[7] = Int64GetDatum(buffer_allocs);
+		values[7] = Int64GetDatum(buffer_total_allocs);
 		nulls[7] = false;
 
-		values[8] = Int64GetDatum(buffer_total_req_allocs);
+		values[8] = Int64GetDatum(buffer_allocs);
 		nulls[8] = false;
 
-		values[9] = Int64GetDatum(buffer_req_allocs);
+		values[9] = Int64GetDatum(buffer_total_req_allocs);
 		nulls[9] = false;
 
-		values[10] = PointerGetDatum(array);
+		values[10] = Int64GetDatum(buffer_req_allocs);
 		nulls[10] = false;
 
index 0362fda24aa..f8314a0e299 100644
@@ -14,6 +14,12 @@
  */
 #include "postgres.h"
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
@@ -29,15 +35,24 @@ ConditionVariableMinimallyPadded *BufferIOCVArray;
 
-/* *
- * number of buffer partitions */
-#define NUM_CLOCK_SWEEP_PARTITIONS	4
+/*
+ * Minimum number of buffer partitions, no matter the number of NUMA nodes.
+ */
+#define MIN_BUFFER_PARTITIONS	4
 
 /* Array of structs with information about buffer ranges */
 BufferPartitions *BufferPartitionsArray = NULL;
 
+static void buffer_partitions_prepare(void);
 static void buffer_partitions_init(void);
 
@@ -85,25 +100,85 @@ BufferManagerShmemInit(void)
 				foundBufCkpt,
 				foundParts;
 
 	/* allocate the partition registry first */
 	BufferPartitionsArray = (BufferPartitions *)
 		ShmemInitStruct("Buffer Partitions",
 						offsetof(BufferPartitions, partitions) +
-						mul_size(sizeof(BufferPartition), NUM_CLOCK_SWEEP_PARTITIONS),
+						mul_size(sizeof(BufferPartition), numa_partitions),
 						&foundParts);

 
 	/* Align condition variables to cacheline boundary. */
@@ -133,7 +208,10 @@ BufferManagerShmemInit(void)
 	{
 		int			i;
 
-		/* Initialize buffer partitions (calculate buffer ranges). */
+		/*
+		 * Initialize buffer partitions, including moving memory to different
+		 * NUMA nodes (if enabled by GUC).
+		 */
 		buffer_partitions_init();
 
 		/*
@@ -172,19 +250,26 @@ BufferManagerShmemInit(void)
  *

 
 	/* size of stuff controlled by freelist.c */
@@ -201,11 +286,244 @@ BufferManagerShmemSize(void)
 
 	/* account for registry of NUMA partitions */
 	size = add_size(size, MAXALIGN(offsetof(BufferPartitions, partitions) +
-								   mul_size(sizeof(BufferPartition), NUM_CLOCK_SWEEP_PARTITIONS)));
+								   mul_size(sizeof(BufferPartition), numa_partitions)));
 
 	return size;
 }
 
+/*
  * Sanity checks of buffers partitions - there must be no gaps, it must cover
  * the whole range of buffers, etc.
@@ -267,33 +585,137 @@ buffer_partitions_init(void)
 	int			remaining_buffers = NBuffers;
 	int			buffer = 0;
 
-	/* number of buffers per partition (make sure to not overflow) */
-	int			part_buffers
-		= ((int64) NBuffers + (NUM_CLOCK_SWEEP_PARTITIONS - 1)) / NUM_CLOCK_SWEEP_PARTITIONS;
-
-	BufferPartitionsArray->npartitions = NUM_CLOCK_SWEEP_PARTITIONS;
+	BufferPartitionsArray->npartitions = numa_partitions;
+	BufferPartitionsArray->nnodes = numa_nodes;
 
-	for (int n = 0; n < BufferPartitionsArray->npartitions; n++)
+	for (int n = 0; n < numa_nodes; n++)
 	{
-		BufferPartition *part = &BufferPartitionsArray->partitions[n];
+		/* buffers this node should get (last node can get fewer) */
+		int			node_buffers = Min(remaining_buffers, numa_buffers_per_node);
 
-		/* buffers this partition should get (last partition can get fewer) */
-		int			num_buffers = Min(remaining_buffers, part_buffers);
+		/* split node buffers netween partitions (last one can get fewer) */
+		int			part_buffers = (node_buffers + (parts_per_node - 1)) / parts_per_node;
 
-		remaining_buffers -= num_buffers;
+		remaining_buffers -= node_buffers;
 
-		Assert((num_buffers > 0) && (num_buffers <= part_buffers));
-		Assert((buffer >= 0) && (buffer < NBuffers));
+		Assert((node_buffers > 0) && (node_buffers <= NBuffers));
+		Assert((n >= 0) && (n < numa_nodes));
+
+		for (int p = 0; p < parts_per_node; p++)
+		{
+			int			idx = (n * parts_per_node) + p;
+			BufferPartition *part = &BufferPartitionsArray->partitions[idx];
+			int			num_buffers = Min(node_buffers, part_buffers);
 
-		part->num_buffers = num_buffers;
-		part->first_buffer = buffer;
-		part->last_buffer = buffer + (num_buffers - 1);
+			Assert((idx >= 0) && (idx < numa_partitions));
+			Assert((buffer >= 0) && (buffer < NBuffers));
+			Assert((num_buffers > 0) && (num_buffers <= part_buffers));
 
-		buffer += num_buffers;
 
 	AssertCheckBufferPartitions();
+
+	/*
 
 int
@@ -302,14 +724,21 @@ BufferPartitionCount(void)
 	return BufferPartitionsArray->npartitions;
 }
 
+int
+BufferPartitionNodes(void)
+{
+	return BufferPartitionsArray->nnodes;
+}
+
 void
-BufferPartitionGet(int idx, int *num_buffers,
+BufferPartitionGet(int idx, int *node, int *num_buffers,
 				   int *first_buffer, int *last_buffer)
 {
 	if ((idx >= 0) && (idx < BufferPartitionsArray->npartitions))
 	{
 		BufferPartition *part = &BufferPartitionsArray->partitions[idx];
 
+		*node = part->numa_node;
 		*num_buffers = part->num_buffers;
 		*first_buffer = part->first_buffer;
 		*last_buffer = part->last_buffer;
@@ -322,8 +751,82 @@ BufferPartitionGet(int idx, int *num_buffers,
 
 /* return parameters before the partitions are initialized (during sizing) */
 void
-BufferPartitionParams(int *num_partitions)
+BufferPartitionParams(int *num_partitions, int *num_nodes)
 {
 	if (num_partitions)
-		*num_partitions = NUM_CLOCK_SWEEP_PARTITIONS;
+		*num_partitions = numa_partitions;
+
+	if (num_nodes)
+		*num_nodes = numa_nodes;
+}
+
+/* XXX the GUC hooks should probably be somewhere else? */
+bool
+check_debug_numa(char **newval, void **extra, GucSource source)
+{
+	bool		result = true;
+	int			flags;
+
+#if USE_LIBNUMA == 0
+	if (strcmp(*newval, "") != 0)
+	{
+		GUC_check_errdetail("\"%s\" is not supported on this platform.",
+							"debug_numa");
+		result = false;
+	}
+	flags = 0;
+#else
+	List	   *elemlist;
+	ListCell   *l;
+	char	   *rawstring;
+
+	/* Need a modifiable copy of string */
+	rawstring = pstrdup(*newval);
+
+	if (!SplitGUCList(rawstring, ',', &elemlist))
+	{
+		GUC_check_errdetail("Invalid list syntax in parameter \"%s\".",
+							"debug_numa");
+		pfree(rawstring);
+		list_free(elemlist);
+		return false;
+	}
+
+	flags = 0;
+	foreach(l, elemlist)
+	{
+		char	   *item = (char *) lfirst(l);
+
+		if (pg_strcasecmp(item, "buffers") == 0)
+			flags |= NUMA_BUFFERS;
+		else
+		{
+			GUC_check_errdetail("Invalid option \"%s\".", item);
+			result = false;
+			break;
+		}
+	}
+
+	pfree(rawstring);
+	list_free(elemlist);
+#endif
+
+	if (!result)
+		return result;
+
+	/* Save the flags in *extra, for use by assign_debug_io_direct */
+	*extra = guc_malloc(LOG, sizeof(int));
+	if (!*extra)
+		return false;
+	*((int *) *extra) = flags;
+
+	return result;
+}
+
+void
+assign_debug_numa(const char *newval, void *extra)
+{
+	int		   *flags = (int *) extra;
+
+	numa_flags = *flags;
 }
diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index c73a418defa..341eaf55577 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -124,7 +124,9 @@ typedef struct
 	//int			__attribute__((aligned(64))) bgwprocno;
 
 	/* info about freelist partitioning */
+	int			num_nodes;		/* effectively number of NUMA nodes */
 	int			num_partitions;
+	int			num_partitions_per_node;
 
 	/* clocksweep partitions */
 	ClockSweep	sweeps[FLEXIBLE_ARRAY_MEMBER];
@@ -270,16 +272,72 @@ ClockSweepTick(ClockSweep *sweep)
  * calculate_partition_index
  *		calculate the buffer / clock-sweep partition to use
  *
- * use PID to determine the buffer partition
- *
- * XXX We could use NUMA node / core ID to pick partition, but we'd need
- * to handle cases with fewer nodes/cores than partitions somehow. Although,
- * maybe the balancing would handle that too.
+ * With libnuma, use the NUMA node and CPU to pick the partition. Otherwise
+ * use just PID instead of CPU (we assume everything is a single NUMA node).
  */
 static int
 calculate_partition_index(void)
 {
-	return (MyProcPid % StrategyControl->num_partitions);
+	int		cpu,
+			node,
+			index;
+
+	/*
+	 * The buffers are partitioned, so determine the CPU/NUMA node, and pick a
+	 * partition based on that.
+	 *
+	 * Without NUMA assume everything is a single NUMA node, and we pick the
+	 * partition based on PID (we may not have sched_getcpu).
+	 */
+#ifdef USE_LIBNUMA
+	cpu = sched_getcpu();
+
+	if (cpu < 0)
+		elog(ERROR, "sched_getcpu failed: %m");
+
+	node = numa_node_of_cpu(cpu);
+#else
+	cpu = MyProcPid;
+	node = 0;
+#endif
+
+	Assert(StrategyControl->num_partitions ==
+		   (StrategyControl->num_nodes * StrategyControl->num_partitions_per_node));
+
+	/*
+	 * XXX We should't get nodes that we haven't considered while building the
+	 * partitions. Maybe if we allow this (e.g. due to support adjusting the
+	 * NUMA stuff at runtime), we should just do our best to minimize the
+	 * conflicts somehow. But it'll make the mapping harder, so for now we
+	 * ignore it.
+	 */
+	if (node > StrategyControl->num_nodes)
+		elog(ERROR, "node out of range: %d > %u", cpu, StrategyControl->num_nodes);
+
+	/*
+	 * Find the partition. If we have a single partition per node, we can
+	 * calculate the index directly from node. Otherwise we need to do two
+	 * steps, using node and then cpu.
+	 */
+	if (StrategyControl->num_partitions_per_node == 1)
+	{
+		/* fast-path */
+		index = (node % StrategyControl->num_partitions);
+	}
+	else
+	{
+		int			index_group,
+					index_part;
+
+		/* two steps - calculate group from node, partition from cpu */
+		index_group = (node % StrategyControl->num_nodes);
+		index_part = (cpu % StrategyControl->num_partitions_per_node);
+
+		index = (index_group * StrategyControl->num_partitions_per_node)
+			+ index_part;
+	}
+
+	return index;
 }
 
 /*
@@ -947,7 +1005,7 @@ StrategyShmemSize(void)
 	Size		size = 0;
 	int			num_partitions;
 
-	BufferPartitionParams(&num_partitions);
+	BufferPartitionParams(&num_partitions, NULL);
 
 	/* size of lookup hash table ... see comment in StrategyInitialize */
 	size = add_size(size, BufTableShmemSize(NBuffers + NUM_BUFFER_PARTITIONS));
@@ -974,9 +1032,17 @@ StrategyInitialize(bool init)
 {
 	bool		found;
 
+	int			num_nodes;
 	int			num_partitions;
 
 	num_partitions = BufferPartitionCount();
 
 	/*
 	 * Initialize the shared buffer lookup hashtable.
@@ -1011,7 +1077,8 @@ StrategyInitialize(bool init)
 		/* Initialize the clock sweep pointers (for all partitions) */
 		for (int i = 0; i < num_partitions; i++)
 		{
-			int			num_buffers,
+			int			node,
+						num_buffers,
 						first_buffer,
 						last_buffer;
 
@@ -1020,7 +1087,8 @@ StrategyInitialize(bool init)
 			pg_atomic_init_u32(&StrategyControl->sweeps[i].nextVictimBuffer, 0);
 
 			/* get info about the buffer partition */
-			BufferPartitionGet(i, &num_buffers, &first_buffer, &last_buffer);
+			BufferPartitionGet(i, &node, &num_buffers,
+							   &first_buffer, &last_buffer);
 
 			/*
 			 * FIXME This may not quite right, because if NBuffers is not a
@@ -1056,6 +1124,8 @@ StrategyInitialize(bool init)
 
 		/* initialize the partitioned clocksweep */
 		StrategyControl->num_partitions = num_partitions;
+		StrategyControl->num_nodes = num_nodes;
+		StrategyControl->num_partitions_per_node = num_partitions_per_node;
 	}
 	else
 		Assert(!init);
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index d6fc8333850..1da2a43220a 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -906,6 +906,16 @@
   boot_val => 'true',
 },

 #ifdef HAVE_SYSLOG
 
 #ifdef USE_LIBNUMA
 
index 776be11dec1..4ae0c32dfe5 100644
 
 typedef union BufferDescPadded
 {
@@ -490,8 +490,8 @@ extern void AtEOXact_LocalBuffers(bool isCommit);
 
 extern int	BufferPartitionCount(void);
 extern int	BufferPartitionNodes(void);
-extern void BufferPartitionGet(int idx, int *num_buffers,
+extern void BufferPartitionGet(int idx, int *node, int *num_buffers,
 							   int *first_buffer, int *last_buffer);
-extern void BufferPartitionParams(int *num_partitions);
+extern void BufferPartitionParams(int *num_partitions, int *num_nodes);
 
 #endif							/* BUFMGR_INTERNALS_H */
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index 4e7b1fcd4ab..510018db115 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -156,10 +156,12 @@ typedef struct ReadBuffersOperation ReadBuffersOperation;
 /*
  * information about one partition of shared buffers
  *
+ * numa_nod specifies node for this partition (-1 means allocated on any node)
  * first/last buffer - the values are inclusive
  */
 typedef struct BufferPartition
 {
+	int			numa_node;		/* NUMA node (-1 no node) */
 	int			num_buffers;	/* number of buffers */
 	int			first_buffer;	/* first buffer of partition */
 	int			last_buffer;	/* last buffer of partition */
@@ -169,6 +171,7 @@ typedef struct BufferPartition
 typedef struct BufferPartitions
 {
 	int			npartitions;	/* number of partitions */
+	int			nnodes;			/* number of NUMA nodes */
 	BufferPartition partitions[FLEXIBLE_ARRAY_MEMBER];
 } BufferPartitions;
 
@@ -346,6 +349,7 @@ extern void EvictRelUnpinnedBuffers(Relation rel,
 /* in buf_init.c */
 extern void BufferManagerShmemInit(void);
 extern Size BufferManagerShmemSize(void);
+extern int	BufferGetNode(Buffer buffer);
 
 /* in localbuf.c */

 #include "miscadmin.h"
 
 /*
 
+/*
+ * pg_numa_move_to_node
 
 /* Empty wrappers */
 
+void

From 59aeeec92549e95c52446c59a1c1308d548d277b Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 6 Aug 2025 01:09:57 +0200
Subject: [PATCH v20251101 5/7] clock-sweep: weighted balancing

Note: This may be more important with NUMA-aware partitioning, which
restricts the allowed sizes of partiions (especially with huge pages).
---
 src/backend/storage/buffer/freelist.c | 63 ++++++++++++++++++++++-----
 1 file changed, 53 insertions(+), 10 deletions(-)

diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index 38f339b02c4..c73a418defa 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -619,6 +619,20 @@ StrategySyncBalance(void)
 			avg_allocs,			/* average allocations (per partition) */
 			delta_allocs = 0;	/* sum of allocs above average */
 
+	/*
@@ -645,16 +659,27 @@ StrategySyncBalance(void)

 	/*
 
 	/*
@@ -662,8 +687,14 @@ StrategySyncBalance(void)
 
 	/*
@@ -726,6 +757,10 @@ StrategySyncBalance(void)
 		ClockSweep *sweep = &StrategyControl->sweeps[i];
 		uint8		balance[MAX_BUFFER_PARTITIONS];

@@ -733,7 +768,7 @@ StrategySyncBalance(void)
 		memset(balance, 0, sizeof(uint8) * MAX_BUFFER_PARTITIONS);
 
@@ -747,22 +782,30 @@ StrategySyncBalance(void)

 

From e4252a2559496947f6862ee0851a88dd2f59ea11 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 15 Oct 2025 13:59:29 +0200
Subject: [PATCH v20251101 4/7] clock-sweep: scan all partitions

When looking for a free buffer, scan all clock-sweep partitions, not
just the "home" one. All buffers in the home partition may be pinned, in
which case we should not fail. Instead, advance to the next partition,
in a round-robin way, and only fail after scanning through all of them.
---
 src/backend/storage/buffer/freelist.c     | 91 ++++++++++++++++-------
 src/test/recovery/t/027_stream_regress.pl |  5 --
 2 files changed, 63 insertions(+), 33 deletions(-)

diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index 8897f2e9b0e..38f339b02c4 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -167,6 +167,9 @@ static BufferDesc *GetBufferFromRing(BufferAccessStrategy strategy,
 static void AddBufferToRing(BufferAccessStrategy strategy,
 							BufferDesc *buf);
 static ClockSweep *ChooseClockSweep(bool balance);
+static BufferDesc *StrategyGetBufferPartition(ClockSweep *sweep,
+											  BufferAccessStrategy strategy,
+											  uint32 *buf_state);
 
 /*
@@ -201,10 +204,9 @@ static int clocksweep_count = 0;
 
 	/*
 	 * Atomically move hand ahead one buffer - if there's several processes
@@ -370,7 +372,8 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 {
 	BufferDesc *buf;
 	int			bgwprocno;
-	int			trycounter;
+	ClockSweep *sweep,
+			   *sweep_start;		/* starting clock-sweep partition */
 
 	*from_ring = false;
 
@@ -424,37 +427,69 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 	/*

 		/*
@@ -482,7 +517,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
@@ -500,7 +535,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r

From a2cb64b97d17bf271ec0aa9496601b1d644a94e2 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 29 Oct 2025 21:45:34 +0100
Subject: [PATCH v20251101 3/7] clock-sweep: balancing of allocations

If backends only allocate buffers from the "home" partition, that may
cause significant misbalance. Some partitions might be overused, while
other partitions would be left unused. In other words, shared buffers
would not be used efficiently.

index 14e750beeff..2c4d560514d 100644
@@ -21,7 +21,10 @@ CREATE VIEW pg_buffercache_partitions AS
 
index 96eee21932f..b0b9112fdba 100644

@@ -27,7 +29,7 @@

 PG_MODULE_MAGIC_EXT(
 
 	if (SRF_IS_FIRSTCALL())
 	{
 
 		/* Switch context when allocating stuff to be used in later calls */
@@ -824,6 +828,12 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 						   INT8OID, -1, 0);
 		TupleDescInitEntry(tupledesc, (AttrNumber) 8, "num_allocs",
 						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 10, "total_req_allocs",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 11, "num_req_allocs",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 12, "weigths",
+						   typentry->typarray, -1, 0);

@@ -844,11 +854,17 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 					first_buffer,
 					last_buffer;

 		uint32		complete_passes,
 					next_victim_buffer,
-					buffer_allocs;
+					buffer_allocs,
+					buffer_req_allocs;
+
+		int		   *weights;
+		Datum	   *dweights;
+		ArrayType  *array;
 
 		Datum		values[NUM_BUFFERCACHE_PARTITIONS_ELEM];
 		bool		nulls[NUM_BUFFERCACHE_PARTITIONS_ELEM];
@@ -857,8 +873,16 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 						   &first_buffer, &last_buffer);

 		values[0] = Int32GetDatum(i);
 		nulls[0] = false;
@@ -884,6 +908,15 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		values[7] = Int64GetDatum(buffer_allocs);
 		nulls[7] = false;
 
+		values[8] = Int64GetDatum(buffer_total_req_allocs);
+		nulls[8] = false;
+
+		values[9] = Int64GetDatum(buffer_req_allocs);
+		nulls[9] = false;
+
+		values[10] = PointerGetDatum(array);
+		nulls[10] = false;
+
 		/* Build and return the tuple. */
 		tuple = heap_form_tuple((TupleDesc) funcctx->user_fctx, values, nulls);
 		result = HeapTupleGetDatum(tuple);
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 15ca6f60d57..176a6e6358c 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3913,6 +3913,9 @@ BgBufferSync(WritebackContext *wb_context)
 	/* assume we can hibernate, any partition can set to false */
 	bool		hibernate = true;

index 1d2f97b4d1a..8897f2e9b0e 100644

+/*
 
+	/*
 
+	/*
 
 /*
@@ -130,7 +166,33 @@ static BufferDesc *GetBufferFromRing(BufferAccessStrategy strategy,
 									 uint32 *buf_state);
 
 /*
  * ClockSweepTick - Helper routine for StrategyGetBuffer()
@@ -142,7 +204,7 @@ static inline uint32
 ClockSweepTick(void)
 {
 	uint32		victim;
-	ClockSweep *sweep = ChooseClockSweep();
+	ClockSweep *sweep = ChooseClockSweep(true);
 
 	/*
@@ -233,11 +295,59 @@ calculate_partition_index(void)

 /*
@@ -309,7 +419,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 
 	/*
 	 * Use the "clock sweep" algorithm to find a free buffer
@@ -417,6 +527,224 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 	}
 }
 
+/*
+ * StrategySyncBalance
@@ -443,6 +771,7 @@ StrategySyncPrepare(int *num_parts, uint32 *num_buf_alloc)
 
@@ -627,7 +956,21 @@ StrategyInitialize(bool init)
 
 		/* No pending notification */
@@ -1000,8 +1343,10 @@ StrategyRejectBuffer(BufferAccessStrategy strategy, BufferDesc *buf, bool from_r
 
 void
 
@@ -1009,11 +1354,21 @@ ClockSweepPartitionGetInfo(int idx,

 
index 1386856046c..776be11dec1 100644
 
+extern void StrategySyncBalance(void);
 extern void StrategySyncPrepare(int *num_parts, uint32 *num_buf_alloc);
 extern int	StrategySyncStart(int partition, uint32 *complete_passes,
 							  int *first_buffer, int *num_buffers);
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index 7052f9de57c..4e7b1fcd4ab 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -360,11 +360,13 @@ extern int	GetAccessStrategyPinLimit(BufferAccessStrategy strategy);

 /* inline functions */
 
-- 
2.51.0

From 1eb17c28164b30f759208b56eaa699236582c008 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 29 Oct 2025 21:39:47 +0100
Subject: [PATCH v20251101 2/7] clock-sweep: basic partitioning

Partitions the "clock-sweep" algorithm to work on individual partitions,
one by one. Each backend process is mapped to one "home" partition, with
an independent clock hand. This reduces contention for workloads with
significant buffer pressure.

The patch extends the "pg_buffercache_partitions" view to include
information about the clock-sweep activity.

Note: There's a problem with some tests running out of unpinned buffers,
due to (intentionally) setting shared buffers very low. That happens
because StrategyGetBuffer() only searches a single partition, and it
has a couple more issues.
---
 .../pg_buffercache--1.6--1.7.sql              |   8 +-
 contrib/pg_buffercache/pg_buffercache_pages.c |  38 ++-
 src/backend/storage/buffer/buf_init.c         |   7 +
 src/backend/storage/buffer/bufmgr.c           | 186 ++++++++----
 src/backend/storage/buffer/freelist.c         | 283 +++++++++++++++---
 src/include/storage/buf_internals.h           |   5 +-
 src/include/storage/bufmgr.h                  |   5 +
 src/test/recovery/t/027_stream_regress.pl     |   5 +
 src/tools/pgindent/typedefs.list              |   1 +
 9 files changed, 432 insertions(+), 106 deletions(-)

diff --git a/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql b/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
index f1c20960b7e..14e750beeff 100644
--- a/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
+++ b/contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql
@@ -15,7 +15,13 @@ CREATE VIEW pg_buffercache_partitions AS
 	(partition integer,			-- partition index
 	 num_buffers integer,		-- number of buffers in the partition
 	 first_buffer integer,		-- first buffer of partition
-	 last_buffer integer);		-- last buffer of partition
+	 last_buffer integer,		-- last buffer of partition
+
+	 -- clocksweep counters
+	 num_passes bigint,			-- clocksweep passes
+	 next_buffer integer,		-- next victim buffer for clocksweep
+	 total_allocs bigint,		-- handled allocs (running total)
+	 num_allocs bigint);		-- handled allocs (current cycle)
 
index dfec1380736..96eee21932f 100644
 
 #define NUM_BUFFERCACHE_NUMA_ELEM	3
-#define NUM_BUFFERCACHE_PARTITIONS_ELEM	4
+#define NUM_BUFFERCACHE_PARTITIONS_ELEM	8
 
 PG_MODULE_MAGIC_EXT(
 					.name = "pg_buffercache",
@@ -810,12 +810,20 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
 		TupleDescInitEntry(tupledesc, (AttrNumber) 1, "partition",
 						   INT4OID, -1, 0);
-		TupleDescInitEntry(tupledesc, (AttrNumber) 1, "num_buffers",
+		TupleDescInitEntry(tupledesc, (AttrNumber) 2, "num_buffers",
 						   INT4OID, -1, 0);
-		TupleDescInitEntry(tupledesc, (AttrNumber) 2, "first_buffer",
+		TupleDescInitEntry(tupledesc, (AttrNumber) 3, "first_buffer",
 						   INT4OID, -1, 0);
-		TupleDescInitEntry(tupledesc, (AttrNumber) 3, "last_buffer",
+		TupleDescInitEntry(tupledesc, (AttrNumber) 4, "last_buffer",
 						   INT4OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 5, "num_passes",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 6, "next_buffer",
+						   INT4OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 7, "total_allocs",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 8, "num_allocs",
+						   INT8OID, -1, 0);

@@ -836,12 +844,22 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 					first_buffer,
 					last_buffer;

 		BufferPartitionGet(i, &num_buffers,
 						   &first_buffer, &last_buffer);

@@ -854,6 +872,18 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		values[3] = Int32GetDatum(last_buffer);
 		nulls[3] = false;
 
+		values[4] = Int64GetDatum(complete_passes);
+		nulls[4] = false;
+
+		values[5] = Int32GetDatum(next_victim_buffer);
+		nulls[5] = false;
+
+		values[6] = Int64GetDatum(buffer_total_allocs);
+		nulls[6] = false;
+
+		values[7] = Int64GetDatum(buffer_allocs);
+		nulls[7] = false;
+
 		/* Build and return the tuple. */
 		tuple = heap_form_tuple((TupleDesc) funcctx->user_fctx, values, nulls);
 		result = HeapTupleGetDatum(tuple);
diff --git a/src/backend/storage/buffer/buf_init.c b/src/backend/storage/buffer/buf_init.c
index 007f7806dc4..0362fda24aa 100644
--- a/src/backend/storage/buffer/buf_init.c
+++ b/src/backend/storage/buffer/buf_init.c
@@ -320,3 +320,10 @@ BufferPartitionGet(int idx, int *num_buffers,
 	elog(ERROR, "invalid partition index");
 }
 
+/* return parameters before the partitions are initialized (during sizing) */
+void
+BufferPartitionParams(int *num_partitions)
+{
+	if (num_partitions)
+		*num_partitions = NUM_CLOCK_SWEEP_PARTITIONS;
+}
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index e8544acb784..15ca6f60d57 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3609,33 +3609,29 @@ BufferSync(int flags)
 }
 
 /*
 
 	/* Moving averages of allocation rate and clean-buffer density */
 	static float smoothed_alloc = 0;
@@ -3663,25 +3659,16 @@ BgBufferSync(WritebackContext *wb_context)
 	long		new_strategy_delta;
 	uint32		new_recent_alloc;

 	/*
 	 * Compute strategy_delta = how many buffers have been scanned by the
@@ -3693,17 +3680,17 @@ BgBufferSync(WritebackContext *wb_context)
 	 */
 	if (saved_info_valid)
 	{
-		int32		passes_delta = strategy_passes - prev_strategy_passes;
+		int32		passes_delta = strategy_passes - prev_strategy_passes[partition];

 
@@ -3711,11 +3698,11 @@ BgBufferSync(WritebackContext *wb_context)
@@ -3735,9 +3722,9 @@ BgBufferSync(WritebackContext *wb_context)
@@ -3751,15 +3738,16 @@ BgBufferSync(WritebackContext *wb_context)

 	/*
@@ -3767,9 +3755,9 @@ BgBufferSync(WritebackContext *wb_context)
@@ -3779,7 +3767,7 @@ BgBufferSync(WritebackContext *wb_context)
 
 	/*
@@ -3787,10 +3775,10 @@ BgBufferSync(WritebackContext *wb_context)
 
@@ -3817,7 +3805,7 @@ BgBufferSync(WritebackContext *wb_context)
 
@@ -3842,20 +3830,20 @@ BgBufferSync(WritebackContext *wb_context)

@@ -3869,7 +3857,7 @@ BgBufferSync(WritebackContext *wb_context)
 
 #ifdef BGW_DEBUG
@@ -3899,8 +3887,74 @@ BgBufferSync(WritebackContext *wb_context)
 #endif
 	}

 /*
diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index 7fe34d3ef4c..1d2f97b4d1a 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -15,27 +15,47 @@
  */
 #include "postgres.h"
 
+#ifdef USE_LIBNUMA
+#include <sched.h>
+#endif
+
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
 #include "pgstat.h"

 /*
 
 	/*
 
@@ -46,11 +66,32 @@ typedef struct
 	uint32		completePasses; /* Complete cycles of the clock-sweep */
 	pg_atomic_uint32 numBufferAllocs;	/* Buffers allocated since last reset */

 /* Pointers to shared state */
@@ -89,6 +130,7 @@ static BufferDesc *GetBufferFromRing(BufferAccessStrategy strategy,
 									 uint32 *buf_state);
 static void AddBufferToRing(BufferAccessStrategy strategy,
 							BufferDesc *buf);
+static ClockSweep *ChooseClockSweep(void);
 
 /*
  * ClockSweepTick - Helper routine for StrategyGetBuffer()
@@ -100,6 +142,7 @@ static inline uint32
 ClockSweepTick(void)
 {
 	uint32		victim;
+	ClockSweep *sweep = ChooseClockSweep();
 
 	/*
 	 * Atomically move hand ahead one buffer - if there's several processes
@@ -107,14 +150,14 @@ ClockSweepTick(void)
 	 * apparent order.
 	 */
 	victim =
-		pg_atomic_fetch_add_u32(&StrategyControl->nextVictimBuffer, 1);
+		pg_atomic_fetch_add_u32(&sweep->nextVictimBuffer, 1);

 
 		/*
@@ -140,19 +183,61 @@ ClockSweepTick(void)

+ * use PID to determine the buffer partition
+ * XXX We could use NUMA node / core ID to pick partition, but we'd need
+ * to handle cases with fewer nodes/cores than partitions somehow. Although,
+ * maybe the balancing would handle that too.
+	return (MyProcPid % StrategyControl->num_partitions);
 
 /*
@@ -224,9 +309,35 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 
@@ -306,6 +417,46 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 	}
 }
 
+/*
@@ -313,37 +464,44 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r

 	if (complete_passes)
 	{
 
 		/*
 
-	if (num_buf_alloc)
-	{
 
 /*
@@ -380,6 +538,9 @@ Size
 StrategyShmemSize(void)
 {
 	Size		size = 0;
+	int			num_partitions;
+
+	BufferPartitionParams(&num_partitions);
 
@@ -387,6 +548,10 @@ StrategyShmemSize(void)

@@ -402,6 +567,10 @@ StrategyInitialize(bool init)
 {
 	bool		found;
 
+	int			num_partitions;
+
@@ -419,7 +588,8 @@ StrategyInitialize(bool init)
 
 	if (!found)
@@ -431,15 +601,40 @@ StrategyInitialize(bool init)

+			int			num_buffers,
+			BufferPartitionGet(i, &num_buffers, &first_buffer, &last_buffer);
 
-		/* Clear statistics */
 
@@ -802,3 +997,23 @@ StrategyRejectBuffer(BufferAccessStrategy strategy, BufferDesc *buf, bool from_r
 
 	return true;
 }
+
+void
index 80b0c14831c..1386856046c 100644

 extern Size StrategyShmemSize(void);
@@ -489,5 +491,6 @@ extern int	BufferPartitionCount(void);
 extern int	BufferPartitionNodes(void);
 extern void BufferPartitionGet(int idx, int *num_buffers,
 							   int *first_buffer, int *last_buffer);
+extern void BufferPartitionParams(int *num_partitions);
 
 #endif							/* BUFMGR_INTERNALS_H */
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index 24860c6c2c4..7052f9de57c 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -359,6 +359,11 @@ extern int	GetAccessStrategyBufferCount(BufferAccessStrategy strategy);
 extern int	GetAccessStrategyPinLimit(BufferAccessStrategy strategy);

 /* inline functions */
 
index e24703d6688..f7730ece976 100644

From 03ed6e5116bd5769032823c59e5169b8219ad26b Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 17 Sep 2025 23:04:29 +0200
Subject: [PATCH v20251101 1/7] Infrastructure for partitioning shared buffers

The patch introduces a simple "registry" of buffer partitions, keeping
track of the first/last buffer, etc. This serves as a source of truth
for later patches (e.g. to partition clock-sweep).

The registry is a small BufferPartitions array in shared memory, with
partitions sized to be a fair share of shared buffers. Later patches may
improve this to consider NUMA, and similar details.

Notes:

Note: This partitioning is independent of the partitions defined in
lwlock.h, which defines 128 partitions to reduce lock conflict on the
buffer mapping hashtable. The number of partitions introduced by this
patch is expected to be much lower (a dozen or so).
---
 contrib/pg_buffercache/Makefile               |   2 +-
 contrib/pg_buffercache/meson.build            |   1 +
 .../pg_buffercache--1.6--1.7.sql              |  25 +++
 contrib/pg_buffercache/pg_buffercache.control |   2 +-
 contrib/pg_buffercache/pg_buffercache_pages.c |  86 +++++++++++
 src/backend/storage/buffer/buf_init.c         | 145 +++++++++++++++++-
 src/include/storage/buf_internals.h           |   6 +
 src/include/storage/bufmgr.h                  |  19 +++
 src/tools/pgindent/typedefs.list              |   2 +
 9 files changed, 285 insertions(+), 3 deletions(-)
 create mode 100644 contrib/pg_buffercache/pg_buffercache--1.6--1.7.sql

index 00000000000..f1c20960b7e
@@ -0,0 +1,25 @@
index 3df04c98959..dfec1380736 100644
 
 #define NUM_BUFFERCACHE_NUMA_ELEM	3
+#define NUM_BUFFERCACHE_PARTITIONS_ELEM	4
 
 PG_MODULE_MAGIC_EXT(

 /* Only need to touch memory once per backend process lifetime */
@@ -777,3 +779,87 @@ pg_buffercache_evict_all(PG_FUNCTION_ARGS)
 
 	PG_RETURN_DATUM(result);
 }
+
+/*
+		TupleDescInitEntry(tupledesc, (AttrNumber) 1, "num_buffers",
+		TupleDescInitEntry(tupledesc, (AttrNumber) 2, "first_buffer",
+		TupleDescInitEntry(tupledesc, (AttrNumber) 3, "last_buffer",
+		int			num_buffers,
+		BufferPartitionGet(i, &num_buffers,
+		values[1] = Int32GetDatum(num_buffers);
+		values[2] = Int32GetDatum(first_buffer);
+		values[3] = Int32GetDatum(last_buffer);
index 6fd3a6bbac5..007f7806dc4 100644
@@ -17,6 +17,11 @@
 
 BufferDescPadded *BufferDescriptors;
 char	   *BufferBlocks;
@@ -24,6 +29,14 @@ ConditionVariableMinimallyPadded *BufferIOCVArray;
 WritebackContext BackendWritebackContext;
 CkptSortItem *CkptBufferIds;
 
+/* *
+ * number of buffer partitions */
+#define NUM_CLOCK_SWEEP_PARTITIONS	4
+
+/* Array of structs with information about buffer ranges */
+BufferPartitions *BufferPartitionsArray = NULL;
+
+static void buffer_partitions_init(void);
 
 /*
  * Data Structures:
@@ -70,7 +83,15 @@ BufferManagerShmemInit(void)
 	bool		foundBufs,
 				foundDescs,
 				foundIOCV,
-				foundBufCkpt;
+				foundBufCkpt,
+				foundParts;
+
+	/* allocate the partition registry first */
+	BufferPartitionsArray = (BufferPartitions *)
+		ShmemInitStruct("Buffer Partitions",
+						offsetof(BufferPartitions, partitions) +
+						mul_size(sizeof(BufferPartition), NUM_CLOCK_SWEEP_PARTITIONS),
+						&foundParts);
 
 	/* Align descriptors to a cacheline boundary. */
 	BufferDescriptors = (BufferDescPadded *)
@@ -112,6 +133,9 @@ BufferManagerShmemInit(void)
 	{
 		int			i;
 
+		/* Initialize buffer partitions (calculate buffer ranges). */
+		buffer_partitions_init();
+
 		/*
 		 * Initialize all the buffer headers.
 		 */
@@ -175,5 +199,124 @@ BufferManagerShmemSize(void)
 	/* size of checkpoint sort array in bufmgr.c */
 	size = add_size(size, mul_size(NBuffers, sizeof(CkptSortItem)));
 
+								   mul_size(sizeof(BufferPartition), NUM_CLOCK_SWEEP_PARTITIONS)));
+	/* number of buffers per partition (make sure to not overflow) */
+	int			part_buffers
+		= ((int64) NBuffers + (NUM_CLOCK_SWEEP_PARTITIONS - 1)) / NUM_CLOCK_SWEEP_PARTITIONS;
+	BufferPartitionsArray->npartitions = NUM_CLOCK_SWEEP_PARTITIONS;
+	for (int n = 0; n < BufferPartitionsArray->npartitions; n++)
+		BufferPartition *part = &BufferPartitionsArray->partitions[n];
+		/* buffers this partition should get (last partition can get fewer) */
+		int			num_buffers = Min(remaining_buffers, part_buffers);
+		remaining_buffers -= num_buffers;
+BufferPartitionGet(int idx, int *num_buffers,
index c1206a46aba..80b0c14831c 100644
 
 /* in buf_init.c */

+extern int	BufferPartitionCount(void);
+extern int	BufferPartitionNodes(void);
+extern void BufferPartitionGet(int idx, int *num_buffers,
+							   int *first_buffer, int *last_buffer);
+
 #endif							/* BUFMGR_INTERNALS_H */
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index b5f8f3c5d42..24860c6c2c4 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -153,6 +153,25 @@ struct ReadBuffersOperation

+/*
 
index ac2da4c98cf..e24703d6688 100644

---

From: Jakub Wartak <[email-masked]>
Date: Tue, 4 Nov 2025 13:10:58 +0100

On Fri, Oct 31, 2025 at 12:57 PM Tomas Vondra <[email-masked]> wrote:
> I had a couple discussions about these patches at pgconf.eu last week,[..]

I've just had a quick look at this and oh, my, I've started getting
into this partitioned clocksweep and that's ambitious! Yes, this
sequencing of patches makes it much more understandable. Anyway I've
spotted some things, attempted to fix some and have some basic
questions too (so small baby steps, all of this was on 4s/4 NUMA nodes
with HP on) -- the 000X refers to question/issue/bug in specific
patchset file:

0001: you mention 'debug_numa = buffers' in commitmsg, but there's
nothing there like that? it comes with 0006

0002: dunno, but wouldn't it make some educational/debugging sense to
add a debug function returning clocksweep partition index
(calculate_partition_index) for backend? (so that we know which
partition we are working on right now)

0003: those two "elog(INFO, "rebalance skipped:" should be at DEBUG2+
IMHO (they are way too verbose during runs)

0006a: Needs update - s/patches later in the patch series/patches
earlier in the patch series/

0006b: IMHO longer term, we should hide some complexity of those calls
via src/port numa shims (pg_numa_sched_cpu()?)

0006c: after GUC commit fce7c73fba4e5, apply complains with:
error: patch failed: src/backend/utils/misc/guc_parameters.dat:906
error: src/backend/utils/misc/guc_parameters.dat: patch does not apply

0007a: pg_buffercache_pgproc returns pgproc_ptr and fastpath_ptr in
bigint and not hex? I've wanted to adjust that to TEXTOID, but instead
I've thought it is going to be simpler to use to_hex() -- see 0009
attached.

0007b: pg_buffercache_pgproc -- nitpick, but maybe it would be better
called pg_shm_pgproc?

0007c with check_numa='buffers,procs' throws 'mbind Invalid argument'
during start:

    2025-11-04 10:02:27.055 CET [58464] DEBUG:  NUMA:
pgproc_init_partition procs 0x7f8d30400000 endptr 0x7f8d30800000
num_procs 2523 node 0
    2025-11-04 10:02:27.057 CET [58464] DEBUG:  NUMA:
pgproc_init_partition procs 0x7f8d30800000 endptr 0x7f8d30c00000
num_procs 2523 node 1
    2025-11-04 10:02:27.059 CET [58464] DEBUG:  NUMA:
pgproc_init_partition procs 0x7f8d30c00000 endptr 0x7f8d31000000
num_procs 2523 node 2
    2025-11-04 10:02:27.061 CET [58464] DEBUG:  NUMA:
pgproc_init_partition procs 0x7f8d31000000 endptr 0x7f8d31400000
num_procs 2523 node 3
    2025-11-04 10:02:27.062 CET [58464] DEBUG:  NUMA:
pgproc_init_partition procs 0x7f8d31400000 endptr 0x7f8d31407cb0
num_procs 38 node -1
    mbind: Invalid argument
    mbind: Invalid argument
    mbind: Invalid argument
    mbind: Invalid argument

0007d: so we probably need numa_warn()/numa_error() wrappers (this was
initially part of NUMA observability patches but got removed during
the course of action), I'm attaching 0008. With that you'll get
something a little more up to our standards:
    2025-11-04 10:27:07.140 CET [59696] DEBUG:
fastpath_parititon_init node = 3, ptr = 0x7f4f4d400000, endptr =
0x7f4f4d4b1660
    2025-11-04 10:27:07.140 CET [59696] WARNING:  libnuma: ERROR: mbind

0007e: elog DEBUG says it's pg_proc_init_partition but it's
pgproc_partition_init() actually ;)

0007f: The "mbind: Invalid argument"" issue itself with the below  addition:
    +elog(DEBUG1, "NUMA: fastpath_partition_init ptr %p endptr %p
num_procs %d node %d", ptr, endptr, num_procs, node);
    showed this:
    2025-11-04 11:30:51.089 CET [61841] DEBUG:  NUMA:
fastpath_partition_init ptr 0x7f39eea00000 endptr 0x7f39eeab1660
num_procs 2523 node 0
    2025-11-04 11:30:51.089 CET [61841] WARNING:  libnuma: ERROR: mbind
    2025-11-04 11:30:51.089 CET [61841] DEBUG:  NUMA:
fastpath_partition_init ptr 0x7f39eec00000 endptr 0x7f39eecb1660
num_procs 2523 node 1
    2025-11-04 11:30:51.089 CET [61841] WARNING:  libnuma: ERROR: mbind
    2025-11-04 11:30:51.089 CET [61841] DEBUG:  NUMA:
fastpath_partition_init ptr 0x7f39eee00000 endptr 0x7f39eeeb1660
num_procs 2523 node 2
    2025-11-04 11:30:51.089 CET [61841] WARNING:  libnuma: ERROR: mbind
    [..]

    Meanwhile it's full hugepage size (e.g. 0x7f39eec00000−0x7f39eea00000 = 2MB)
    $ grep --color 7f39ee[ace] /proc/61841/smaps
    7f39ee800000-7f39eea00000 rw-s 87de00000 00:11 122710
      /anon_hugepage (deleted)
    7f39eea00000-7f39eec00000 rw-s 87e000000 00:11 122710
      /anon_hugepage (deleted)
    7f39eec00000-7f39eee00000 rw-s 87e200000 00:11 122710
      /anon_hugepage (deleted)
    7f39eee00000-7f39ef000000 rw-s 87e400000 00:11 122710
      /anon_hugepage (deleted)

    but mbind() was called for just 0x7f39eeab1660−0x7f39eea00000 =
0xB1660 = 726624 bytes, but if adjust blindly endptr in that
fastpath_partition_init() to be "char *endptr = ptr + 2*1024*1024;"
(HP) it doesn't complain anymore and I get success:
    2025-11-04 12:08:30.147 CET [62352] DEBUG:  NUMA:
fastpath_partition_init ptr 0x7f7bf7000000 endptr 0x7f7bf7200000
num_procs 2523 node 0
    2025-11-04 12:08:30.147 CET [62352] DEBUG:  NUMA:
fastpath_partition_init ptr 0x7f7bf7200000 endptr 0x7f7bf7400000
num_procs 2523 node 1
    2025-11-04 12:08:30.147 CET [62352] DEBUG:  NUMA:
fastpath_partition_init ptr 0x7f7bf7400000 endptr 0x7f7bf7600000
num_procs 2523 node 2
    2025-11-04 12:08:30.147 CET [62352] DEBUG:  NUMA:
fastpath_partition_init ptr 0x7f7bf7600000 endptr 0x7f7bf7800000
num_procs 2523 node 3
    2025-11-04 12:08:30.147 CET [62352] DEBUG:  NUMA:
fastpath_partition_init ptr 0x7f7bf7800000 endptr 0x7f7bf7a00000
num_procs 38 node -1
    2025-11-04 12:08:30.239 CET [62352] LOG:  starting PostgreSQL
19devel on x86_64-linux, compiled by gcc-12.2.0, 64-bit

0006d: I've got one SIGBUS during a call to select
pg_buffercache_numa_pages(); and it looks like that memory accessed is
simply not mapped? (bug)

    Program received signal SIGBUS, Bus error.
    pg_buffercache_numa_pages (fcinfo=0x561a97e8e680) at
../contrib/pg_buffercache/pg_buffercache_pages.c:386
    386                                     pg_numa_touch_mem_if_required(ptr);
    (gdb) print ptr
    $1 = 0x7f4ed0200000 <error: Cannot access memory at address 0x7f4ed0200000>
    (gdb) where
    #0  pg_buffercache_numa_pages (fcinfo=0x561a97e8e680) at
../contrib/pg_buffercache/pg_buffercache_pages.c:386
    #1  0x0000561a672a0efe in ExecMakeFunctionResultSet
(fcache=0x561a97e8e5d0, econtext=econtext@entry=0x561a97e8dab8,
argContext=0x561a97ec62a0, isNull=0x561a97e8e578,
isDone=isDone@entry=0x561a97e8e5c0) at
../src/backend/executor/execSRF.c:624
    [..]

    Postmaster had still attached shm (visible via smaps), and if you
compare closely 0x7f4ed0200000 against sorted smaps:

    7f4921400000-7f4b21400000 rw-s 252600000 00:11 151111
      /anon_hugepage (deleted)
    7f4b21400000-7f4d21400000 rw-s 452600000 00:11 151111
      /anon_hugepage (deleted)
    7f4d21400000-7f4f21400000 rw-s 652600000 00:11 151111
      /anon_hugepage (deleted)
    7f4f21400000-7f4f4bc00000 rw-s 852600000 00:11 151111
      /anon_hugepage (deleted)
    7f4f4bc00000-7f4f4c000000 rw-s 87ce00000 00:11 151111
      /anon_hugepage (deleted)

    it's NOT there at all (there's no mmap region starting with
0x"7f4e" ). It looks like because pg_buffercache_numa_pages() is not
aware of this new mmaped() regions and instead does simple loop over
all NBuffers with "for (char *ptr = startptr; ptr < endptr; ptr +=
os_page_size)"?

0006e:
    I'm seeking confirmation, but is this the issue we have discussed
on PgconfEU related to lack of detection of Mems_allowed, right? e.g.
    $ numactl --membind="0,1" --cpunodebind="0,1"
/usr/pgsql19/bin/pg_ctl -D /path start
    still shows 4 NUMA nodes used. Current patches use
numa_num_configured_nodes(), but it says 'This count includes any
nodes that are currently DISABLED'. So I was wondering if I could help
by migrating towards numa_num_task_nodes() / numa_get_mems_allowed()?
It's the same as You wrote earlier to Alexy?

    > Instead of just using all *configured* nodes. And I agree with that.

    So are you already on it ?

> ran into a bunch of weird issues. I saw two types of failures:

I did run (with io_uring) a short test(< 10min with -c 128) and didn't
get those. Could you please share specific tips/workload for
reproducing this?

That's all for today, I hope it helps a little.

-J.

---

From: Tomas Vondra <[email-masked]>
Date: Tue, 4 Nov 2025 22:21:24 +0100

> nothing there like that? it comes with 0006

Right, I forgot to remove that reference.

> partition we are working on right now)

Perhaps. I didn't need that, but it might be interesting during
development. I probably would not keep that in the final version.

Agreed.

Agreed.

> via src/port numa shims (pg_numa_sched_cpu()?)

Yeah, there's definitely room for moving more of the code to src/port.

> error: src/backend/utils/misc/guc_parameters.dat: patch does not apply

Will fix.

> I've thought it is going to be simpler to use to_hex() -- see 0009

I don't know. I added simply because it might be useful for development,
but we probably don't want to expose these pointers at all.

> 0007b: pg_buffercache_pgproc -- nitpick, but maybe it would be better

Right. It does not belong to pg_buffercache at all, I just added it
there because I've been messing with that code already.

> pgproc_init_partition procs 0x7f8d31400000 endptr 0x7f8d31407cb0

I'll take a look, but I don't recall seeing such errors.

>     2025-11-04 10:27:07.140 CET [59696] WARNING:  libnuma: ERROR: mbind

Not sure.

> 19devel on x86_64-linux, compiled by gcc-12.2.0, 64-bit

Hmm, so it seems like another hugepage-related issue. The mbind manpage
says this about "len":

  EINVAL An invalid value was specified for flags or mode; or addr + len
  was less than addr; or addr is not a multiple of the system page size.

I don't think that requires (addr+len) to be a multiple of page size,
but maybe that is required.

> all NBuffers with "for (char *ptr = startptr; ptr < endptr; ptr +=

I'm confused. How could that mapping be missing? Was this with huge
pages / how many did you reserve on the nodes? Maybe there were not
enough huge pages left on one of the nodes?

I believe I got some SIGBUS in those cases.

> It's the same as You wrote earlier to Alexy?

If "mems_allowed" refers to nodes allowing memory allocation, then yes,
this would be one way to get into that issue. Oh, is this what happened
in 0006d?

> get those. Could you please share specific tips/workload for

I did get a couple of "operation canceled" failures, but only on fairly
old kernel versions (6.1 which came as default with the VM). I heard
some suggestions this is a bug in older kernels - I don't have any link
to a bug report / fix, though. But I've been unable to reproduce this on
6.17, so maybe it's true.

For me the failures always happened 10 seconds after the start of the
benchmark (and starting the instance), so it's probably sufficient to
keep the runs ~20 seconds (and maybe restart in between?).

But even then it's fairly rare. I've seen ~10 failures for 500 runs.

I haven't seen more "bad address" cases, I have no idea why. I'm still
guessing it's related to huge pages, so maybe I happened to reserve
enough of them.

regards

-- 
Tomas Vondra

---

From: Jakub Wartak <[email-masked]>
Date: Thu, 6 Nov 2025 15:02:57 +0100

On Tue, Nov 4, 2025 at 10:21 PM Tomas Vondra <[email-masked]> wrote:

Hi Tomas,

> there because I've been messing with that code already.

Please keep them in for at least for some time (perhaps standalone
patch marked as not intended to be commited would work?). I find the
view extermely useful as it will allow us pinpointing local-vs-remote
NUMA fetches (we need to know the addres).

> I'll take a look, but I don't recall seeing such errors.

Alexy also reported this earlier, here
https://www.postgresql.org/message-id/92e23c85-f646-4bab-b5e0-df30d8ddf4bd%40postgrespro.ru
(just use HP, set some high max_connections). I've double checked this
too , numa_tonode_memory() len needs to HP size.

> >     2025-11-04 10:27:07.140 CET [59696] WARNING:  libnuma: ERROR: mbind

Any particular objections? We need to somehow emit them into the logs.

[..]
[..]
> but maybe that is required.

I do think that 'system page size' means above HP page size, but this
time it's just for fastpath_partition_init(), the earlier one seems to
aligned fine (?? -- i havent really checked but there's no error)

> pages / how many did you reserve on the nodes?

OK I made and error and paritally got it correct (it crashes reliably)
and partially mislead You, appologies, let me explain. There were two
questions for me:
a) why we make single mmap() and after numa_tonode_memory() we get
plenty of mappings
b) why we get SIGBUS (I've thought they are not continus, but they are
after triple-checking)

ad a) My testing shows that on HP,as stated initially ("all of this
was on 4s/4 NUMA nodes with HP on"). That's what the codes does, you
get single mmaps() (resulting in single entry in smaps), but afte
noda_tonode_memory() there's many of them. Even on laptop:

System has 1 NUMA nodes (0 to 0).
Attempting to allocate 8.000000 MB of HugeTLB memory...
Successfully allocated HugeTLB memory at 0x755828800000, smaps before:
755828800000-755829000000 rw-s 00000000 00:11 259808
  /anon_hugepage (deleted)
Pinning first part (from 0x755828800000) to NUMA node 0...
smaps after:
755828800000-755828c00000 rw-s 00000000 00:11 259808
  /anon_hugepage (deleted)
755828c00000-755829000000 rw-s 00400000 00:11 259808
  /anon_hugepage (deleted)
Pinning second part (from 0x755828c00000) to NUMA node 0...
smaps after:
755828800000-755828c00000 rw-s 00000000 00:11 259808
  /anon_hugepage (deleted)
755828c00000-755829000000 rw-s 00400000 00:11 259808
  /anon_hugepage (deleted)

It gets even more funny, below I have 8MB HP=on, but just issue 2x
numa_tonode_memory(for len 2MB on 4MB ptr to node0) (two times for
ptr, second time in half of that):

System has 1 NUMA nodes (0 to 0).
Attempting to allocate 8.000000 MB of HugeTLB memory...
Successfully allocated HugeTLB memory at 0x7302dda00000, smaps before:
7302dda00000-7302de200000 rw-s 00000000 00:11 284859
  /anon_hugepage (deleted)
Pinning first part (from 0x7302dda00000) to NUMA node 0...
smaps after:
7302dda00000-7302ddc00000 rw-s 00000000 00:11 284859
  /anon_hugepage (deleted)
7302ddc00000-7302de200000 rw-s 00200000 00:11 284859
  /anon_hugepage (deleted)
Pinning second part (from 0x7302dde00000) to NUMA node 0...
smaps after:
7302dda00000-7302ddc00000 rw-s 00000000 00:11 284859
  /anon_hugepage (deleted)
7302ddc00000-7302dde00000 rw-s 00200000 00:11 284859
  /anon_hugepage (deleted)
7302dde00000-7302de000000 rw-s 00400000 00:11 284859
  /anon_hugepage (deleted)
7302de000000-7302de200000 rw-s 00600000 00:11 284859
  /anon_hugepage (deleted)

Why 4 instead of 1? Because some mappings are now "default" becauswe
their policy was not altered:

$ grep huge /proc/$(pidof testnumammapsplit)/numa_maps
7302dda00000 bind:0 file=/anon_hugepage\040(deleted) huge
7302ddc00000 default file=/anon_hugepage\040(deleted) huge
7302dde00000 bind:0 file=/anon_hugepage\040(deleted) huge
7302de000000 default file=/anon_hugepage\040(deleted) huge

Back to originnal error, they are consecutive regions and earlier problem is

error: 0x7f4ed0200000 <error: Cannot access memory at address 0x7f4ed0200000>
start: 0x7f4921400000
end:   0x7f4f4c000000

so it fits into that range (that was my mistate earlier, using just
grep not checking are they really within that), but...

> Maybe there were not enough huge pages left on one of the nodes?

ad b) right, something like that. I've investigated that SIGBUS there
(it's going to be long):

with shared_buffers=32GB, huge_pages 17715 (+1 from what postgres -C
shared_memory_size_in_huge_pages returns), right after startup, but no
touch:

Program received signal SIGBUS, Bus error.
pg_buffercache_numa_pages (fcinfo=0x5572038790b8) at
../contrib/pg_buffercache/pg_buffercache_pages.c:386
386                                     pg_numa_touch_mem_if_required(ptr);
(gdb) where
#0  pg_buffercache_numa_pages (fcinfo=0x5572038790b8) at
../contrib/pg_buffercache/pg_buffercache_pages.c:386
#1  0x00005571f54ddb7d in ExecMakeTableFunctionResult
(setexpr=0x557203870d40, econtext=0x557203870ba8,
argContext=<optimized out>, expectedDesc=0x557203870f80,
randomAccess=false) at ../src/backend/executor/execSRF.c:234
[..]
(gdb) print ptr
$1 = 0x7f6cf8400000 <error: Cannot access memory at address 0x7f6cf8400000>
(gdb)

then it shows?! no available hugepage on one of the nodes (while gdb
is hanging and preving autorestart):

root@swiatowid:/sys/devices/system/node# grep -r -i HugePages_Free node*/meminfo
node0/meminfo:Node 0 HugePages_Free:    299
node1/meminfo:Node 1 HugePages_Free:    299
node2/meminfo:Node 2 HugePages_Free:    299
node3/meminfo:Node 3 HugePages_Free:      0

but they are equal in terms of size:
node0/meminfo:Node 0 HugePages_Total:  4429
node1/meminfo:Node 1 HugePages_Total:  4429
node2/meminfo:Node 2 HugePages_Total:  4429
node3/meminfo:Node 3 HugePages_Total:  4428

smaps shows that this address (7f6cf8400000) is mapped in this mapping:
7f6b49c00000-7f6d49c00000 rw-s 652600000 00:11 86064
  /anon_hugepage (deleted)

numa_maps for this region shows this is this mapping on node3 (notice
N3 + bind:3 matches lack of memory on Node 3 HugePAges_Free):
7f6b49c00000 bind:3 file=/anon_hugepage\040(deleted) huge dirty=3444
N3=3444 kernelpagesize_kB=2048

the surrounding area of this looks like that:

7f6549c00000 bind:0 file=/anon_hugepage\040(deleted) huge dirty=4096
N0=4096 kernelpagesize_kB=2048
7f6749c00000 bind:1 file=/anon_hugepage\040(deleted) huge dirty=4096
N1=4096 kernelpagesize_kB=2048
7f6949c00000 bind:2 file=/anon_hugepage\040(deleted) huge dirty=4096
N2=4096 kernelpagesize_kB=2048
7f6b49c00000 bind:3 file=/anon_hugepage\040(deleted) huge dirty=3444
N3=3444 kernelpagesize_kB=2048 <-- this is the one
7f6d49c00000 default file=/anon_hugepage\040(deleted) huge dirty=107
mapmax=6 N3=107 kernelpagesize_kB=2048

Notice it's just N3=3444, while the others are much larger. So
something was using that hugepages memory on N3:

# grep kernelpagesize_kB=2048 /proc/1679/numa_maps | grep -Po
N[0-4]=[0-9]+ | sort
N0=2
N0=4096
N1=2
N1=4096
N2=2
N2=4096
N3=1
N3=1
N3=1
N3=1
N3=107
N3=13
N3=3
N3=3444

So per above it's not there (at least not as 2MB HP). But the number
of mappings is wild there! (node where it is failing has plenty of
memory, no hugepage memory left, but it has like 40k+ of small
mappings!)

# grep -Po 'N[0-3]=' /proc/1679/numa_maps | sort | uniq -c
     17 N0=
     10 N1=
      3 N2=
  40434 N3=

most of them are `anon_inode:[io_uring]` (and I had
max_connections=10k). You may ask why in spite of Andres optimization
for reducing number segments for uring, it's not working for me ? Well
I've just noticed way too silent failure to active this (altough I'm
on 6.14.x):
    2025-11-06 13:34:49.128 CET [1658] DEBUG:  can't use combined
memory mapping for io_uring, kernel or liburing too old
and I dont have io_uring_queue_init_mem()/HAVE_LIBURING_QUEUE_INIT_MEM
apparently on liburing-2.3 (Debian's default). See [1] for more info
(fix is not commited yet sadly).

Next try, now with io_method = worker and right before start:

root@swiatowid:/sys/devices/system/node# grep -r -i HugePages_Total
node*/meminfo
node0/meminfo:Node 0 HugePages_Total:  4429
node1/meminfo:Node 1 HugePages_Total:  4429
node2/meminfo:Node 2 HugePages_Total:  4429
node3/meminfo:Node 3 HugePages_Total:  4428
and HugePages_Free were 100% (if postgresql was down). After start
(but without doing anything else):
root@swiatowid:/sys/devices/system/node# grep -r -i HugePages_Free node*/meminfo
node0/meminfo:Node 0 HugePages_Free:   4393
node1/meminfo:Node 1 HugePages_Free:   4395
node2/meminfo:Node 2 HugePages_Free:   4395
node3/meminfo:Node 3 HugePages_Free:   3446

So sadly the picture is the same (something stole my HP on N3 and it's
PostgreSQL on it's own). After some time of investigating that ("who
stole my hugepage across whole OS"), I've just added MAP_POPULATE to
the mix of PG_MMAP_FLAGS and got this after start:

root@swiatowid:/sys/devices/system/node# grep -r -i HugePages_Free node*/meminfo
node0/meminfo:Node 0 HugePages_Free:      0
node1/meminfo:Node 1 HugePages_Free:      0
node2/meminfo:Node 2 HugePages_Free:      0
node3/meminfo:Node 3 HugePages_Free:      1

and then the SELECT to pg_buffercache_numa works fine(!).

Another ways that I have found to eliminate that SIGBUS
a. Would be to throw much more HugePages (so that node does not run to
HugePages_Free), but that's not real option.
b. Then I've reminded myself that I could be running custom kernel
with experimental CONFIG_READ_ONLY_THP_FOR_FS (to reduce iTLB misses
tranparently with specially linked PG; will double check exact stuff
later), so I've thrown never into
/sys/kernel/mm/transparent_hugepage/enabled and defrag too (yes ,
disabled THP) and with that -- drumroll -- that SELECT works. The very
same PG picture after startup (where earlier it would crash), now
after SELECT it looks like that:

root@swiatowid:/sys/devices/system/node# grep -r -i HugePages_Free node*/meminfo
node0/meminfo:Node 0 HugePages_Free:     83
node1/meminfo:Node 1 HugePages_Free:      0
node2/meminfo:Node 2 HugePages_Free:     81
node3/meminfo:Node 3 HugePages_Free:     82

Hope that helps a little. To me it sounds like THP used that memory
somehow and we've also wanted to use. With numa_interleave_ptr() that
wouldn't be a problem because probably it would something else
available, but not here as we indicated exact node.

> this would be one way to get into that issue. Oh, is this what happened

OK, thanks for confirmation. No, 0006d was about normal numactl run,
without --membind.

> old kernel versions (6.1 which came as default with the VM).

OK, I'll try to see that later too.

btw QQ regarding partitioned clockwise as I had thought: does this
opens a road towards multiple BGwriters? (outside of this
$thread/v1/PoC)

-J.

[1] - https://www.postgresql.org/message-id/CAKZiRmzxj6Lt1w2ffDoUmN533TgyDeYVULEH1PQFLRyBJSFP6w%40mail.gmail.com

---

From: Tomas Vondra <[email-masked]>
Date: Tue, 11 Nov 2025 12:52:01 +0100

Hi,

here's a rebased patch series, fixing most of the smaller issues from
v20251101, and making cfbot happy (hopefully).

> NUMA fetches (we need to know the addres).

Are you referring to the _pgproc view specifically, or also to the view
with buffer partitions? I don't intend to remove the view for shared
buffers, that's indeed useful.

> too , numa_tonode_memory() len needs to HP size.

OK, I'll investigate this.

> Any particular objections? We need to somehow emit them into the logs.

No idea, I think it'd be better to make sure this failure can't happen,
but maybe it's not possible. I don't understand the mbind failure well
enough.

> aligned fine (?? -- i havent really checked but there's no error)

Hmmm, ok. Will check. But maybe let's not focus too much on the PGPROC
partitioning, I don't think that's likely to go into 19.

> OK, thanks for confirmation. No, 0006d was about normal numactl run,

I didn't have time to look into all this info about mappings, io_uring
yet, so no response from me.

> opens a road towards multiple BGwriters? (outside of this

I don't think the clocksweep partitioning is required for multiple
bgwriters, but it might make it easier.

regards

-- 
Tomas Vondra
From 7f8096d294ed9d69c953ff97be7f46db058cd2cb Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 11 Nov 2025 12:10:03 +0100
Subject: [PATCH v20251111 7/7] NUMA: partition PGPROC

 - PGPROC array / node #0
 - PGPROC array / node #1

 src/backend/storage/lmgr/proc.c               | 551 +++++++++++++++++-
 15 files changed, 723 insertions(+), 73 deletions(-)

index 1379d54cc5d..f340fcf420c 100644

 PG_MODULE_MAGIC_EXT(

@@ -931,3 +934,94 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
index ea43b432daf..7d589bac115 100644
@@ -575,7 +575,7 @@ TransactionGroupUpdateXidStatus(TransactionId xid, XidStatus status,
 
 		/*
@@ -634,7 +634,7 @@ TransactionGroupUpdateXidStatus(TransactionId xid, XidStatus status,
 	 */
 	while (wakeidx != INVALID_PROC_NUMBER)
 	{
-		PGPROC	   *wakeproc = &ProcGlobal->allProcs[wakeidx];
+		PGPROC	   *wakeproc = ProcGlobal->allProcs[wakeidx];
 
index 89d0bfa7760..e0e17293536 100644

index c4a888a081c..f5844aa5b6a 100644

 /*
index 587859a5754..bdcdbcc6b5f 100644
 
index 810a549efce..0937292643f 100644
 
 	/*

 
 /*

+	AssertCheckAllProcs();
+

 

+	AssertCheckAllProcs();
+
 	/*
 
+	AssertCheckAllProcs();
+
 	myoff = proc->pgxactoff;

 

+	AssertCheckAllProcs();
+
 	/*

 
 
-
 /*

 			/*

 

 			/*

 

 		if (proc->pid == pid)
 		{
 
 			result = proc->pid;
 			break;
 
 		if (proc == MyProc)

 
 		/*
 
 		if (proc->pid == 0)
 
 		if (proc->pid == 0)

 		if (proc->pid == 0)

 

 
index 1504fafe6d8..0a0ce98b725 100644
@@ -29,22 +29,33 @@
  */
 #include "postgres.h"
 
+#ifdef USE_LIBNUMA
+#include <sched.h>
+#endif
+
 #include <signal.h>
 #include <unistd.h>
 #include <sys/time.h>
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
 #include "access/xlogwait.h"
@@ -77,8 +88,8 @@ NON_EXEC_STATIC slock_t *ProcStructLock = NULL;

 
@@ -91,6 +102,29 @@ static void AuxiliaryProcKill(int code, Datum arg);
 static void CheckDeadLock(void);

@@ -101,11 +135,41 @@ PGProcShmemSize(void)

+	/*
 
@@ -130,6 +194,60 @@ FastPathLockShmemSize(void)

+	/*
 
@@ -141,6 +259,9 @@ ProcGlobalShmemSize(void)
 {
 	Size		size = 0;
 
@@ -149,6 +270,8 @@ ProcGlobalShmemSize(void)

@@ -193,7 +316,7 @@ ProcGlobalSemas(void)
 void
 InitProcGlobal(void)
 {
-	PGPROC	   *procs;
+	PGPROC	  **procs;
 	int			i,
 				j;
 	bool		found;
@@ -212,6 +335,9 @@ InitProcGlobal(void)
 		ShmemInitStruct("Proc Header", sizeof(PROC_HDR), &found);
 	Assert(!found);
 
@@ -226,6 +352,15 @@ InitProcGlobal(void)
 
@@ -241,21 +376,110 @@ InitProcGlobal(void)
 						  requestSize,
 						  &found);

 
+	/*
@@ -291,23 +515,91 @@ InitProcGlobal(void)
 	/* Reserve space for semaphores. */
 	PGReserveSemaphores(ProcGlobalSemas());

 
 		/*

 
 		/*
 		 * Set up per-PGPROC semaphore, latch, and fpInfoLock.  Prepared xact
@@ -371,9 +663,6 @@ InitProcGlobal(void)
 		pg_atomic_init_u64(&(proc->waitStart), 0);
 	}
 
@@ -440,7 +729,51 @@ InitProcess(void)
 
@@ -651,7 +984,7 @@ InitAuxiliaryProcess(void)
@@ -1059,7 +1392,7 @@ AuxiliaryProcKill(int code, Datum arg)

 
@@ -1108,7 +1441,7 @@ AuxiliaryPidGetProc(int pid)

 		if (proc->pid == pid)
 		{
@@ -1998,7 +2331,7 @@ ProcSendSignal(ProcNumber procNumber)
 	if (procNumber < 0 || procNumber >= ProcGlobal->allProcCount)
 		elog(ERROR, "procNumber out of range");

 /*
@@ -2073,3 +2406,173 @@ BecomeLockGroupMember(PGPROC *leader, int pid)
 
 	return ok;
 }
+
+/*
+	elog(DEBUG1, "NUMA: pgproc_partition_init procs %p endptr %p num_procs %d node %d",

 #ifdef USE_LIBNUMA

 

 

 /*
 
 /*
 
 #ifdef EXEC_BACKEND

+extern int	ProcPartitionCount(void);
+extern void ProcPartitionGet(int idx, int *node, int *nprocs, void **procsptr, void **fpptr);
+
 #endif							/* _PROC_H_ */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index cb52c417592..b11b04818be 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1881,6 +1881,7 @@ PGP_MPI
 PGP_PubKey
 PGP_S2K
 PGPing
+PGProcPartition
 PGQueryClass
 PGRUsage
 PGSemaphore
-- 
2.51.1

From 705ce91522059afc98ff339cc640ab7aa38d4ac7 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 11 Nov 2025 12:05:35 +0100
Subject: [PATCH v20251111 6/7] NUMA: shared buffers partitioning

The effect is similar to

     numactl --interleave=all

Notes:

index 8536e2debef..1379d54cc5d 100644
@@ -813,19 +813,21 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
@@ -849,7 +851,8 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 
-		int			num_buffers,
+		int			numa_node,
+					num_buffers,
 					first_buffer,
 					last_buffer;
 
@@ -868,7 +871,7 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		Datum		values[NUM_BUFFERCACHE_PARTITIONS_ELEM];
 		bool		nulls[NUM_BUFFERCACHE_PARTITIONS_ELEM];

 		ClockSweepPartitionGetInfo(i,
@@ -886,36 +889,39 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		values[0] = Int32GetDatum(i);
 		nulls[0] = false;

 

 

 

index 0362fda24aa..587859a5754 100644
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
 
-/* *

 

 

 
 		/*

 

 	return size;
 }
 
+/*
+ * patches earlier in the patch series. Those patches however use the same

 

 
 
-		buffer += num_buffers;

 int
 
+int

index 8be77a9c8b1..810a549efce 100644

 
 /*

 
+	int			num_nodes;
 	int			num_partitions;

 	/*

 
 			/*
 
 		/* initialize the partitioned clocksweep */
 		StrategyControl->num_partitions = num_partitions;
+		StrategyControl->num_nodes = num_nodes;
+		StrategyControl->num_partitions_per_node = num_partitions_per_node;
 	}
 	else
 		Assert(!init);
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 1128167c025..8192c27066b 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -636,6 +636,16 @@
   options => 'debug_logical_replication_streaming_options',
 },
 
+{ name => 'debug_numa', type => 'string', context => 'PGC_POSTMASTER', group => 'DEVELOPER_OPTIONS',
+  short_desc => 'NUMA-aware partitioning of shared memory.',
+  long_desc => 'An empty string disables NUMA-aware partitioning.',
+  flags => 'GUC_LIST_INPUT | GUC_NOT_IN_SAMPLE',
+  variable => 'debug_numa_string',
+  boot_val => '""',
+  check_hook => 'check_debug_numa',
+  assign_hook => 'assign_debug_numa',
+},
+
 { name => 'debug_parallel_query', type => 'enum', context => 'PGC_USERSET', group => 'DEVELOPER_OPTIONS',
   short_desc => 'Forces the planner\'s use parallel query nodes.',
   long_desc => 'This can be useful for testing the parallel query infrastructure by forcing the planner to generate plans that contain nodes that perform tuple communication between workers and the main process.',
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 0209b2067a2..404eb3432f9 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -595,6 +595,7 @@ static char *server_version_string;
 static int	server_version_num;
 static char *debug_io_direct_string;
 static char *restrict_nonsystem_relation_kind_string;
+static char *debug_numa_string;
 
 #ifdef HAVE_SYSLOG
 
 #ifdef USE_LIBNUMA
 
index 1118b386228..33377841c57 100644
@@ -299,10 +299,10 @@ typedef struct BufferDesc
@@ -312,7 +312,7 @@ typedef struct BufferDesc
 
 typedef union BufferDescPadded
 {
@@ -555,8 +555,8 @@ extern void AtEOXact_LocalBuffers(bool isCommit);

 
 
 /* in localbuf.c */

 #include "miscadmin.h"
 
 /*
 
+/*
+ * pg_numa_move_to_node
 
 /* Empty wrappers */
 
+void
2.51.1

From c97635bdd35858b44de03012268f521678cb1edc Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 6 Aug 2025 01:09:57 +0200
Subject: [PATCH v20251111 5/7] clock-sweep: weighted balancing

index 3af82e267c6..8be77a9c8b1 100644
 
+	/*

 	/*
 
 	/*
 
 	/*

 

 

2.51.1

From be684d592380765e857bcc85fb32807c5212e8b2 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 15 Oct 2025 13:59:29 +0200
Subject: [PATCH v20251111 4/7] clock-sweep: scan all partitions

index 169071032b4..3af82e267c6 100644
 
 /*
 
 	/*
 
 	*from_ring = false;

 
 		/*
@@ -501,7 +536,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint32 *buf_state, bool *from_r
 
2.51.1

From 39316faa1c8bd0f20320b4481cb0c05bab14dd46 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 29 Oct 2025 21:45:34 +0100
Subject: [PATCH v20251111 3/7] clock-sweep: balancing of allocations

 
index 1856efb8786..8536e2debef 100644

@@ -27,7 +29,7 @@

 PG_MODULE_MAGIC_EXT(
 					.name = "pg_buffercache",
@@ -794,6 +796,8 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 
 	if (SRF_IS_FIRSTCALL())
 	{
 
@@ -823,6 +827,12 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)

@@ -843,11 +853,17 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 					first_buffer,
 					last_buffer;

 		uint32		complete_passes,
 					next_victim_buffer,
-					buffer_allocs;
+					buffer_allocs,
+					buffer_req_allocs;
+
+		int		   *weights;
+		Datum	   *dweights;
+		ArrayType  *array;
 
 		Datum		values[NUM_BUFFERCACHE_PARTITIONS_ELEM];
 		bool		nulls[NUM_BUFFERCACHE_PARTITIONS_ELEM];
@@ -856,8 +872,16 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 						   &first_buffer, &last_buffer);

 		values[0] = Int32GetDatum(i);
 		nulls[0] = false;
@@ -883,6 +907,15 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		values[7] = Int64GetDatum(buffer_allocs);
 		nulls[7] = false;
 
index a3092ce801d..82c645a3b00 100644
@@ -3912,6 +3912,9 @@ BgBufferSync(WritebackContext *wb_context)

index d40b09f7e69..169071032b4 100644

+/*
 
+	/*
 
+	/*
 
 /*
 
 /*
 
 	/*

 /*
 
 	/*
 
+/*
+ * StrategySyncBalance
+		elog(DEBUG1, "rebalance skipped: not enough allocations (allocs: %u)",
+		elog(DEBUG1, "rebalance skipped: delta within limit (delta: %u, threshold: %u)",

 		/* No pending notification */
@@ -1001,8 +1344,10 @@ StrategyRejectBuffer(BufferAccessStrategy strategy, BufferDesc *buf, bool from_r
 
 void
 
@@ -1010,11 +1355,21 @@ ClockSweepPartitionGetInfo(int idx,

 
index 3307190f611..1118b386228 100644
@@ -508,6 +508,7 @@ extern BufferDesc *StrategyGetBuffer(BufferAccessStrategy strategy,

 
 /* inline functions */
 
-- 
2.51.1

From 34c5d3574fa7efa48de7d5a99921911b63d04b30 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 11 Nov 2025 12:03:32 +0100
Subject: [PATCH v20251111 2/7] clock-sweep: basic partitioning

 src/backend/storage/buffer/buf_init.c         |   8 +
 9 files changed, 433 insertions(+), 106 deletions(-)

index 8a980bd1864..1856efb8786 100644

 PG_MODULE_MAGIC_EXT(
@@ -809,12 +809,20 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)

@@ -835,12 +843,22 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 					first_buffer,
 					last_buffer;

 
 
@@ -853,6 +871,18 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		values[3] = Int32GetDatum(last_buffer);
 		nulls[3] = false;
 
index 528a368a8b7..0362fda24aa 100644
@@ -319,3 +319,11 @@ BufferPartitionGet(int idx, int *num_buffers,
 
 	elog(ERROR, "invalid partition index");
 }
+
+/* return parameters before the partitions are initialized (during sizing) */
+void
+BufferPartitionParams(int *num_partitions)
+{
+	if (num_partitions)
+		*num_partitions = NUM_CLOCK_SWEEP_PARTITIONS;
+}
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 327ddb7adc8..a3092ce801d 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3608,33 +3608,29 @@ BufferSync(int flags)
 }
 
 /*
 
 	/* Moving averages of allocation rate and clean-buffer density */
 	static float smoothed_alloc = 0;
@@ -3662,25 +3658,16 @@ BgBufferSync(WritebackContext *wb_context)
 	long		new_strategy_delta;
 	uint32		new_recent_alloc;

 	/*
 	 * Compute strategy_delta = how many buffers have been scanned by the
@@ -3692,17 +3679,17 @@ BgBufferSync(WritebackContext *wb_context)
 	 */
 	if (saved_info_valid)
 	{
-		int32		passes_delta = strategy_passes - prev_strategy_passes;
+		int32		passes_delta = strategy_passes - prev_strategy_passes[partition];

 
@@ -3710,11 +3697,11 @@ BgBufferSync(WritebackContext *wb_context)
@@ -3734,9 +3721,9 @@ BgBufferSync(WritebackContext *wb_context)
@@ -3750,15 +3737,16 @@ BgBufferSync(WritebackContext *wb_context)

 	/*
@@ -3766,9 +3754,9 @@ BgBufferSync(WritebackContext *wb_context)
@@ -3778,7 +3766,7 @@ BgBufferSync(WritebackContext *wb_context)
 
 	/*
@@ -3786,10 +3774,10 @@ BgBufferSync(WritebackContext *wb_context)
 
@@ -3816,7 +3804,7 @@ BgBufferSync(WritebackContext *wb_context)
 
@@ -3841,20 +3829,20 @@ BgBufferSync(WritebackContext *wb_context)

@@ -3868,7 +3856,7 @@ BgBufferSync(WritebackContext *wb_context)
 
 #ifdef BGW_DEBUG
@@ -3898,8 +3886,74 @@ BgBufferSync(WritebackContext *wb_context)
 #endif
 	}

 /*
diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index 28d952b3534..d40b09f7e69 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -15,27 +15,47 @@
  */
 #include "postgres.h"
 
+#ifdef USE_LIBNUMA
+#include <sched.h>
+#endif
+
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
 #include "pgstat.h"

 /*
 
 	/*

 
 
 /*
 
 	/*

 
 		/*

 
 /*

+/*

 	if (complete_passes)
 	{
 
 		/*
 
-	if (num_buf_alloc)
-	{
 
 /*
@@ -380,6 +538,9 @@ Size
 StrategyShmemSize(void)
 {
 	Size		size = 0;
+	int			num_partitions;
+

 
 
+	int			num_partitions;
+
 
 	if (!found)

 
-		/* Clear statistics */
 
@@ -803,3 +998,23 @@ StrategyRejectBuffer(BufferAccessStrategy strategy, BufferDesc *buf, bool from_r
 
 	return true;
 }
+
+void
index 139055a4a7d..3307190f611 100644
@@ -508,7 +508,9 @@ extern BufferDesc *StrategyGetBuffer(BufferAccessStrategy strategy,

@@ -554,5 +556,6 @@ extern int	BufferPartitionCount(void);

 /* inline functions */
 
index 56e25026fbf..cb52c417592 100644
2.51.1

From fc35b864c5350b8640000368ddc1d82bd4ad3756 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 17 Sep 2025 23:04:29 +0200
Subject: [PATCH v20251111 1/7] Infrastructure for partitioning shared buffers

Notes:

* Maybe the number of partitions should be configurable? Right now it's
  hard-coded as 4, but testing shows increasing to e.g. 16) can be
  beneficial.

* This partitioning is independent of the partitions defined in
 src/backend/storage/buffer/buf_init.c         | 144 +++++++++++++++++-
 9 files changed, 284 insertions(+), 3 deletions(-)

index ab790533ff6..8a980bd1864 100644

 PG_MODULE_MAGIC_EXT(

 /* Only need to touch memory once per backend process lifetime */
@@ -776,3 +778,87 @@ pg_buffercache_evict_all(PG_FUNCTION_ARGS)
 
 	PG_RETURN_DATUM(result);
 }
+
+/*
index 6fd3a6bbac5..528a368a8b7 100644

+/* *
 
 /*
  * Data Structures:

@@ -175,5 +199,123 @@ BufferManagerShmemSize(void)
 
index 5400c56a965..139055a4a7d 100644
@@ -345,6 +345,7 @@ typedef struct WritebackContext
 
 /* in buf_init.c */
 
@@ -549,4 +550,9 @@ extern void DropRelationLocalBuffers(RelFileLocator rlocator,
 extern void DropRelationAllLocalBuffers(RelFileLocator rlocator);
 extern void AtEOXact_LocalBuffers(bool isCommit);

 
+/*
 
index 432509277c9..56e25026fbf 100644
2.51.1

---

From: Jakub Wartak <[email-masked]>
Date: Mon, 17 Nov 2025 10:23:50 +0100

Hi Tomas,

> buffers, that's indeed useful.

Both, even the _pgproc.

> partitioning, I don't think that's likely to go into 19.

Oh ok.

[..]
> yet, so no response from me.

Ok, so the proper HP + SIGBUS explanation:

Appologies, earlier I wrote that disabling THP does workaround this,
but I've probably made an error there and used wrong binary back there
(with MAP_POPULATE in PG_MMAP_FLAGS), so please ignore that.

1. Before starting PG, with shared_buffers=32GB, huge_pages=on (2MB
ones), vm.nr_hugepages=17715, 4 NUMA nodes, kernel 6.14.x,
max_connections=10k, wal_buffers=1GB:

node0/hugepages/hugepages-2048kB/free_hugepages:4429
node1/hugepages/hugepages-2048kB/free_hugepages:4429
node2/hugepages/hugepages-2048kB/free_hugepages:4429
node3/hugepages/hugepages-2048kB/free_hugepages:4428

2. Just startup the PG with the older NUMA patchset 20251101. There
will be deficit across NUMA nodes right after startup, mostly one node
NUMA will allocate much more:

node0/hugepages/hugepages-2048kB/free_hugepages:4397
node1/hugepages/hugepages-2048kB/free_hugepages:3453
node2/hugepages/hugepages-2048kB/free_hugepages:4397
node3/hugepages/hugepages-2048kB/free_hugepages:4396

3. Check layout of NUMA maps for postmaster PID

7fc9cb200000 default file=/anon_hugepage\040(deleted) huge dirty=517
mapmax=8 N1=517 kernelpagesize_kB=2048 [!!!]
7fca0d600000 bind:0 file=/anon_hugepage\040(deleted) huge dirty=32
mapmax=2 N0=32 kernelpagesize_kB=2048
7fca11600000 bind:1 file=/anon_hugepage\040(deleted) huge dirty=32
mapmax=2 N1=32 kernelpagesize_kB=2048
7fca15600000 bind:2 file=/anon_hugepage\040(deleted) huge dirty=32
mapmax=2 N2=32 kernelpagesize_kB=2048
7fca19600000 bind:3 file=/anon_hugepage\040(deleted) huge dirty=32
mapmax=2 N3=32 kernelpagesize_kB=2048
7fca1d600000 default file=/anon_hugepage\040(deleted) huge
7fca1d800000 bind:0 file=/anon_hugepage\040(deleted) huge
7fcc1d800000 bind:1 file=/anon_hugepage\040(deleted) huge
7fce1d800000 bind:2 file=/anon_hugepage\040(deleted) huge
7fd01d800000 bind:3 file=/anon_hugepage\040(deleted) huge
7fd21d800000 default file=/anon_hugepage\040(deleted) huge dirty=425
mapmax=8 N1=425 kernelpagesize_kB=2048 [!!!]

So your patch doesn't do anything special for anything other than
Buffer Blocks and PGPROC in the above picture, so the the default
mmap() just keeps on with "default" NUMA policy which takes per above
(517+425) * 2MB = ~1884 MB of really used memory as per N1 entires. PG
does touch those regions on startup, but it doesnt really touch Buffer
Blocks. Anyway, this causes the missing amount of free huge pages on
the N1 (generates pressure on this Node 1).

So as it stands, the patchset is missing some form balancing to use
equal memory across nodes:
- each node to be forced to get certain amount of BufferBlocks/NUMA nodes blocks
- yet we do nothing and leave at the "defaults" the others regions
(e..g $SegHDR (start of shm) .. first Buffers Block), as those are
placed on the current node (due default policy), which in causes turns
this memory overallocation imbalance (so in the example N1 will get
Buffer Blocks + everything else, but that only happens on real access
not during mmap() due to lazy/first touch policy)

Currently, any launch of anything that touches imbalanced NUMA node
memory with deficit (N1 above) - use of pg_shm_allocations,
pg_buffercache - it will cause stress there and end up in SIGBUS.
This looks by design on Linux kernel side: exc:page_fault() ->
do_user_addr_fault() -> do_sigbus() AKA force_sig_fault(). But, if I
hack pg to hack do interleave (or just numactl --interleave=all ... )
to effectivley interleave those 3 "default" regions instead, so I'll
get "interleave" like that:

7fb2dd000000 interleave:0-3 file=/anon_hugepage\040(deleted) huge
dirty=517 mapmax=8 N0=129 N1=132 N2=128 N3=128 kernelpagesize_kB=2048
7fb31f400000 bind:0 file=/anon_hugepage\040(deleted) huge dirty=32
mapmax=2 N0=32 kernelpagesize_kB=2048
7fb323400000 bind:1 file=/anon_hugepage\040(deleted) huge dirty=32
mapmax=2 N1=32 kernelpagesize_kB=2048
7fb327400000 bind:2 file=/anon_hugepage\040(deleted) huge dirty=32
mapmax=2 N2=32 kernelpagesize_kB=2048
7fb32b400000 bind:3 file=/anon_hugepage\040(deleted) huge dirty=32
mapmax=2 N3=32 kernelpagesize_kB=2048
7fb32f400000 interleave:0-3 file=/anon_hugepage\040(deleted) huge
7fb32f600000 bind:0 file=/anon_hugepage\040(deleted) huge
7fb52f600000 bind:1 file=/anon_hugepage\040(deleted) huge
7fb72f600000 bind:2 file=/anon_hugepage\040(deleted) huge
7fb92f600000 bind:3 file=/anon_hugepage\040(deleted) huge
7fbb2f600000 interleave:0-3 file=/anon_hugepage\040(deleted) huge
dirty=425 N0=106 N1=106 N2=105 N3=108 kernelpagesize_kB=2048

then even after fully touching everything (via select to
pg_shm_allocations), it'll run, I'll get much better balance, and wont
have SIGBUS issues:

node0/hugepages/hugepages-2048kB/free_hugepages:23
node1/hugepages/hugepages-2048kB/free_hugepages:23
node2/hugepages/hugepages-2048kB/free_hugepages:23
node3/hugepages/hugepages-2048kB/free_hugepages:22

This somehow demonstrates that enough free memory is out there, it's
just imbalance that causes SIGBUS. I hope this somehow hopefully
answers one of Your's main questions as per in the very first messages
what we should do with remaining shared_buffer members. I would like
to hear your thoughts on this, before I start benchmarking this for
real as I didnt want to bench it yet, as such interleaving could alter
the the test results.

Other things I've noticed:
- smaps Size: && Shared_Hugetlb: reporting are a lie and are showing
really touched memory, not assigned memory
- same goes for procfs's numa_maps, ignore the N[0-3] sizes, it's only
"really used", not assigned
- the best is just to manually calculate size from pointers/address range itself

-J.

---

From: Tomas Vondra <[email-masked]>
Date: Mon, 17 Nov 2025 18:28:34 +0100

> real as I didnt want to bench it yet, as such interleaving could alter

Thanks for investigating this. If I understand the findings correctly,
it agrees with my imprecise explanation in [1], right? There I said:

> shared_memory_size_in_huge_pages, right? And if we were partitioning
> the whole memory segment, that'd be true. But we only to that for
> shared buffers, and there's a lot of other shared memory - could be
> 1-2GB or so, depending on the configuration.
>
> limit on that particular node. And so it doesn't have enough huge
> pages to back the partition of shared buffers.
> ...

Which I think is mostly the same thing you're saying, and you have the
maps to support it.

In any case, I think setting "interleave" as the default policy, and
then overriding it for the areas we partition explicitly (buffers,
pgproc), seems like the right solution. The only other solution would be
balance it ourselves, but how is that different from interleaving?

So I think this makes sense, and you can do --interleave=all for the
benchmark.

[1]

I suppose we may need to adjust shared_memory_size_in_huge_pages,
because the interleave followed by explicit partitioning may still leave
behind a bit of imbalance. It should be only a couple pages, but I
haven't done the math yet.

regards

-- 
Tomas Vondra

---

From: Tomas Vondra <[email-masked]>
Date: Fri, 21 Nov 2025 19:49:25 +0100

Hi,

Here's an updated version of the patch series.

It fixes a bunch of issues in pg_buffercache_pages.c - duplicate attnums
and a incorrect array length.

The main change is in 0006 - it sets the default allocation policy for
shmem to interleaving, before doing the explicit partitioning for shared
buffers. It does it by calling numa_set_membind before the mmap(), and
then numa_interleave_memory() on the allocated shmem. It does this to
allow using MAP_POPULATE - but that's commented out by default.

This does seem to solve the SIGBUS failures for me. I still think there
might be a small chance of hitting that, because of locating an extra
"boundary" page on one of the nodes. But it should be solvable by
reserving a couple more pages.

Jakub, what do you think?

regards

-- 
Tomas Vondra
From 80415fe4e0de3c642fc15721085ed5af3d6441b1 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 11 Nov 2025 12:10:03 +0100
Subject: [PATCH v20251121 7/7] NUMA: partition PGPROC

 - PGPROC array / node #0
 - PGPROC array / node #1

 15 files changed, 722 insertions(+), 72 deletions(-)

index 179a38fd6ed..cb33bf28b35 100644

 PG_MODULE_MAGIC_EXT(

 		/*

 

 
 /*

 	/*

 
 /*

+	AssertCheckAllProcs();
+

 

+	AssertCheckAllProcs();
+
 	/*
 
+	AssertCheckAllProcs();
+
 	myoff = proc->pgxactoff;

 

+	AssertCheckAllProcs();
+
 	/*

 
 
-
 /*

 			/*

 

 			/*

 

 		if (proc->pid == pid)
 		{
 
 			result = proc->pid;
 			break;
 
 		if (proc == MyProc)

 
 		/*
 
 		if (proc->pid == 0)
 
 		if (proc->pid == 0)

 		if (proc->pid == 0)

 
index 9cb78ead105..f82e664ad3f 100644

 
 
+#ifdef USE_LIBNUMA
+#include <sched.h>
+#endif
+
 #include <signal.h>
 #include <unistd.h>
 #include <sys/time.h>
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+

 

 
+	/*

 
+	/*

 

 

 
+	/*

 
 		/*

 
 		/*

 

 
 
 		if (proc->pid == pid)
 		{

 /*
 
 	return ok;
 }
+
+/*

 #ifdef USE_LIBNUMA

 

 

 /*
 
 /*
 
 #ifdef EXEC_BACKEND

+extern int	ProcPartitionCount(void);
+extern void ProcPartitionGet(int idx, int *node, int *nprocs, void **procsptr, void **fpptr);
+
 #endif							/* _PROC_H_ */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index fa3bb09effe..c7d0bf10555 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1884,6 +1884,7 @@ PGP_MPI
 PGP_PubKey
 PGP_S2K
 PGPing
+PGProcPartition
 PGQueryClass
 PGRUsage
 PGSemaphore
-- 
2.51.1

From a31e641c96beccea652f4e93ecee22398ff80b15 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 11 Nov 2025 12:05:35 +0100
Subject: [PATCH v20251121 6/7] NUMA: shared buffers partitioning

The effect is similar to

     numactl --interleave=all

Notes:

 contrib/pg_buffercache/pg_buffercache_pages.c |  52 +-
 src/backend/port/sysv_shmem.c                 |  32 +
 12 files changed, 772 insertions(+), 72 deletions(-)

index aa8ea08e1bb..179a38fd6ed 100644
@@ -29,7 +29,7 @@

 PG_MODULE_MAGIC_EXT(
@@ -813,25 +813,27 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
-		TupleDescInitEntry(tupledesc, (AttrNumber) 9, "total_req_allocs",
-		TupleDescInitEntry(tupledesc, (AttrNumber) 10, "num_req_allocs",
-		TupleDescInitEntry(tupledesc, (AttrNumber) 11, "weigths",
 						   typentry->typarray, -1, 0);

-		int			num_buffers,
+		int			numa_node,
+					num_buffers,
 					first_buffer,
 					last_buffer;

 

 

 

 

+		values[11] = PointerGetDatum(array);
+		nulls[11] = false;
+
 		/* Build and return the tuple. */
 		tuple = heap_form_tuple((TupleDesc) funcctx->user_fctx, values, nulls);
 		result = HeapTupleGetDatum(tuple);
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 197926d44f6..6019bee334d 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -19,6 +19,7 @@
  */
 #include "postgres.h"
 
+#include <numa.h>
 #include <signal.h>
 #include <unistd.h>
 #include <sys/file.h>
@@ -602,6 +603,14 @@ CreateAnonymousSegment(Size *size)
 	void	   *ptr = MAP_FAILED;
 	int			mmap_errno = 0;
 
+	/*
+	 * Set the memory policy to interleave to all NUMA nodes before calling
+	 * mmap, in case we use MAP_POPULATE to prefault all the pages.
+	 *
+	 * XXX Probably not needed without that, but also costs nothing.
+	 */
+	numa_set_membind(numa_all_nodes_ptr);
+
 #ifndef MAP_HUGETLB
 	/* PGSharedMemoryCreate should have dealt with this case */
 	Assert(huge_pages != HUGE_PAGES_ON);
@@ -616,6 +625,9 @@ CreateAnonymousSegment(Size *size)
 
 		GetHugePageSize(&hugepagesize, &mmap_flags);
 
+		// prefault the memory at start?
+		// mmap_flags |= MAP_POPULATE;
+
 		if (allocsize % hugepagesize != 0)
 			allocsize += hugepagesize - (allocsize % hugepagesize);
 
@@ -638,6 +650,11 @@ CreateAnonymousSegment(Size *size)
 
 	if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
 	{
+		int			mmap_flags = 0;
+
+		// prefault the memory at start?
+		// mmap_flags |= MAP_POPULATE;
+
 		/*
 		 * Use the original size, not the rounded-up value, when falling back
 		 * to non-huge pages.
@@ -663,6 +680,21 @@ CreateAnonymousSegment(Size *size)
 						 allocsize) : 0));
 	}
 
+	/* undo the earlier num_set_membind() call. */
+	numa_set_localalloc();
+
+	/*
+	 * Before touching the memory, set the allocation policy, so that
+	 * it gets interleaved by default. We have to do this to distribute
+	 * the memory that's not located explicitly. We need this especially
+	 * with huge pages, where we could run out of huge pages on some
+	 * nodes and crash otherwise.
+	 *
+	 * XXX Probably not needed with MAP_POPULATE, in which case the policy
+	 * was already set by num_set_membind() earlier. But doesn't hurt.
+	 */
+	numa_interleave_memory(ptr, allocsize, numa_all_nodes_ptr);
+
 	*size = allocsize;
 	return ptr;
 }
diff --git a/src/backend/storage/buffer/buf_init.c b/src/backend/storage/buffer/buf_init.c
index 0362fda24aa..587859a5754 100644
--- a/src/backend/storage/buffer/buf_init.c
+++ b/src/backend/storage/buffer/buf_init.c
@@ -14,6 +14,12 @@
  */
 #include "postgres.h"
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
 
-/* *

 

 

 
 		/*

 

 	return size;
 }
 
+/*

 

 
 
-		buffer += num_buffers;

 int
 
+int

 

 /*

 
+	int			num_nodes;
 	int			num_partitions;

 	/*

 
 			/*

 
 #ifdef HAVE_SYSLOG
 
 #ifdef USE_LIBNUMA

 

 
 /* in localbuf.c */
 
+extern bool check_debug_numa(char **newval, void **extra, GucSource source);
+extern void assign_debug_numa(const char *newval, void *extra);
+
 #endif							/* GUC_HOOKS_H */
diff --git a/src/port/pg_numa.c b/src/port/pg_numa.c
index 540ada3f8ef..d9c3841e078 100644
--- a/src/port/pg_numa.c
+++ b/src/port/pg_numa.c
@@ -18,6 +18,9 @@
 
 #include "miscadmin.h"
 
 /*
  * At this point we provide support only for Linux thanks to libnuma, but in
@@ -116,6 +119,36 @@ pg_numa_get_max_node(void)
 	return numa_max_node();
 }
 
+/*
+ * pg_numa_move_to_node
 
 /* Empty wrappers */
@@ -138,4 +171,35 @@ pg_numa_get_max_node(void)
 	return 0;
 }
 
+void

From 1f98649b8db51c26d6488fc3cd30de8298518800 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 6 Aug 2025 01:09:57 +0200
Subject: [PATCH v20251121 5/7] clock-sweep: weighted balancing

 
+	/*

 	/*
 
 	/*
 
 	/*

 

 

From 4cbe8788366287b0347f006ee90bf1470af74877 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 15 Oct 2025 13:59:29 +0200
Subject: [PATCH v20251121 4/7] clock-sweep: scan all partitions

 
 /*
 
 	/*
 
 	*from_ring = false;

 
 		/*

From ee6b06ec3c011eb4530d39f071b2b2c09416cabc Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 29 Oct 2025 21:45:34 +0100
Subject: [PATCH v20251121 3/7] clock-sweep: balancing of allocations

 
index 11c22dda9b3..aa8ea08e1bb 100644

@@ -27,7 +29,7 @@

 PG_MODULE_MAGIC_EXT(
 
 	if (SRF_IS_FIRSTCALL())
 	{
 
 		/* Switch context when allocating stuff to be used in later calls */
@@ -823,6 +827,12 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 						   INT8OID, -1, 0);
 		TupleDescInitEntry(tupledesc, (AttrNumber) 8, "num_allocs",
 						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 9, "total_req_allocs",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 10, "num_req_allocs",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 11, "weigths",
+						   typentry->typarray, -1, 0);

 
 
 		uint32		complete_passes,
 					next_victim_buffer,
-					buffer_allocs;
+					buffer_allocs,
+					buffer_req_allocs;
+
+		int		   *weights;
+		Datum	   *dweights;
+		ArrayType  *array;

 

 

+/*
 
+	/*
 
+	/*
 
 /*
 
 /*
 
 	/*

 /*
 
 	/*
 
+/*
+ * StrategySyncBalance

 
 void

 

 
 
 /* inline functions */
 
-- 
2.51.1

From 920df09b7e585a44c2a528613786527461c5581c Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 11 Nov 2025 12:03:32 +0100
Subject: [PATCH v20251121 2/7] clock-sweep: basic partitioning

 9 files changed, 430 insertions(+), 103 deletions(-)

index f11ef46dbed..11c22dda9b3 100644

 PG_MODULE_MAGIC_EXT(
 					.name = "pg_buffercache",
@@ -815,6 +815,14 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 						   INT4OID, -1, 0);
 		TupleDescInitEntry(tupledesc, (AttrNumber) 4, "last_buffer",
 						   INT4OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 5, "num_passes",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 6, "next_buffer",
+						   INT4OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 7, "total_allocs",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 8, "num_allocs",
+						   INT8OID, -1, 0);

 

 

 
 /*

 
 	/*

 

 	/*
 
 	/*

 

 #ifdef BGW_DEBUG

 /*
 
+#ifdef USE_LIBNUMA
+#include <sched.h>
+#endif
+
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
 #include "pgstat.h"

 /*
 
 	/*

 
 
 /*
 
 	/*

 
 		/*

 
 /*

+/*

 	if (complete_passes)
 	{
 
 		/*
 
-	if (num_buf_alloc)
-	{
 
 /*
@@ -380,6 +538,9 @@ Size
 StrategyShmemSize(void)
 {
 	Size		size = 0;
+	int			num_partitions;
+

 
 
+	int			num_partitions;
+
 
 	if (!found)

 
-		/* Clear statistics */

 	return true;
 }
+
+void

 

 /* inline functions */
 
index 4bc3c31d638..fa3bb09effe 100644

From 9aef8aa328dda058ba3ea2e0dc4c65297ea09c3e Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 17 Sep 2025 23:04:29 +0200
Subject: [PATCH v20251121 1/7] Infrastructure for partitioning shared buffers

Notes:

 
index c29b784dfa1..f11ef46dbed 100644

 PG_MODULE_MAGIC_EXT(

 	PG_RETURN_DATUM(result);
 }
+
+/*

+/* *
 
 /*
  * Data Structures:

 
 
 /* in buf_init.c */

 
 
+/*
 
index 27a4d131897..4bc3c31d638 100644

---

From: Jakub Wartak <[email-masked]>
Date: Tue, 25 Nov 2025 15:12:46 +0100

Hi Tomas!

[..]
> Which I think is mostly the same thing you're saying, and you have the maps to support it.

Right, the thread is kind of long, you were right back then, well but
at least we've got a solid explanation with data.

> Here's an updated version of the patch series.

Just for double confirmation, I've used those ones (v20251121*) and
they indeed interleaved parts of shm memory.

> and a incorrect array length.

You'll need to rebase again, pg_buffercache_numa got updated again on
Monday and clashes with 0006.

> reserving a couple more pages.

I can confirm, never got any SIGBUS during the later described
benchmarks, so it's much better now.

> Jakub, what do you think?

On one side not using MAP_POPULATE gives instant startup, but on the
other it gives much better predictability latencies especially fresh
after starting up (this might matter to folks who like to benchmark --
us?, but initially I've just used it as a simple hack to touch
memory). I would be wary of using MAP_POPULATE with s_b when it would
be sized in hundreths of GBs, it could take minutes in startup, which
would be terrible if someone would hit SIGSEGV on production and
expect restart_after_crash=true to save him. I mean WAL redo crash
would be terrible, but that would be terrible * 2. Also pretty
long-term with DIO, we'll get much bigger s_b anyway (hopefully), so
it would hurt even more, so I think that would be a bad path(?)

I've benchmarked the thing in two scenarios (readonly pgbench < s_b
size across variations of code and connections and 2nd one with
seqconcurrrentscans) in solid stable conditions: 4s32c64t == 4 NUMA
nodes, 128GB RAM, 31GB shared_buffers dbsize ~29GB, 6.14.x, no idle
CPU states, no turbo boost, and so on, literally great home heater
when there's -3C outside!)

The data is baseline "100%" for master along with HP on/off (so it's
showing diff % from respective HP setting):

scenario I: pgbench -S

                 connections
branch   HP      1       8       64      128     1024
master   off     100.00% 100.00% 100.00% 100.00% 100.00%
master   on      100.00% 100.00% 100.00% 100.00% 100.00%
numa16   off     99.13%  100.46% 99.66%  99.44%  89.60%
numa16   on      101.80% 100.89% 99.36%  99.89%  93.43%
numa4    off     96.82%  100.61% 99.37%  99.92%  94.41%
numa4    on      101.83% 100.61% 99.35%  99.69%  101.48%
pgproc16 off     99.13%  100.84% 99.38%  99.85%  91.15%
pgproc16 on      101.72% 101.40% 99.72%  100.14% 95.20%
pgproc4  off     98.63%  101.44% 100.05% 100.14% 90.97%
pgproc4  on      101.05% 101.46% 99.92%  100.31% 97.60%
sweep16  off     99.53%  101.14% 100.71% 100.75% 101.52%
sweep16  on      97.63%  102.49% 100.42% 100.75% 105.56%
sweep4   off     99.43%  101.59% 100.06% 100.45% 104.63%
sweep4   on      97.69%  101.59% 100.70% 100.69% 104.70%

I would consider everything +/- 3% as noise (technically each branch
was a different compilation/ELF binary, as changing this #define
required to do so to get 4 vs 16; please see attached script). I miss
the explanation why without HP it deteriorates so much with for c=1024
with the patches.

scenario II: pgbench -f seqconcurrscans.pgb; 64 partitions from
pgbench --partitions=64 -i -s 2000 [~29GB] being hammered in modulo
without PQ by:
    \set num (:client_id % 8) + 1
    select sum(octet_length(filler)) from pgbench_accounts_:num;

                 connections
branch   HP      1       8       64      128
master   off     100.00% 100.00% 100.00% 100.00%
master   on      100.00% 100.00% 100.00% 100.00%
numa16   off     115.62% 108.87% 101.08% 111.56%
numa16   on      107.68% 104.90% 102.98% 105.51%
numa4    off     113.55% 111.41% 101.45% 113.10%
numa4    on      107.90% 106.60% 103.68% 106.98%
pgproc16 off     111.70% 108.27% 98.69%  109.36%
pgproc16 on      106.98% 100.69% 101.98% 103.42%
pgproc4  off     112.41% 106.15% 100.03% 112.03%
pgproc4  on      106.73% 105.77% 103.74% 101.13%
sweep16  off     100.63% 100.38% 98.41%  103.46%
sweep16  on      109.03% 99.15%  101.17% 99.19%
sweep4   off     102.04% 101.16% 101.71% 91.86%
sweep4   on      108.33% 101.69% 97.14%  100.92%

The benefit varies with like +3-10% depending on connection count.
Quite frankly I was expecting a little bit more, especially after
re-reading [1]. Maybe you preloaded it there using pg_prewarm? (here
I've randomly warmed it using pgbench). Probably it's something with
my test, I'll take yet another look hopefully soon. The good thing is
that it never crashed and I haven't seen any errors like "Bad address"
probably related to AIO as you saw in [1], perhaps I wasn't using
uring.

0007 (PROCs) still complains with "mbind: Invalid argument" (aligment issue)

-J.

[1] - https://www.postgresql.org/message-id/e4d7e6fc-b5c5-4288-991c-56219db2edd5%40vondra.me

---

From: Tomas Vondra <[email-masked]>
Date: Wed, 26 Nov 2025 17:19:12 +0100

> Monday and clashes with 0006.

Rebased patch series attached.

> benchmarks, so it's much better now.

Good!

> it would hurt even more, so I think that would be a bad path(?)

I think the MAP_POPULATE should be optional, enabled by GUC.

> the explanation why without HP it deteriorates so much with for c=1024

I wouldn't expect a big difference for "pgbench -S". That workload has
so much other fairly expensive stuff (e.g. initializing index scans
etc.), the cost of buffer replacement is going to be fairly limited.

The regressions for numa/pgproc patches with 1024 clients are annoying,
but how realistic is such scenario? With 32/64 CPUs, having 1024 active
connections is a substantial overload. If we can fix this, great. But I
think such regression may be OK if we get benefits for reasonable setups
(with fewer clients).

I don't know why it's happening, though. I haven't been testing cases
with so many clients (compared to number of CPUs).

> probably related to AIO as you saw in [1], perhaps I wasn't using

Hmmm. I'd have expected better results for this workload. So I tried
re-running my seqscan benchmark on the 176-core instance, and I got this:

    clients   master   0001   0002   0003   0004   0005   0006   0007
    -----------------------------------------------------------------
     64           44     43     35     40     53     53     46     45
     96           55     54     42     47     57     58     53     53
    128           59     59     46     50     58     58     57     60

    clients   0001   0002   0003   0004   0005   0006   0007
    --------------------------------------------------------
     64        98%    79%    92%   122%   122%   105%   104%
     96        99%    76%    86%   104%   105%    97%    97%
    128        99%    77%    84%    98%    98%    97%   101%

I did the benchmark for individual parts of the patch series. There's a
clear (~20%) speedup for 0005, but 0006 and 0007 make it go away. The
0002/0003 regress it quite a bit. And with 128 clients there's no
improvement at all.

This was with the default number of partitions (i.e. 4). If I increase
the number to 16, I get this:

    clients   master   0001   0002   0003   0004   0005   0006   0007
    -----------------------------------------------------------------
     64           44     43     69     82     87     87     78     79
     96           55     54     65     85     91     91     86     86
    128           59     59     66     77     83     83     82     86

    clients   0001   0002   0003   0004   0005   0006   0007
    --------------------------------------------------------
     64        99%   158%   189%   199%   199%   180%   180%
     96       100%   119%   156%   167%   167%   157%   158%
    128        99%   112%   130%   140%   140%   139%   145%

And with 32 partitions, I get this:

    clients   master   0001   0002   0003   0004   0005   0006   0007
    -----------------------------------------------------------------
     64           44     44     88     91     90     90     84     84
     96           55     54     89     93     93     92     90     91
    128           59     59     85     84     86     85     88     87

    clients   0001   0002   0003   0004   0005   0006   0007
    --------------------------------------------------------
     64       100%   202%   208%   207%   207%   193%   193%
     96       100%   163%   169%   171%   168%   165%   166%
    128        99%   144%   142%   146%   144%   149%   146%

Those are clearly much better results, so I guess the default number of
partitions may be too low.

What bothers me is that this seems like a very narrow benchmark. I mean,
few systems are doing concurrent seqscans putting this much pressure on
buffer replacement. And once the plans start to do other stuff, the
contention on clock sweep seems to go down substantially (as shown by
the read-only pgbench). So the question is - is this really worth it?

> 0007 (PROCs) still complains with "mbind: Invalid argument" (aligment issue)

Should be fixed by the attached patches. The 0006 patch has an issue
with mbind too, but it was visible only when the buffers were not a nice
multiple of memory pages (and multiples of 1GB are fine).

This also moves the memset() until after placing the PGPROC partitions
to different NUMA nodes.

The results above are from v20251121. I'll rerun the tests with the nw
version of the patches. But it can only change the 0006/0007 results, of
course. The 0001-0005 are the same.

regards

-- 
Tomas Vondra
From 6a7b5f0c48b861a78baafad699802752edfce669 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 26 Nov 2025 17:13:09 +0100
Subject: [PATCH v20251126 9/9] mbind: procs

---
 src/backend/storage/lmgr/proc.c | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)

diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index 0a0ce98b725..3d76a3a6429 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -206,11 +206,14 @@ FastPathLockShmemSize(void)
 	 * when not strictly needed (if it's already aligned). And we always
 	 * assume we'll add a whole page, even if the alignment needs only less
 	 * memory.
+	 *
+	 * XXX We need two extra pages. One for the non-NUMA part (aux processes),
+	 * and one to keep the size of the last chunk aligned too.
 	 */
 	if (((numa_flags & NUMA_PROCS) != 0) && numa_can_partition)
 	{
 		Assert(numa_nodes > 0);
-		size = add_size(size, mul_size((numa_nodes + 1), numa_page_size));
+		size = add_size(size, mul_size((numa_nodes + 2), numa_page_size));
 	}
 
 	return size;
@@ -324,6 +327,7 @@ InitProcGlobal(void)
 
 	/* Used for setup of per-backend fast-path slots. */
 	char	   *fpPtr,
+			   *fpPtrOrig,
 			   *fpEndPtr PG_USED_FOR_ASSERTS_ONLY;
 	Size		fpLockBitsSize,
 				fpRelIdSize;
@@ -507,7 +511,7 @@ InitProcGlobal(void)
 							requestSize,
 							&found);
 
-	MemSet(fpPtr, 0, requestSize);
+	fpPtrOrig = fpPtr;
 
 	/* For asserts checking we did not overflow. */
 	fpEndPtr = fpPtr + requestSize;
@@ -595,6 +599,9 @@ InitProcGlobal(void)
 		Assert(fpPtr <= fpEndPtr);
 	}
 
+	/* zero the memory only after locating the memory to NUMA nodes */
+	MemSet(fpPtrOrig, 0, requestSize);
+
 	for (i = 0; i < TotalProcs; i++)
 	{
 		PGPROC	   *proc = procs[i];
@@ -2521,7 +2528,10 @@ fastpath_partition_init(char *ptr, int num_procs, int allprocs_index, int node,
 	 * memory, to make sure it's not mapped to any node yet
 	 */
 	if (node != -1)
+	{
+		endptr = (char *) TYPEALIGN(numa_page_size, endptr);
 		pg_numa_move_to_node(ptr, endptr, node);
+	}
 
 	/*
 	 * Now point the PGPROC entries to the fast-path arrays, and also advance
@@ -2550,7 +2560,8 @@ fastpath_partition_init(char *ptr, int num_procs, int allprocs_index, int node,
 		allprocs_index++;
 	}
 
-	Assert(ptr == endptr);
+	Assert(ptr <= endptr);
+	Assert((node == -1) || (char *) TYPEALIGN(numa_page_size, ptr) == endptr);
 
 	return endptr;
 }
-- 
2.51.1

From b6e4879a397210aff0f0f720708471568fe99ea4 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 11 Nov 2025 12:10:03 +0100
Subject: [PATCH v20251126 8/9] NUMA: partition PGPROC

 - PGPROC array / node #0
 - PGPROC array / node #1

 .../pg_buffercache--1.7--1.8.sql              |  19 +

diff --git a/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql b/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
index 43d2e84f9d2..265c35c8252 100644
--- a/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
+++ b/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
@@ -31,3 +31,22 @@ REVOKE ALL ON pg_buffercache_partitions FROM PUBLIC;
 
index eae75375152..c6629f767d1 100644
 
 #define NUM_BUFFERCACHE_OS_PAGES_ELEM	3
 #define NUM_BUFFERCACHE_PARTITIONS_ELEM	12
+#define NUM_BUFFERCACHE_PGPROC_ELEM	5
 
 PG_MODULE_MAGIC_EXT(
@@ -105,6 +107,7 @@ PG_FUNCTION_INFO_V1(pg_buffercache_evict);

@@ -981,3 +984,94 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 
 		/*

 

 
 /*
diff --git a/src/backend/storage/buffer/buf_init.c b/src/backend/storage/buffer/buf_init.c
index 9ba455488a0..55016cce93f 100644
--- a/src/backend/storage/buffer/buf_init.c
+++ b/src/backend/storage/buffer/buf_init.c
@@ -824,6 +824,8 @@ check_debug_numa(char **newval, void **extra, GucSource source)

 	/*

 
 /*

+	AssertCheckAllProcs();
+

 

+	AssertCheckAllProcs();
+
 	/*
 
+	AssertCheckAllProcs();
+
 	myoff = proc->pgxactoff;

 

+	AssertCheckAllProcs();
+
 	/*

 
 
-
 /*

 			/*

 

 			/*

 

 		if (proc->pid == pid)
 		{
 
 			result = proc->pid;
 			break;
 
 		if (proc == MyProc)

 
 		/*
 
 		if (proc->pid == 0)
 
 		if (proc->pid == 0)

 		if (proc->pid == 0)

 

 
 
+#ifdef USE_LIBNUMA
+#include <sched.h>
+#endif
+
 #include <signal.h>
 #include <unistd.h>
 #include <sys/time.h>
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+

 

 
+	/*

 
+	/*

 

 

 
+	/*

 
 		/*

 
 		/*

 

 
 
 		if (proc->pid == pid)
 		{

 /*
 
 	return ok;
 }
+
+/*

 #ifdef USE_LIBNUMA

 

 

 /*
 
 /*
 
 #ifdef EXEC_BACKEND

+extern int	ProcPartitionCount(void);
+extern void ProcPartitionGet(int idx, int *node, int *nprocs, void **procsptr, void **fpptr);
+
 #endif							/* _PROC_H_ */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 241f175e9da..e1bf02a3567 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1888,6 +1888,7 @@ PGP_MPI
 PGP_PubKey
 PGP_S2K
 PGPing
+PGProcPartition
 PGQueryClass
 PGRUsage
 PGSemaphore
-- 
2.51.1

From 191b5b6503c0884260d14479fe5b19fc1cc7cc86 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 26 Nov 2025 17:12:57 +0100
Subject: [PATCH v20251126 7/9] mbind: buffers

---
 src/backend/storage/buffer/buf_init.c | 29 +++++++++++++++++++++++++--
 1 file changed, 27 insertions(+), 2 deletions(-)

diff --git a/src/backend/storage/buffer/buf_init.c b/src/backend/storage/buffer/buf_init.c
index 587859a5754..9ba455488a0 100644
--- a/src/backend/storage/buffer/buf_init.c
+++ b/src/backend/storage/buffer/buf_init.c
@@ -272,6 +272,13 @@ BufferManagerShmemSize(void)
 	size = add_size(size, Max(numa_page_size, PG_IO_ALIGN_SIZE));
 	size = add_size(size, mul_size(NBuffers, BLCKSZ));
 
+	/*
+	 * Extra alignment, so that the partitions are whole memory pages (we
+	 * may need to pad the last one, so one page is enough). Without this
+	 * we may get mbind() failures in pg_numa_move_to_node().
+	 */
+	size = add_size(size, Max(numa_page_size, PG_IO_ALIGN_SIZE));
+
 	/* size of stuff controlled by freelist.c */
 	size = add_size(size, StrategyShmemSize());
 
@@ -695,6 +702,15 @@ buffer_partitions_init(void)
 		/* first map buffers */
 		startptr = buffers_ptr;
 		endptr = startptr + ((Size) num_buffers * BLCKSZ);
+
+		/*
+		 * Make sure the partition is a multiple of memory page, so that we
+		 * don't get mbind failures in move_to_node calls. This matters only
+		 * for the last partition, the earlier ones should be always sized
+		 * as multiples of pages.
+		 */
+		endptr = (char *) TYPEALIGN(numa_page_size, endptr);
+
 		buffers_ptr = endptr;	/* start of the next partition */
 
 		elog(DEBUG1, "NUMA: buffer_partitions_init: %d => buffers %d start %p end %p (size %zd)",
@@ -705,6 +721,15 @@ buffer_partitions_init(void)
 		/* now do the same for buffer descriptors */
 		startptr = descriptors_ptr;
 		endptr = startptr + ((Size) num_buffers * sizeof(BufferDescPadded));
+
+		/*
+		 * Make sure the partition is a multiple of memory page, so that we
+		 * don't get mbind failures in move_to_node calls. This matters only
+		 * for the last partition, the earlier ones should be always sized
+		 * as multiples of pages.
+		 */
+		endptr = (char *) TYPEALIGN(numa_page_size, endptr);
+
 		descriptors_ptr = endptr;
 
 		elog(DEBUG1, "NUMA: buffer_partitions_init: %d => descriptors %d start %p end %p (size %zd)",
@@ -714,8 +739,8 @@ buffer_partitions_init(void)
 	}
 
 	/* we should have consumed the arrays exactly */
-	Assert(buffers_ptr == BufferBlocks + (Size) NBuffers * BLCKSZ);
-	Assert(descriptors_ptr == (char *) BufferDescriptors + (Size) NBuffers * sizeof(BufferDescPadded));
+	Assert(buffers_ptr <= (char *) TYPEALIGN(numa_page_size, BufferBlocks + (Size) NBuffers * BLCKSZ));
+	Assert(descriptors_ptr == (char *) TYPEALIGN(numa_page_size, (char *) BufferDescriptors + (Size) NBuffers * sizeof(BufferDescPadded)));
 }
 
 int
-- 
2.51.1

From 4b3779f92217900c0eea91ef68810867362a7170 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 11 Nov 2025 12:05:35 +0100
Subject: [PATCH v20251126 6/9] NUMA: shared buffers partitioning

The effect is similar to

     numactl --interleave=all

Notes:

 .../pg_buffercache--1.7--1.8.sql              |   1 +
 src/backend/port/sysv_shmem.c                 |  34 +-
 12 files changed, 773 insertions(+), 73 deletions(-)

diff --git a/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql b/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
index 1834599c4b3..43d2e84f9d2 100644
--- a/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
+++ b/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
@@ -11,6 +11,7 @@ LANGUAGE C PARALLEL SAFE;
 CREATE VIEW pg_buffercache_partitions AS
 	SELECT P.* FROM pg_buffercache_partitions() AS P
 	(partition integer,			-- partition index
+	 numa_node integer,			-- NUMA node of the partitioon
 	 num_buffers integer,		-- number of buffers in the partition
 	 first_buffer integer,		-- first buffer of partition
 	 last_buffer integer,		-- last buffer of partition
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 81665209084..eae75375152 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -29,7 +29,7 @@
 #define NUM_BUFFERCACHE_EVICT_ALL_ELEM 3
 
 #define NUM_BUFFERCACHE_OS_PAGES_ELEM	3
-#define NUM_BUFFERCACHE_PARTITIONS_ELEM	11
+#define NUM_BUFFERCACHE_PARTITIONS_ELEM	12
 
 PG_MODULE_MAGIC_EXT(
@@ -863,25 +863,27 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 
 		funcctx->user_fctx = BlessTupleDesc(tupledesc);
@@ -899,7 +901,8 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 	{
 		uint32		i = funcctx->call_cntr;
 
-		int			num_buffers,
+		int			numa_node,
+					num_buffers,
 					first_buffer,
 					last_buffer;
 
@@ -918,7 +921,7 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		Datum		values[NUM_BUFFERCACHE_PARTITIONS_ELEM];
 		bool		nulls[NUM_BUFFERCACHE_PARTITIONS_ELEM];

 		ClockSweepPartitionGetInfo(i,
@@ -936,36 +939,39 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		values[0] = Int32GetDatum(i);
 		nulls[0] = false;

 

 

 

index 197926d44f6..78a0c5199f1 100644
 
+#include <numa.h>
 #include <signal.h>
 #include <unistd.h>
 #include <sys/file.h>
 
+	/*

 
@@ -638,13 +650,18 @@ CreateAnonymousSegment(Size *size)
 
 	if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
 	{
+		int			mmap_flags = 0;
+
+		// prefault the memory at start?
+		// mmap_flags |= MAP_POPULATE;
+
 		/*
 		 * Use the original size, not the rounded-up value, when falling back
 		 * to non-huge pages.
 		 */
 		allocsize = *size;
 		ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
-				   PG_MMAP_FLAGS, -1, 0);
+				   PG_MMAP_FLAGS | mmap_flags, -1, 0);
 		mmap_errno = errno;
 	}

 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
 
-/* *

 

 

 
 		/*

 

 	return size;
 }
 
+/*

 

 
 
-		buffer += num_buffers;

 int
 
+int

 

 /*

 
+	int			num_nodes;
 	int			num_partitions;

 	/*

 
 			/*
 
index 3b9d8349078..b13e667ac9e 100644
@@ -643,6 +643,16 @@
 
index f87b558c2c6..3f29eeaf5ae 100644
 
 #ifdef HAVE_SYSLOG
 
 #ifdef USE_LIBNUMA

 

 
 /* in localbuf.c */

 #include "miscadmin.h"
 
 /*
 
+/*
+ * pg_numa_move_to_node
 
 /* Empty wrappers */
 
+void

From 16013b08d9a524493c455278b1f4d97f237ec99e Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 6 Aug 2025 01:09:57 +0200
Subject: [PATCH v20251126 5/9] clock-sweep: weighted balancing

 
+	/*

 	/*
 
 	/*
 
 	/*

 

 

From 5df379b3a4f9773de589120c3a609d2008663620 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 15 Oct 2025 13:59:29 +0200
Subject: [PATCH v20251126 4/9] clock-sweep: scan all partitions

 
 /*
 
 	/*
 
 	*from_ring = false;

 
 		/*

From d5e333e0300db911e499e9bf2a7db95ff46e674d Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 29 Oct 2025 21:45:34 +0100
Subject: [PATCH v20251126 3/9] clock-sweep: balancing of allocations

 .../pg_buffercache--1.7--1.8.sql              |   5 +-

diff --git a/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql b/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
index f702a9db1a8..1834599c4b3 100644
--- a/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
+++ b/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
@@ -19,7 +19,10 @@ CREATE VIEW pg_buffercache_partitions AS
 	 num_passes bigint,			-- clocksweep passes
 	 next_buffer integer,		-- next victim buffer for clocksweep
 	 total_allocs bigint,		-- handled allocs (running total)
-	 num_allocs bigint);		-- handled allocs (current cycle)
+	 num_allocs bigint,			-- handled allocs (current cycle)
+	 total_req_allocs bigint,	-- requested allocs (running total)
+	 num_req_allocs bigint,		-- handled allocs (current cycle)
+	 weights int[]);			-- balancing weights
 
index aed0ecb59e9..81665209084 100644

@@ -27,7 +29,7 @@
 
 #define NUM_BUFFERCACHE_OS_PAGES_ELEM	3
-#define NUM_BUFFERCACHE_PARTITIONS_ELEM	8
+#define NUM_BUFFERCACHE_PARTITIONS_ELEM	11
 
 PG_MODULE_MAGIC_EXT(
 					.name = "pg_buffercache",
@@ -844,6 +846,8 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 
 	if (SRF_IS_FIRSTCALL())
 	{
 
@@ -873,6 +877,12 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)

@@ -893,11 +903,17 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 					first_buffer,
 					last_buffer;

 		uint32		complete_passes,
 					next_victim_buffer,
-					buffer_allocs;
+					buffer_allocs,
+					buffer_req_allocs;
+
+		int		   *weights;
+		Datum	   *dweights;
+		ArrayType  *array;
 
 		Datum		values[NUM_BUFFERCACHE_PARTITIONS_ELEM];
 		bool		nulls[NUM_BUFFERCACHE_PARTITIONS_ELEM];
@@ -906,8 +922,16 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 						   &first_buffer, &last_buffer);

 		values[0] = Int32GetDatum(i);
 		nulls[0] = false;
@@ -933,6 +957,15 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		values[7] = Int64GetDatum(buffer_allocs);
 		nulls[7] = false;

 

+/*
 
+	/*
 
+	/*
 
 /*
 
 /*
 
 	/*

 /*
 
 	/*
 
+/*
+ * StrategySyncBalance

 
 void

 

 
 
 /* inline functions */
 
-- 
2.51.1

From 670ef95cfbec187fbc3e2cce641dec9bd79a092e Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 11 Nov 2025 12:03:32 +0100
Subject: [PATCH v20251126 2/9] clock-sweep: basic partitioning

 .../pg_buffercache--1.7--1.8.sql              |   8 +-

diff --git a/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql b/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
index d62b8339bfc..f702a9db1a8 100644
--- a/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
+++ b/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
@@ -13,7 +13,13 @@ CREATE VIEW pg_buffercache_partitions AS
 	(partition integer,			-- partition index
 	 num_buffers integer,		-- number of buffers in the partition
 	 first_buffer integer,		-- first buffer of partition
-	 last_buffer integer);		-- last buffer of partition
+	 last_buffer integer,		-- last buffer of partition
+
+	 -- clocksweep counters
+	 num_passes bigint,			-- clocksweep passes
+	 next_buffer integer,		-- next victim buffer for clocksweep
+	 total_allocs bigint,		-- handled allocs (running total)
+	 num_allocs bigint);		-- handled allocs (current cycle)
 
index 8c89855192f..aed0ecb59e9 100644
 
 #define NUM_BUFFERCACHE_OS_PAGES_ELEM	3
-#define NUM_BUFFERCACHE_PARTITIONS_ELEM	4
+#define NUM_BUFFERCACHE_PARTITIONS_ELEM	8
 
 PG_MODULE_MAGIC_EXT(
@@ -865,6 +865,14 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)

@@ -885,12 +893,22 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 					first_buffer,
 					last_buffer;

 
 
@@ -903,6 +921,18 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		values[3] = Int32GetDatum(last_buffer);
 		nulls[3] = false;

 
 /*

 
 	/*

 

 	/*
 
 	/*

 

 #ifdef BGW_DEBUG

 /*
 
+#ifdef USE_LIBNUMA
+#include <sched.h>
+#endif
+
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
 #include "pgstat.h"

 /*
 
 	/*

 
 
 /*
 
 	/*

 
 		/*

 
 /*

+/*

 	if (complete_passes)
 	{
 
 		/*
 
-	if (num_buf_alloc)
-	{
 
 /*
@@ -380,6 +538,9 @@ Size
 StrategyShmemSize(void)
 {
 	Size		size = 0;
+	int			num_partitions;
+

 
 
+	int			num_partitions;
+
 
 	if (!found)

 
-		/* Clear statistics */

 	return true;
 }
+
+void

 

 /* inline functions */
 
index d3edff346a8..241f175e9da 100644

From abe7cf9f1314bdaae2361ad237ef859e00b69c07 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 17 Sep 2025 23:04:29 +0200
Subject: [PATCH v20251126 1/9] Infrastructure for partitioning shared buffers

Notes:

* This partitioning is independent of the partitions defined in
  lwlock.h, which defines 128 partitions to reduce lock conflict on the
  buffer mapping hashtable. The number of partitions introduced by this
  patch is expected to be much lower (a dozen or so).
---
 .../pg_buffercache--1.7--1.8.sql              |  23 +++
 contrib/pg_buffercache/pg_buffercache.control |   2 +-
 contrib/pg_buffercache/pg_buffercache_pages.c |  86 +++++++++++
 src/backend/storage/buffer/buf_init.c         | 144 +++++++++++++++++-
 src/include/storage/buf_internals.h           |   6 +
 src/include/storage/bufmgr.h                  |  19 +++
 src/tools/pgindent/typedefs.list              |   2 +
 7 files changed, 280 insertions(+), 2 deletions(-)
 create mode 100644 contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql

diff --git a/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql b/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
new file mode 100644
index 00000000000..d62b8339bfc
--- /dev/null
+++ b/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
@@ -0,0 +1,23 @@
+\echo Use "ALTER EXTENSION pg_buffercache UPDATE TO '1.8'" to load this file. \quit
index 11499550945..d2fa8ba53ba 100644
-default_version = '1.7'
+default_version = '1.8'
index ae1712fc93c..8c89855192f 100644
 
 #define NUM_BUFFERCACHE_OS_PAGES_ELEM	3
+#define NUM_BUFFERCACHE_PARTITIONS_ELEM	4
 
 PG_MODULE_MAGIC_EXT(
@@ -101,6 +102,7 @@ PG_FUNCTION_INFO_V1(pg_buffercache_usage_counts);

 /* Only need to touch memory once per backend process lifetime */
@@ -826,3 +828,87 @@ pg_buffercache_evict_all(PG_FUNCTION_ARGS)
 
 	PG_RETURN_DATUM(result);
 }
+
+/*

+/* *
 
 /*
  * Data Structures:

 
 
 /* in buf_init.c */

 
 
+/*
 
index dfcd619bfee..d3edff346a8 100644

---

From: Jakub Wartak <[email-masked]>
Date: Tue, 2 Dec 2025 13:26:33 +0100

> Rebased patch series attached.

Thanks. BTW still with the old patchset series, One additional thing
that I've found out related to interleave is that in
CreateAnonymousSegment() with the default check_debug='', we still
issue numa_interleave_memory(ptr..). It should be optional (this also
affects earlier calls too). Tiny patch attached.

> I think the MAP_POPULATE should be optional, enabled by GUC.

OK, but you mean it's a new option to debug_numa, right? (not some
separate) so debug_numa='prefault' then?

> etc.), the cost of buffer replacement is going to be fairly limited.

Right. OK, so I've got the seqconcurrentscans comparison done right,
that is when prewarmed and not naturally filled:

@master, 29GB/s mem bandwidth
latency average = 1255.572 ms
latency stddev = 417.162 ms
tps = 50.451925 (without initial connection time)

@v20251121 patchset, 41GB/s (~10GB/s per socket)
latency average = 719.931 ms
latency stddev = 14.874 ms
tps = 88.362091 (without initial connection time)

The main PMC difference seems to be much lower "backend cycles idle"
(51% master and vs 31% for the NUMA debug_numa="buffers,procs", so
less is waiting on memory, thus it gets that speedup and better IPC).

Anyway, the biggest gripe right now (at least to me) is reliable
benchmarking. Below runs are all apples and oranges comparisons (they
measure different stuff although looks the same initially)
- restart and just select pg_shmem_allocations_numa or prewarm puts
everything into 1 NUMA node with check_numa='', because of prefaulting
happening during select-view case
- restart and pgbench -i -s XX (same issue as above) then pgbench -
you get the same, everything on potential one NUMA node (because
pgbench prefaults just on one)
- restart and pgbench -c 64.. with debug_numa='' (off) MIGHT get
random NUMA layout, how's that is supposed to be deterministic? at
least with debug_numa='buffers' you get determinism..
- the shared_buffers size vs size of dataset read, the moment you
start doing something CPU intensive (or like calling syscalls just for
VFS cache), the benefit seems to disappear at least on my hardware

Anyway, depending on the scenario I could get varied results like
34tps .. 88tps here. The debug_numa='buffers,..' gives just assurance
of the proper layout of shared memory is there (one could even argue
that such performance deviations across runs are bug ;)).

> with so many clients (compared to the number of CPUs).

The only thing in my mind about deterioration of high-connection count
(AKA -c 1024 scenario) with pgprocs, would be related to the question
you raised in 0007 "Note: The scheduler may migrate the process to a
different CPU/node later. Maybe we should consider pinning the process
to the node?"

I think the answer is yes, so to fetch MyProc based on sched_getcpu()
and then maybe with additional numa_flags & new PROCS_PIN_NODE simply
numa_run_on_node(node)? I've tried this:

pgbench -c 1024 -j 64 -P 1 -T 30 -S -M prepared got:

@numa-empty-debug_numa         ~434k TPS, ~12k CPU migrations/second
@numa+buffers+pgproc           ~412k TPS, 7-8k CPU migrations/second
@numa+buffers+pgproc+pinnode   ~434k TPS, still with 7-8k CPU
migrations/second (so same)
but I've verified for the last one, with bpftrace on that
tracepoint:sched:sched_migrate_task did not performed node-to-node
process bounces anymore (it did for pgbench but not for postgres
itself with this numa_run_on_node())

[..]

[..]
Thanks!

[..]
> the read-only pgbench). So the question is - is this really worth it?

Are you thinking here about whole NUMA patchset or just clocksweep? I
think multiple clocksweep are just not shining because other
bottlenecks hammer the efficiency here. Andres talk about it exactly
here https://youtu.be/V75KpACdl6E?t=1990 (He mentions out of order
execution, I see btrees in reports as top#1). So maybe it's just too
early to see the results of this optimization?

As for classic readonly pgbench -S I still see roughly 1:8 local to
remote (!) DRAM access (1 <-> 3 sockets) even with those patches, so
potentially something could be improved in far future for sure (that
would require some memaddr monitoring for most remote DRAM misses <->
pg inter-shm ptr mapping; think of pg_shmem_allocations_numa with
local/remote counters or maybe just fallback to perf-c2c).

To sum up, IMHO I understand this $thread's NUMA implementation as:
- it's strictly a guard mechanism to get determinism (for most cases)
-- it fixes "imbalance"
- no performance boost for OLTP as such
- for analytics it could be win (in-memory workloads; well PG is not
fully built for this, but it could be one day/or already is with 3rd
party TAMs and extensions), and:
-- we can provide performance jump for seqconcurrentjobs or memory
fitting workloads (patchset does this already). Note: I think PG will
eventually get into such classes in the longer run, we are just ahead
with NUMA, but PG is without proper vectorized executor stuff.
-- we could further enhance PQ here: the leader and PQ workers would
stick to the same NUMA node with some affinity (the earlier thread
measurements for this  [1] -- we could have session GUC to enable this
for planned big PQ whole-NUMA SELECTs; this would be probably done
close to dsm_impl_posix())
- new idea: we could allow exposing tables(spaces) into NUMA nodes or
make it per-user toggle too while we are at it (imagine HTAP-like
workloads: NUMA node #0 for OLTP, node #1 for analytics). Sounds cool
and rather easy and has valid use, but dunno if that would be really
useful?

Way out of scope:
- superlocking btress that Andres mentioned on his presentation

-J.

[1] - https://www.postgresql.org/message-id/attachment/178120/NUMA_pq_cpu_pinning_results.txt
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 6019bee334d..e5c4752d9f6 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -30,6 +30,7 @@
 
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
+#include "port/pg_numa.h"
 #include "portability/mem.h"
 #include "storage/dsm.h"
 #include "storage/fd.h"
@@ -609,7 +610,8 @@ CreateAnonymousSegment(Size *size)
 	 *
 	 * XXX Probably not needed without that, but also costs nothing.
 	 */
-	numa_set_membind(numa_all_nodes_ptr);
+	if ((numa_flags & NUMA_BUFFERS) != 0)
+		numa_set_membind(numa_all_nodes_ptr);
 
 #ifndef MAP_HUGETLB
 	/* PGSharedMemoryCreate should have dealt with this case */
@@ -681,7 +683,8 @@ CreateAnonymousSegment(Size *size)
 	}
 
 	/* undo the earlier num_set_membind() call. */
-	numa_set_localalloc();
+	if ((numa_flags & NUMA_BUFFERS) != 0)
+		numa_set_localalloc();
 
 	/*
 	 * Before touching the memory, set the allocation policy, so that
@@ -693,7 +696,8 @@ CreateAnonymousSegment(Size *size)
 	 * XXX Probably not needed with MAP_POPULATE, in which case the policy
 	 * was already set by num_set_membind() earlier. But doesn't hurt.
 	 */
-	numa_interleave_memory(ptr, allocsize, numa_all_nodes_ptr);
+	if ((numa_flags & NUMA_BUFFERS) != 0)
+		numa_interleave_memory(ptr, allocsize, numa_all_nodes_ptr);
 
 	*size = allocsize;
 	return ptr;

---

From: Tomas Vondra <[email-masked]>
Date: Mon, 8 Dec 2025 21:02:27 +0100

Hi,

I've spent the last couple days considering what to do about this patch
series in this thread. The approach assumes partitioning shared memory
in a NUMA-aware way has enough benefits to justify the extra complexity.
I'm getting somewhat skeptical about that being a good trade off.

We've been able to demonstrate some significant benefits of the patches.
See for example the results from about a month ago [1], showing that in
some cases the throughput almost doubles.

But I'm skeptical about this for two reasons:

* Most of the benefit comes from patches unrelated to NUMA. The initial
patches partition clockweep, in a NUMA oblivious way. In fact, applying
the NUMA patches often *reduces* the throughput. So if we're concerned
about contention on the clocksweep hand, we could apply just these first
patches. That way we wouldn't have to deal with huge pages.

* Furthermore, I'm not quite sure clocksweep really is a bottleneck in
realistic cases. The benchmark used in this thread does many concurrent
sequential scans, on data that exceeds shared buffers / fits into RAM.
Perhaps that happens, but I doubt it's all that common.

I've been unable to demonstrate any benefits on other workloads, even if
there's a lot of buffer misses / reads into shared buffers. As soon as
the query starts doing something else, the clocksweep contention becomes
a non-issue. Consider for example read-only pgbench with database much
larger than shared buffers (but still within RAM). The cost of the index
scans (and other nodes) seems to reduce the pressure on clocksweep.

So I'm skeptical about clocksweep pressure being a serious issue, except
for some very narrow benchmarks (like the concurrent seqscan test). And
even if this happened for some realistic cases, partitioning the buffers
in a NUMA-oblivious way seems to do the trick.

When discussing this stuff off list, it was suggested this might help
with the scenario Andres presented in [3], where the throughput improves
a lot with multiple databases. I've not observed that in practice, and I
don't think these patches really can help with that. That scenario is
about buffer lock contention, not clocksweep contention.

With a single database there's nothing to do - there's one contended
page, located on a single node. There'll be contention, the no matter
which node it ends up on. With multiple databases (or multiple root
pages), it either happens to work by chance (if the buffers happen to be
from different nodes), or it would require figuring out which buffers
are busy, and place them on different nodes. But the patches did not
even try to do anything like that. So it still was a matter of chance.

That does not mean we don't need to worry about NUMA. There's still the
issue of misbalancing the allocation - with memory coming from just one
node, etc. Which is an issue because it "overloads" the memory subsystem
on that particular node. But that can be solved simply by interleaving
the shared memory segment.

That sidesteps most of the complexity - it does not need to figure out
how to partition buffers, does not need to worry about huge pages, etc.
This is not a new idea - it's more or less what Jakub Wartak initially
proposed in [2], before I hijacked the thread into the more complex
approach.

Attached is a tiny patch doing mostly what Jakub did, except that it
does two things. First, it allows interleaving the shared memory on all
relevant NUMA nodes (per numa_get_mems_allowed). Second, it allows
populating all memory by setting MAP_POPULATE in mmap(). There's a new
GUC to enable each of these.

I think we should try this (much simpler) approach first, or something
close to it. Sorry for dragging everyone into a much more complex
approach, which now seems to be a dead end.

regards

[1]
https://www.postgresql.org/message-id/[email-masked]

[2]
https://www.postgresql.org/message-id/[email-masked]

[3] https://youtu.be/V75KpACdl6E?t=2120
-- 
Tomas Vondra
From 72b0a9b86da7e1a6e239f5fdead24aeaa3d99f27 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Fri, 5 Dec 2025 14:53:47 +0100
Subject: [PATCH v20251208] numa: Simple interleaving and MAP_POPULATE

Allows NUMA interleaving on the shared memory segment, to ensure memory
is balanced between the NUMA nodes. The patch also allows prefaulting
the shared memory (by setting MAP_POPULATE flag), to actually allocate
the pages on nodes.

The commit addes two GUC parameters, both set to 'off' by default:

- shared_memory_interleave (enables NUMA interleaving)

- shared_memory_populate (sets MAP_POPULATE)

The memory is interleaved on all nodes enabled in the cpuset, as
returned by numa_get_mems_allowed(). The interleaving is applied at the
memory page granularity, and is oblivious to what's stored in it.
---
 src/backend/port/sysv_shmem.c                 | 51 ++++++++++++++++++-
 src/backend/utils/misc/guc_parameters.dat     | 12 +++++
 src/backend/utils/misc/guc_tables.c           |  2 +
 src/backend/utils/misc/postgresql.conf.sample |  2 +
 src/include/port/pg_numa.h                    |  4 ++
 src/include/storage/pg_shmem.h                |  3 ++
 src/port/pg_numa.c                            | 42 +++++++++++++++
 7 files changed, 115 insertions(+), 1 deletion(-)

diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 298ceb3e218..b300f5ef4a7 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -19,6 +19,7 @@
  */
 #include "postgres.h"
 
+#include <numa.h>
 #include <signal.h>
 #include <unistd.h>
 #include <sys/file.h>
@@ -29,6 +30,7 @@
 
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
+#include "port/pg_numa.h"
 #include "portability/mem.h"
 #include "storage/dsm.h"
 #include "storage/fd.h"
@@ -602,6 +604,28 @@ CreateAnonymousSegment(Size *size)
 	void	   *ptr = MAP_FAILED;
 	int			mmap_errno = 0;
 
+	/*
+	 * When asked to use NUMA-interleaving, we need to set the policy before
+	 * touching the memory. By default that'll happen later, when either
+	 * initializing some of the shmem structures (e.g. buffer descriptors), or
+	 * when running queries. In that case it's enough to set the policy after
+	 * the mmap() call, and we don't need to do anything here.
+	 *
+	 * With MAP_POPULATE, the mmap() itself will prefault the pages, so we
+	 * need to set the policy to interleave before the mmap() call, and then
+	 * revert to localalloc (so that private memory is allocated locally).
+	 *
+	 * XXX It probably is not a good idea to enable interleaving with regular
+	 * memory pages, because then each buffer will get split on two nodes, and
+	 * the system won't be able to fix that by migrating one of the pages. But
+	 * we leave that up to the admin, instead of forbidding it.
+	 */
+	if (shared_memory_interleave && shared_memory_populate)
+	{
+		/* set the allocation to interleave on nodes allowed by the cpuset */
+		pg_numa_set_interleave();
+	}
+
 #ifndef MAP_HUGETLB
 	/* PGSharedMemoryCreate should have dealt with this case */
 	Assert(huge_pages != HUGE_PAGES_ON);
@@ -619,6 +643,10 @@ CreateAnonymousSegment(Size *size)
 		if (allocsize % hugepagesize != 0)
 			allocsize += hugepagesize - (allocsize % hugepagesize);
 
+		/* populate the shared memory if requested */
+		if (shared_memory_populate)
+			mmap_flags |= MAP_POPULATE;
+
 		ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
 				   PG_MMAP_FLAGS | mmap_flags, -1, 0);
 		mmap_errno = errno;
@@ -638,13 +666,19 @@ CreateAnonymousSegment(Size *size)
 
 	if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
 	{
+		int			mmap_flags = 0;
+
+		/* populate the shared memory if requested */
+		if (shared_memory_populate)
+			mmap_flags |= MAP_POPULATE;
+
 		/*
 		 * Use the original size, not the rounded-up value, when falling back
 		 * to non-huge pages.
 		 */
 		allocsize = *size;
 		ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
-				   PG_MMAP_FLAGS, -1, 0);
+				   PG_MMAP_FLAGS | mmap_flags, -1, 0);
 		mmap_errno = errno;
 	}
 
@@ -663,6 +697,21 @@ CreateAnonymousSegment(Size *size)
 						 allocsize) : 0));
 	}
 
+	/*
+	 * With NUMA interleaving, we need to either apply interleaving for the
+	 * shmem segment we just allocated, or reset the memory policy to local
+	 * allocation (when using MAP_POPULATE).
+	 */
+	if (shared_memory_interleave)
+	{
+		if (shared_memory_populate)
+			/* revert back to using the local node */
+			pg_numa_set_localalloc();
+		else
+			/* apply interleaving to the new memory segment */
+			pg_numa_interleave_memory(ptr, allocsize);
+	}
+
 	*size = allocsize;
 	return ptr;
 }
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 3b9d8349078..ce1c0c4327f 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -2597,6 +2597,18 @@
   max => 'INT_MAX / 2',
 },
 
+{ name => 'shared_memory_interleave', type => 'bool', context => 'PGC_POSTMASTER', group => 'RESOURCES_MEM',
+  short_desc => 'Enables NUMA interleaving of shared memory.',
+  variable => 'shared_memory_interleave',
+  boot_val => 'false',
+},
+
+{ name => 'shared_memory_populate', type => 'bool', context => 'PGC_POSTMASTER', group => 'RESOURCES_MEM',
+  short_desc => 'Populates shared memory at start.',
+  variable => 'shared_memory_populate',
+  boot_val => 'false',
+},
+
 { name => 'shared_memory_size', type => 'int', context => 'PGC_INTERNAL', group => 'PRESET_OPTIONS',
   short_desc => 'Shows the size of the server\'s main shared memory area (rounded up to the nearest MB).',
   flags => 'GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE | GUC_UNIT_MB | GUC_RUNTIME_COMPUTED',
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f87b558c2c6..cfee0df987f 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -580,6 +580,8 @@ static int	ssl_renegotiation_limit;
 int			huge_pages = HUGE_PAGES_TRY;
 int			huge_page_size;
 int			huge_pages_status = HUGE_PAGES_UNKNOWN;
+bool		shared_memory_interleave = false;
+bool		shared_memory_populate = false;
 
 /*
  * These variables are all dummies that don't do anything, except in some
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index dc9e2255f8a..de1276f6897 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -152,6 +152,8 @@
                                         #   sysv
                                         #   windows
                                         # (change requires restart)
+#shared_memory_interleave = off         # interleave all memory on available NUMA nodes
+#shared_memory_populate = off           # prefault shared memory on start
 #dynamic_shared_memory_type = posix     # the default is usually the first option
                                         # supported by the operating system:
                                         #   posix
diff --git a/src/include/port/pg_numa.h b/src/include/port/pg_numa.h
index 9d1ea6d0db8..dc9f13d9fa0 100644
--- a/src/include/port/pg_numa.h
+++ b/src/include/port/pg_numa.h
@@ -18,6 +18,10 @@ extern PGDLLIMPORT int pg_numa_init(void);
 extern PGDLLIMPORT int pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status);
 extern PGDLLIMPORT int pg_numa_get_max_node(void);
 
+extern PGDLLIMPORT void pg_numa_set_interleave(void);
+extern PGDLLIMPORT void pg_numa_set_localalloc(void);
+extern PGDLLIMPORT void pg_numa_interleave_memory(void *ptr, Size size);
+
 #ifdef USE_LIBNUMA
 
 /*
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 5f7d4b83a60..7b56bd5b44f 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -47,6 +47,9 @@ extern PGDLLIMPORT int huge_pages;
 extern PGDLLIMPORT int huge_page_size;
 extern PGDLLIMPORT int huge_pages_status;
 
+extern PGDLLIMPORT bool shared_memory_interleave;
+extern PGDLLIMPORT bool shared_memory_populate;
+
 /* Possible values for huge_pages and huge_pages_status */
 typedef enum
 {
diff --git a/src/port/pg_numa.c b/src/port/pg_numa.c
index 540ada3f8ef..a91d339033f 100644
--- a/src/port/pg_numa.c
+++ b/src/port/pg_numa.c
@@ -116,6 +116,33 @@ pg_numa_get_max_node(void)
 	return numa_max_node();
 }
 
+/*
+ * Set allocation memory to interleave on all memory nodes in the cpuset.
+ */
+void
+pg_numa_set_interleave(void)
+{
+	numa_set_membind(numa_get_mems_allowed());
+}
+
+/*
+ * Set allocation memory to localalloc.
+ */
+void
+pg_numa_set_localalloc(void)
+{
+	numa_set_localalloc();
+}
+
+/*
+ * Set policy for memory to interleaving (on all nodes per cpuset).
+ */
+void
+pg_numa_interleave_memory(void *ptr, Size size)
+{
+	numa_interleave_memory(ptr, size, numa_get_mems_allowed());
+}
+
 #else
 
 /* Empty wrappers */
@@ -138,4 +165,19 @@ pg_numa_get_max_node(void)
 	return 0;
 }
 
+void
+pg_numa_set_interleave(void)
+{
+}
+
+void
+pg_numa_set_localalloc(void)
+{
+}
+
+void
+pg_numa_interleave_memory(void *ptr, Size size)
+{
+}
+
 #endif
-- 
2.51.1

---

From: Tomas Vondra <[email-masked]>
Date: Fri, 5 Jun 2026 14:52:35 +0200

Hi,

Here's an updated version of the NUMA patch series, based on some recent
discussions about this (some at pgconf.dev, but not only that),

The main change is I significantly simplified some of the parts. Whe
patch from 20251126 was ~190K, the new version is maybe 100K, so about
half. Some of that is thanks to dropping the PGPROC partitioning
entirely, but the remaining parts are smaller too. I realize it's not a
great metric, of course.

In this message I'll explain the changes since 20251126. I'm yet to do a
thorough performance evaluation and see if it helps, I'll post that in
the next couple days.

The current patch series has these parts:

v20260605-0001-Add-shmem_populate-and-shmem_interleave-GU.patch
-------------

Somewhat unrelated, I find this useful for benchmarking and as a
baseline (what would happen if we just interleaved the shared segment).

v20260605-0002-Infrastructure-for-partitioning-of-shared-.patch
-------------

Just adds a small registry of partitions (ranges of shared buffers),
stored in shared memory, and pg_buffercache interface to inspect it.
Merely a foundation for the following patches.

v20260605-0003-NUMA-shared-buffers-partitioning.patch
-------------

The interesting part, that places some of the partitions to NUMA nodes.

v20260605-0004-clock-sweep-basic-partitioning.patch
v20260605-0005-clock-sweep-balancing-of-allocations.patch
v20260605-0006-clock-sweep-scan-all-partitions.patch
-------------

Patches that gradually partition clock-sweep. Ultimately, it should
probably be squashed into a single commit (each commit fixes some sort
of issue in the naive partitioning in 0004). But I kept them separate
because it's easier to review / understand what the issue is.

what changed?
-------------

First, I dropped the PGPROC partitioning. We may revisit that in the
future (not sure), but for now it was just a distraction and I see it as
less impactful than shared buffers / clock-sweep.

I also simplified the GUC to use a single on/off parameter (instead of
the debug_io_direct-like approach). We can revisit that, but for now
this seems more convenient.

The most significant change in the remaining parts is simplification of
the shared buffer partitioning. In particular, the partitioning is now
"best-effort" when it maps memory to shared buffers. Let me remind that
NUMA works at memory page granularity - we can't map arbitrary ranges of
memory to a node, it needs to be whole memory pages.

The 20251126 patch went into great lengths to (a) make sure BufferBlocks
and BufferDescriptors start at memory page boundary, are the partitions
are also properly aligned (both for blocks and descriptors). That was a
lot of code, it needs to happen even before we know if huge pages are
used, partitions might have been of (very) different sizes, and so on.

The new patch abandons this "perfect" partitioning, and instead does a
best-effort. It splits the buffers as evenly as possible, i.e. all
partitions have (NBuffers/npartitions) buffers, and then locates as much
memory as possible to a selected NUMA node.

With 4K pages, that's always the whole partition. With huge pages (which
is expected of relevant NUMA systems), there may be a couple buffers at
the beginning/end of a partition. But it's less than one memory page,
per partition, and we expect the systems to have 10s or 100s of GBs, so
in the bigger scheme of things it's negligible (fractions of a percent).

For buffer descriptors the math is a bit worse - descriptors need much
less memory, but even there it should not be more than ~1%.

Seems perfectly fine to me. Or rather, the extra complexity does not
seem worth the possible benefit.

This also allowed dropping a part of the "clock-sweep partitioning"
patches, dealing with cases when the partitions are of different sizes.
With this new best-effort scheme the difference is at most 1 buffer, and
we can just ignore that.

questions
---------

At this point, my main question is whether there's a better way to
partition clock-sweep and/or do the balancing of allocations between
partitions. I believe it does work, but I have a feeling there might be
a more elegant way to do this kind of stuff (like an established
balancing algorithm of some sort).

The other thing I need to verify is how this behaves with
kernel.nr_hugepages. With some earlier versions it was easy to end in a
situation where everything seemed to work, but then much later the
kernel realized it does not have enough huge pages on a particular NUMA
node and crashed with a segfault (or was it sigbus?).

Of course, the other question is performance validation - does it even
help? I plan to repeat the various experiments mentioned in this thread
(by Andres and others) on available NUMA machines. But if someone has an
idea for another benchmark (and/or what metric to measure, not just the
usual duration), let me know.

regards

-- 
Tomas Vondra
From 977831d85199b4b1b8d65de1131e4001459878fb Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 2 Jun 2026 22:39:38 +0200
Subject: [PATCH v20260605 6/6] clock-sweep: scan all partitions

When looking for a free buffer, scan all clock-sweep partitions, not
just the "home" one. All buffers in the home partition may be pinned, in
which case we should not fail. Instead, advance to the next partition,
in a round-robin way, and only fail after scanning through all of them.
---
 src/backend/storage/buffer/freelist.c     | 83 ++++++++++++++++-------
 src/test/recovery/t/027_stream_regress.pl |  5 --
 2 files changed, 57 insertions(+), 31 deletions(-)

diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index a543fb12b21..1ac1e3e3490 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -184,6 +184,9 @@ static BufferDesc *GetBufferFromRing(BufferAccessStrategy strategy,
 static void AddBufferToRing(BufferAccessStrategy strategy,
 							BufferDesc *buf);
 static ClockSweep *ChooseClockSweep(bool balance);
+static BufferDesc *StrategyGetBufferPartition(ClockSweep *sweep,
+											  BufferAccessStrategy strategy,
+											  uint64 *buf_state);
 
 /*
@@ -251,10 +254,9 @@ static int clocksweep_partition_budget = 0;
 
 	/*
 	 * Atomically move hand ahead one buffer - if there's several processes
@@ -486,7 +488,8 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint64 *buf_state, bool *from_r
 {
 	BufferDesc *buf;
 	int			bgwprocno;
-	int			trycounter;
+	ClockSweep *sweep,
+			   *sweep_start;		/* starting clock-sweep partition */
 
 	*from_ring = false;
 
@@ -545,33 +548,61 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint64 *buf_state, bool *from_r
 	/*
-	 * a process "sweeps" only a fraction of buffers, even if the other buffers
-	 * are better candidates for eviction. Maybe there should be some logic to
-	 * "steal" buffers from other partitions or other nodes?
-	 * partitions. Needs to scan partitions if needed, as a fallback.
-	 * XXX Would that also mean we should have multiple bgwriters, one for each
-	 * node, or would one bgwriter still handle all nodes?
+						   uint64 *buf_state)
 		uint64		old_buf_state;
 		uint64		local_buf_state;

 		/*
 		 * Check whether the buffer can be used and pin it if so. Do this
@@ -599,7 +630,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint64 *buf_state, bool *from_r
 					 * one eventually, but it's probably better to fail than
 					 * to risk getting stuck in an infinite loop.
 					 */
-					elog(ERROR, "no unpinned buffers available");
+					return NULL;
 				}
 				break;
 			}
@@ -618,7 +649,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint64 *buf_state, bool *from_r
 				if (pg_atomic_compare_exchange_u64(&buf->state, &old_buf_state,
 												   local_buf_state))
 				{
-					trycounter = NBuffers;
+					trycounter = sweep->numBuffers;
 					break;
 				}
 			}
diff --git a/src/test/recovery/t/027_stream_regress.pl b/src/test/recovery/t/027_stream_regress.pl
index f68e08e5697..ae977297849 100644
--- a/src/test/recovery/t/027_stream_regress.pl
+++ b/src/test/recovery/t/027_stream_regress.pl
@@ -18,11 +18,6 @@ $node_primary->adjust_conf('postgresql.conf', 'max_connections', '25');
 $node_primary->append_conf('postgresql.conf',
 	'max_prepared_transactions = 10');
 
2.54.0

From ad46b08a3eaa018b562fdd4ca786c4c23d358ba5 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 2 Jun 2026 22:28:10 +0200
Subject: [PATCH v20260605 5/6] clock-sweep: balancing of allocations

 src/backend/storage/buffer/freelist.c         | 428 +++++++++++++++++-
 6 files changed, 471 insertions(+), 21 deletions(-)

index 92176fed7f8..43d2e84f9d2 100644
@@ -20,7 +20,10 @@ CREATE VIEW pg_buffercache_partitions AS
 
index 739c63b0cfc..c91f2bc5b4a 100644
 #include "utils/tuplestore.h"
 
@@ -31,7 +33,7 @@
 #define NUM_BUFFERCACHE_MARK_DIRTY_ALL_ELEM 3

 PG_MODULE_MAGIC_EXT(
 					.name = "pg_buffercache",
@@ -889,6 +891,8 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 
 	if (SRF_IS_FIRSTCALL())
 	{
 
@@ -920,6 +924,12 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)

@@ -941,11 +951,17 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 					first_buffer,
 					last_buffer;

 		uint32		complete_passes,
 					next_victim_buffer,
-					buffer_allocs;
+					buffer_allocs,
+					buffer_req_allocs;
+
+		int		   *weights;
+		Datum	   *dweights;
+		ArrayType  *array;
 
 		Datum		values[NUM_BUFFERCACHE_PARTITIONS_ELEM];
 		bool		nulls[NUM_BUFFERCACHE_PARTITIONS_ELEM];
@@ -954,8 +970,16 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 						   &first_buffer, &last_buffer);

 		values[0] = Int32GetDatum(i);
 		nulls[0] = false;
@@ -984,6 +1008,15 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		values[8] = Int64GetDatum(buffer_allocs);
 		nulls[8] = false;
 
index 02c75f82e5b..62e541abebd 100644
@@ -4135,6 +4135,9 @@ BgBufferSync(WritebackContext *wb_context)

index 2d56579682e..a543fb12b21 100644
@@ -35,6 +35,26 @@

+/*
+ * XXX We need to make ClockSweep fixed-size, so that we can have an array
+ * in shared memory. The easiest way is to pick a sufficiently high value
+ * that no system will actually need. 32 seems high enough.
+ *
+ * XXX We should enforce this in bufmgr.c, when initializing the partitions.
+ */
+#define MAX_BUFFER_PARTITIONS		32
+
+/*
+ * Coefficient used to combine the old and new balance coefficients, using
+ * weighted average, so that we don't flap too much. The higher the value, the
+ * more the old value affects the result.
+ *
+ * XXX Doesn't this obscure the interpretation of weights as probabilities to
+ * allocate from a given partition? Does it still sum to 100%? I don't think
+ * so, it's just a fraction of allocations to go from a given partition.
+ */
+#define CLOCKSWEEP_HISTORY_COEFF	0.5
+
 /*
  * Information about one partition of the ClockSweep (on a subset of buffers).
  *
@@ -68,9 +88,32 @@ typedef struct
 	uint32		completePasses; /* Complete cycles of the clock-sweep */
 	pg_atomic_uint32 numBufferAllocs;	/* Buffers allocated since last reset */
 
+	/*
 
+	/*
+	 * Weights to balance buffer allocations for all the partitions. Each
+	 * partition gets a vector of weights 0-100, determining what fraction
+	 * of buffers to allocate from that partition. So [75, 15, 5, 5] would
+	 * mean 75% allocations should go from partition 0, 15% from partition
+	 * 1, and 5% from partitions 2&3. Each partition gets a different vector
+	 * of weights.
+	 *
+	 * Backends use the budget from it's "home" partition, so that a busy
+	 * partitions (with a lot of processes on that NUMA node etc.) spread
+	 * the allocations evenly.
+	 *
+	 * XXX Allocate a fixed-length array, to simplify working with array of
+	 * the structs, etc.
+	 */
+	uint8		balance[MAX_BUFFER_PARTITIONS];
 } ClockSweep;
 
 /*
@@ -140,7 +183,66 @@ static BufferDesc *GetBufferFromRing(BufferAccessStrategy strategy,
 									 uint64 *buf_state);
 static void AddBufferToRing(BufferAccessStrategy strategy,
 							BufferDesc *buf);
-static ClockSweep *ChooseClockSweep(void);
+static ClockSweep *ChooseClockSweep(bool balance);
+
+/*
+ * clocksweep allocation balancing
+ *
+ * To balance allocations from clocksweep partitions, each partition gets a
+ * budget for allocating buffers from other partitions. A process that
+ * "exhausts" a budget in it's home partition gets redirected to the other
+ * partitions, driven by the budgets.
+ *
+ * For example, a partition may have budget [25, 25, 25, 25], which means
+ * each of the 4 partitions should get 1/4 of allocations. Or the buget
+ * can be [50, 50, 0, 0], which means all allocations will go to the first
+ * two partitions (one of them being the "home" one);
+ *
+ * We could do that based on a random number generator, but for now we
+ * simply treat the values as a budget, i.e. a number of allocations to
+ * serve from other partitions, and move in round-robin way.
+ *
+ * This is very simple/cheap, and over many allocations it has the same
+ * effect. For periods of low activity it may diverge, but that does not
+ * matter much (we care about high-activity periods much more).
+ *
+ * We intentionally keep the "budget" fairly low, with the sum for a given
+ * partition 100. That means we get to the same partition after only 100
+ * allocations, keeping it more balanced. We can make the budgets higher
+ * (say, to match the expected number of allocations, i.e. bout the average
+ * number of allocations from the past interval). Or maybe configurable.
+ *
+ * XXX We should always start allocating from the "home" partition, i.e.
+ * from from it, and only then redirect to other partitions.
+ *
+ * XXX It probably is not great all the processes from that "home"
+ * partition are coordinated, and move to between partitions at about the
+ * same time. Not sure what to do about this.
+ *
+ * XXX We should also prefer other partitions from the same NUMA node (if
+ * there are some). Probably by setting the budgets.
+ *
+ * FIXME Explain at which point are the budgets recalculated, by which
+ * process, and how that affects other processes allocating buffers.
+ */
+
+/*
+ * The "optimal" clock-sweep partition. After a backend gets moved to a
+ * different NUMA node, we restart the balancing so that it uses the
+ * correct "budget" from the new home partition.
+ */
+static int clocksweep_partition_home = -1;
+
+/*
+ * The partition the backend is currently allocating from (either the
+ * home one, or one of the redirected ones).
+ */
+static int clocksweep_partition_current = -1;
+
+/*
+ * The number of buffers to allocate from the current partition.
+ */
+static int clocksweep_partition_budget = 0;
 
 /*
  * ClockSweepTick - Helper routine for StrategyGetBuffer()
@@ -152,7 +254,7 @@ static inline uint32
 ClockSweepTick(void)
 {
 	uint32		victim;
-	ClockSweep *sweep = ChooseClockSweep();
+	ClockSweep *sweep = ChooseClockSweep(true);
 
 	/*
 	 * Atomically move hand ahead one buffer - if there's several processes
@@ -300,11 +402,68 @@ ClockSweepPartitionIndex(void)
  * and that's cheaper. But how would that deal with odd number of nodes?
  */
 static ClockSweep *
-ChooseClockSweep(void)
+ChooseClockSweep(bool balance)
 {
+	/* What's the "optimal" partition for this backend? */
 	int			index = ClockSweepPartitionIndex();
+	ClockSweep *sweep = &StrategyControl->sweeps[index];
+
+	/*
+	 * Was the process migrated to a different NUMA node? If the home partition
+	 * changed, we need to reset the budget and start over, so that we correctly
+	 * prefer "nearby" partitions etc.
+	 *
+	 * XXX Could this be a problem when processes move all the time? I don't
+	 * think so - if a process moves between many partitions, that alone will
+	 * spread the allocations over partitions. Similarly, if there are many
+	 * processes, that should make it even more even.
+	 */
+	if (clocksweep_partition_home != index)
+	{
+		clocksweep_partition_home = index;
+		clocksweep_partition_current = index;
+		clocksweep_partition_budget = sweep->balance[index];
+	}
+
+	/* we should have a valid partition */
+	Assert(clocksweep_partition_home != -1);
+	Assert(clocksweep_partition_current != -1);
+	Assert(clocksweep_partition_budget >= 0);
+
+	/*
+	 * When balancing allocations, redirect the allocations to other partitions
+	 * according to the budgets. We move through partitions in a round-robin way,
+	 * after allocating the "budget" of allocations from the current one.
+	 */
+	if (balance)
+	{
+		/*
+		 * Ran out of budget from the current partition? Move to the next one
+		 * with non-zero budget.
+		 */
+		while (clocksweep_partition_budget == 0)
+		{
+			/* wrap around at the end */
+			clocksweep_partition_current++;
+			if (clocksweep_partition_current >= StrategyControl->num_partitions)
+				clocksweep_partition_current = 0;
+
+			clocksweep_partition_budget
+				= sweep->balance[clocksweep_partition_current];
+		}
 
-	return &StrategyControl->sweeps[index];
+		/* account for the current allocation */
+		--clocksweep_partition_budget;
+
+		/*
+		 * Account for the allocation in the "home" partition, so that the next
+		 * round of rebalancing (recalculating the budgets) knows about the
+		 * allocation traffic in various partitions.
+		 */
+		pg_atomic_fetch_add_u32(&sweep->numRequestedAllocs, 1);
+	}
+
+	return &StrategyControl->sweeps[clocksweep_partition_current];
 }
 
 /*
@@ -381,7 +540,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint64 *buf_state, bool *from_r
 	 * between CPUs / NUMA nodes in between, these call may pick different
 	 * partitions, confusing the logic a bit.
 	 */
-	pg_atomic_fetch_add_u32(&ChooseClockSweep()->numBufferAllocs, 1);
+	pg_atomic_fetch_add_u32(&ChooseClockSweep(false)->numBufferAllocs, 1);
 
 	/*
 	 * Use the "clock sweep" algorithm to find a free buffer
@@ -485,6 +644,229 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint64 *buf_state, bool *from_r
 	}
 }
 
+/*
+ * StrategySyncBalance
+ *		update partition budgets, to balance the buffer allocations
+ *
+ * We want to give preference to allocating buffers on the same NUMA node,
+ * but that might lead to imbalance - a single process would only use a
+ * fraction of shared buffers. We don't want that, we want to utilize the
+ * whole shared buffers. The number of allocations in each partition may
+ * also change over time, so we need to adapt to that.
+ *
+ * To allow this "adaptive balancing", each partition has a set of weights,
+ * determining what fraction of allocations to direct to other partitions.
+ * For simplicity the coefficients are integers 0-100, expressing the
+ * percentage of allocations redirected to that partition.
+ *
+ * Consider for example weights [50, 25, 25, 0] for one of 4 partitions.
+ * This means 50% of allocations will be redirected to partition 0, 25%
+ * to partitions 1 and 2, and no allocations will go to partition 3.
+ *
+ * This means an allocation may be requested in partition A (i.e. the
+ * home partition of the process requesting it), but end up allocating
+ * the buffer in partition B. We have a counter for both - the number of
+ * allocations requested in a partition, and the number of allocations
+ * actually handled by that partition. The former is used for calculating
+ * weights, the latter is used only for monitoring.
+ *
+ * The balancing happens in intervals - it adjusts future allocations
+ * based on stats about recent allocations, namely:
+ *
+ * - numBufferAllocs - number of allocations served by a partition
+ *
+ * - numRequestedAllocs - number of allocatios requested in a partition
+ *
+ * We're trying to smooth numBufferAllocs in the next interval, based on
+ * numRequestedAllocs measured in the last interval.
+ *
+ * The balancing algorithm works like this:
+ *
+ * - the target (average number of allocations per partition) is calculated
+ *   from total number of allocations requested in the last intervaal
+ *
+ * - partitions get divided into two groups - those with more allocation
+ *   requests than the target, and those with fewer requests
+ *
+ * - we "distribute" the delta (which is the same between the groups)
+ *   between the groups (one has more, the other fewer)
+ *
+ * Partitions with (nallocs > avg_nallocs) redirect the extra allocations,
+ * with each target allocation getting a proportional part (with respect
+ * to the total delta).
+ *
+ * XXX Currently this does not give preference to other partitions on the
+ * same NUMA node (redirect to it first), but it could.
+ */
+void
+StrategySyncBalance(void)
+{
+	/* snapshot of allocation requests for partitions */
+	uint32	allocs[MAX_BUFFER_PARTITIONS];
+
+	uint32	total_allocs = 0,	/* total number of allocations */
+			avg_allocs,			/* average allocations (per partition) */
+			delta_allocs = 0;	/* sum of allocs above average */
+
+	/*
+	 * Collect the number of allocations requested in the past interval.
+	 * While at it, reset the counter to start the new interval.
+	 *
+	 * XXX We lock the partitions one by one, so this is not a perfectly
+	 * consistent snapshot of the counts, and the resets happen before we
+	 * update the weights too. But we're only looking for heuristics, so
+	 * this should be good enough.
+	 *
+	 * XXX A similar issue applies to the counter reset later - we haven't
+	 * updated the weights yet, so some of the requests counted for the next
+	 * interval will be redirected per current weights. Should be fine, it's
+	 * just an approximate heuristics, and there should be very few requests in
+	 * between. Alternatively, we could reset the request counters when setting
+	 * the new weights, and just ignore the couple requests in between.
+	 *
+	 * XXX Does this need to worry about the completePasses too?
+	 */
+	for (int i = 0; i < StrategyControl->num_partitions; i++)
+	{
+		ClockSweep *sweep = &StrategyControl->sweeps[i];
+
+		/* no need for a spinlock */
+		allocs[i] = pg_atomic_exchange_u32(&sweep->numRequestedAllocs, 0);
+
+		/* add the allocs to running total */
+		pg_atomic_fetch_add_u64(&sweep->numTotalRequestedAllocs, allocs[i]);
+
+		total_allocs += allocs[i];
+	}
+
+	/* Calculate the "fair share" of allocations per partition. */
+	avg_allocs = (total_allocs / StrategyControl->num_partitions);
+
+	/*
+	 * Calculate the "delta" from balanced state for each partition, i.e. how
+	 * many more/fewer allocations it handled relative to the average.
+	 */
+	for (int i = 0; i < StrategyControl->num_partitions; i++)
+	{
+		if (allocs[i] > avg_allocs)
+			delta_allocs += (allocs[i] - avg_allocs);
+	}
+
+	/*
+	 * Skip rebalancing when there's not enough activity, and just keep the
+	 * current weights.
+	 *
+	 * XXX The threshold of 100 allocation is pretty arbitrary.
+	 *
+	 * XXX Maybe a better strategy would be to slowly return to the default
+	 * weights, with each partition allocation only from itself?
+	 *
+	 * XXX Maybe we shouldn't even reset the counters in this case? But it
+	 * should not matter, if the activity is low.
+	 */
+	if (avg_allocs < 100)
+	{
+		elog(DEBUG1, "rebalance skipped: not enough allocations (allocs: %u)",
+			 avg_allocs);
+		return;
+	}
+
+	/*
+	 * Likewise, skip rebalancing if the misbalance is not significant. We
+	 * consider it acceptable if the amount of allocations we'd need to
+	 * redistribute is less than 10% of the average.
+	 *
+	 * XXX Again, these threshold are rather arbitrary. And maybe we should
+	 * do the rabalancing in this case anyway, it's likely cheap and on a big
+	 * system 10% can be quite a lot.
+	 */
+	if (delta_allocs < (avg_allocs * 0.1))
+	{
+		elog(DEBUG1, "rebalance skipped: delta within limit (delta: %u, threshold: %u)",
+			 delta_allocs, (uint32) (avg_allocs * 0.1));
+		return;
+	}
+
+	/*
+	 * The actual rebalancing
+	 *
+	 * Partition with fewer than average allocations, should not redirect any
+	 * allocations to other partitions. So just use weights with a single
+	 * non-zero weight for the partition itself.
+	 *
+	 * Partition with more than average allocations, should not receive any
+	 * redirected allocations, and instead it should redirect excess allocations
+	 * to other partitions.
+	 *
+	 * The redistribution is "proportional" - if the excess allocations of a
+	 * partition represent 10% of the "delta", then each partition that
+	 * needs more allocations will get 10% of the gap from it.
+	 *
+	 * XXX We should add hysteresis, so that it does not oscillate or something
+	 * like that. Maybe CLOCKSWEEP_HISTORY_COEFF already does that?
+	 *
+	 * XXX Ideally, the alternative partitions to use first would be the other
+	 * partitions for the same node (if any).
+	 */
+	for (int i = 0; i < StrategyControl->num_partitions; i++)
+	{
+		ClockSweep *sweep = &StrategyControl->sweeps[i];
+		uint8		balance[MAX_BUFFER_PARTITIONS];
+
+		/* lock, we're going to modify the balance weights */
+		SpinLockAcquire(&sweep->clock_sweep_lock);
+
+		/* reset the weights to start from scratch */
+		memset(balance, 0, sizeof(uint8) * MAX_BUFFER_PARTITIONS);
+
+		/* does this partition has fewer or more than avg_allocs? */
+		if (allocs[i] < avg_allocs)
+		{
+			/* fewer - don't redirect any allocations elsewhere */
+			balance[i] = 100;
+		}
+		else
+		{
+			/*
+			 * more - redistribute the excess allocations
+			 *
+			 * Each "target" partition (with less than avg_allocs) should get
+			 * a fraction proportional to (excess/delta) from this one.
+			 */
+
+			/* fraction of the "total" delta */
+			double	delta_frac = (allocs[i] - avg_allocs) * 1.0 / delta_allocs;
+
+			/* keep just enough allocations to meet the target */
+			balance[i] = (100.0 * avg_allocs / allocs[i]);
+
+			/* redirect the extra allocations */
+			for (int j = 0; j < StrategyControl->num_partitions; j++)
+			{
+				/* How many allocations to receive from i-th partition? */
+				uint32	receive_allocs = delta_frac * (avg_allocs - allocs[j]);
+
+				/* ignore partitions that don't need additional allocations */
+				if (allocs[j] > avg_allocs)
+					continue;
+
+				/* fraction to redirect */
+				balance[j] = (100.0 * receive_allocs / allocs[i]) + 0.5;
+			}
+		}
+
+		/* combine the old and new weights (hysteresis) */
+		for (int j = 0; j < MAX_BUFFER_PARTITIONS; j++)
+		{
+			sweep->balance[j]
+				= CLOCKSWEEP_HISTORY_COEFF * sweep->balance[j] +
+				  (1.0 - CLOCKSWEEP_HISTORY_COEFF) * balance[j];
+		}
+
+		SpinLockRelease(&sweep->clock_sweep_lock);
+	}
+}
+
 /*
  * StrategySyncPrepare -- prepare for sync of all partitions
  *
@@ -657,7 +1039,21 @@ StrategyCtlShmemInit(void *arg)
 		/* Clear statistics */
 		StrategyControl->sweeps[i].completePasses = 0;
 		pg_atomic_init_u32(&StrategyControl->sweeps[i].numBufferAllocs, 0);
+		pg_atomic_init_u32(&StrategyControl->sweeps[i].numRequestedAllocs, 0);
 		pg_atomic_init_u64(&StrategyControl->sweeps[i].numTotalAllocs, 0);
+		pg_atomic_init_u64(&StrategyControl->sweeps[i].numTotalRequestedAllocs, 0);
+
+		/*
+		 * Initialize the weights - start by allocating 100% buffers from
+		 * the current node / partition.
+		 */
+		for (int j = 0; j < MAX_BUFFER_PARTITIONS; j++)
+		{
+			if (i == j)
+				StrategyControl->sweeps[i].balance[i] = 100;
+			else
+				StrategyControl->sweeps[i].balance[j] = 0;
+		}
 	}
 
 	/* No pending notification */
@@ -1025,8 +1421,10 @@ StrategyRejectBuffer(BufferAccessStrategy strategy, BufferDesc *buf, bool from_r
 
 void
 
@@ -1034,11 +1432,21 @@ ClockSweepPartitionGetInfo(int idx,

 
index 5ab0cee4281..887314d43f4 100644
@@ -593,6 +593,7 @@ extern BufferDesc *StrategyGetBuffer(BufferAccessStrategy strategy,
 
+extern void StrategySyncBalance(void);
 extern void StrategySyncPrepare(int *num_parts, uint32 *num_buf_alloc);
 extern int	StrategySyncStart(int partition, uint32 *complete_passes,
 							  int *first_buffer, int *num_buffers);
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index e0bb4cc1df1..02833b19b0c 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -411,11 +411,13 @@ extern int	GetAccessStrategyPinLimit(BufferAccessStrategy strategy);

 /* inline functions */
 
-- 
2.54.0

From 5ab7aba5f8af84b748180eb6caa2b165aceb2590 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 2 Jun 2026 22:16:55 +0200
Subject: [PATCH v20260605 4/6] clock-sweep: basic partitioning

Note: There's a problem with some tests running out of unpinned buffers,
due to (intentionally) setting shared buffers very low. That happens
because StrategyGetBuffer() only searches a single partition, and it
has a couple more issues.
---
 .../pg_buffercache--1.7--1.8.sql              |   8 +-
 contrib/pg_buffercache/pg_buffercache_pages.c |  32 +-
 src/backend/storage/buffer/bufmgr.c           | 202 +++++++----
 src/backend/storage/buffer/freelist.c         | 333 ++++++++++++++++--
 src/include/storage/buf_internals.h           |   4 +-
 src/include/storage/bufmgr.h                  |   5 +
 src/test/recovery/t/027_stream_regress.pl     |   5 +
 src/tools/pgindent/typedefs.list              |   1 +
 8 files changed, 486 insertions(+), 104 deletions(-)

index a6e49fd1652..92176fed7f8 100644
@@ -14,7 +14,13 @@ CREATE VIEW pg_buffercache_partitions AS
 
 -- Don't want these to be available to public.
 REVOKE ALL ON FUNCTION pg_buffercache_partitions() FROM PUBLIC;
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index e3efeeda675..739c63b0cfc 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -31,7 +31,7 @@
 #define NUM_BUFFERCACHE_MARK_DIRTY_ALL_ELEM 3

 PG_MODULE_MAGIC_EXT(
@@ -912,6 +912,14 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)

@@ -933,12 +941,22 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 					first_buffer,
 					last_buffer;

 
 
@@ -954,6 +972,18 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		values[4] = Int32GetDatum(last_buffer);
 		nulls[4] = false;
 
index cc398db124d..02c75f82e5b 100644
@@ -3826,33 +3826,34 @@ BufferSync(int flags)
 }
 
 /*
- * BgBufferSync -- Write out some dirty buffers in the pool.
- *
- * This is called periodically by the background writer process.
+ * Information saved between calls so we can determine the strategy point's
+ * advance rate and avoid scanning already-cleaned buffers.
  *
- * Returns true if it's appropriate for the bgwriter process to go into
- * low-power hibernation mode.  (This happens if the strategy clock-sweep
- * has been "lapped" and no buffer allocations have occurred recently,
- * or if the bgwriter has been effectively disabled by setting
- * bgwriter_lru_maxpages to 0.)
+ * XXX Does it actually make sense to split all of this information per
+ * partition? For example, does per-partition advance rate mean anything?
+ * Maybe we should have a global advance rate? Although, if we want to
+ * keep enough clean buffers in each partition, maybe having per-partition
+ * rates makes sense.
  */
-bool
-BgBufferSync(WritebackContext *wb_context)
+typedef struct BufferSyncPartition
+{
+	int	prev_strategy_buf_id;
+	uint32 prev_strategy_passes;
+	int	next_to_clean;
+	uint32 next_passes;
+} BufferSyncPartition;
+
+static BufferSyncPartition *saved_info = NULL;
+static bool saved_info_valid = false;
+
+static bool
+BgBufferSyncPartition(WritebackContext *wb_context, int num_partitions,
+					  int partition, int recent_alloc_partition,
+					  BufferSyncPartition *saved)
 {
 	/* info obtained from freelist.c */
 	int			strategy_buf_id;
 	uint32		strategy_passes;
-	uint32		recent_alloc;
-
-	/*
-	 * Information saved between calls so we can determine the strategy
-	 * point's advance rate and avoid scanning already-cleaned buffers.
-	 */
-	static bool saved_info_valid = false;
-	static int	prev_strategy_buf_id;
-	static uint32 prev_strategy_passes;
-	static int	next_to_clean;
-	static uint32 next_passes;
 
 	/* Moving averages of allocation rate and clean-buffer density */
 	static float smoothed_alloc = 0;
@@ -3880,25 +3881,16 @@ BgBufferSync(WritebackContext *wb_context)
 	long		new_strategy_delta;
 	uint32		new_recent_alloc;

 	/*
 	 * Compute strategy_delta = how many buffers have been scanned by the
@@ -3910,17 +3902,17 @@ BgBufferSync(WritebackContext *wb_context)
 	 */
 	if (saved_info_valid)
 	{
-		int32		passes_delta = strategy_passes - prev_strategy_passes;
+		int32		passes_delta = strategy_passes - saved->prev_strategy_passes;
 
-		strategy_delta = strategy_buf_id - prev_strategy_buf_id;
-		strategy_delta += (long) passes_delta * NBuffers;
+		strategy_delta = strategy_buf_id - saved->prev_strategy_buf_id;
+		strategy_delta += (long) passes_delta * num_buffers;

-		if ((int32) (next_passes - strategy_passes) > 0)
+		if ((int32) (saved->next_passes - strategy_passes) > 0)
 		{
 			/* we're one pass ahead of the strategy point */
-			bufs_to_lap = strategy_buf_id - next_to_clean;
+			bufs_to_lap = strategy_buf_id - saved->next_to_clean;
 #ifdef BGW_DEBUG
 			elog(DEBUG2, "bgwriter ahead: bgw %u-%u strategy %u-%u delta=%ld lap=%d",
 				 next_passes, next_to_clean,
@@ -3928,11 +3920,11 @@ BgBufferSync(WritebackContext *wb_context)
 				 strategy_delta, bufs_to_lap);
 #endif
 		}
-		else if (next_passes == strategy_passes &&
-				 next_to_clean >= strategy_buf_id)
+		else if (saved->next_passes == strategy_passes &&
+				 saved->next_to_clean >= strategy_buf_id)
 		{
 			/* on same pass, but ahead or at least not behind */
-			bufs_to_lap = NBuffers - (next_to_clean - strategy_buf_id);
+			bufs_to_lap = num_buffers - (saved->next_to_clean - strategy_buf_id);
 #ifdef BGW_DEBUG
 			elog(DEBUG2, "bgwriter ahead: bgw %u-%u strategy %u-%u delta=%ld lap=%d",
 				 next_passes, next_to_clean,
@@ -3952,9 +3944,9 @@ BgBufferSync(WritebackContext *wb_context)
 				 strategy_passes, strategy_buf_id,
 				 strategy_delta);
 #endif
-			next_to_clean = strategy_buf_id;
-			next_passes = strategy_passes;
-			bufs_to_lap = NBuffers;
+			saved->next_to_clean = strategy_buf_id;
+			saved->next_passes = strategy_passes;
+			bufs_to_lap = num_buffers;
 		}
 	}
 	else
@@ -3968,15 +3960,16 @@ BgBufferSync(WritebackContext *wb_context)
 			 strategy_passes, strategy_buf_id);
 #endif
 		strategy_delta = 0;
-		next_to_clean = strategy_buf_id;
-		next_passes = strategy_passes;
-		bufs_to_lap = NBuffers;
+		saved->next_to_clean = strategy_buf_id;
+		saved->next_passes = strategy_passes;
+		bufs_to_lap = num_buffers;
 	}
 
 	/* Update saved info for next time */
-	prev_strategy_buf_id = strategy_buf_id;
-	prev_strategy_passes = strategy_passes;
-	saved_info_valid = true;
+	saved->prev_strategy_buf_id = strategy_buf_id;
+	saved->prev_strategy_passes = strategy_passes;
+	/* XXX this needs to happen only after all partitions */
+	/* saved_info_valid = true; */
 
 	/*
@@ -3984,9 +3977,9 @@ BgBufferSync(WritebackContext *wb_context)
@@ -3996,7 +3989,7 @@ BgBufferSync(WritebackContext *wb_context)
 
 	/*
@@ -4004,10 +3997,10 @@ BgBufferSync(WritebackContext *wb_context)
 
@@ -4034,7 +4027,7 @@ BgBufferSync(WritebackContext *wb_context)
 
 	if (upcoming_alloc_est < (min_scan_buffers + reusable_buffers_est))
 	{
@@ -4059,20 +4052,20 @@ BgBufferSync(WritebackContext *wb_context)
 	/* Execute the LRU scan */
 	while (num_to_scan > 0 && reusable_buffers < upcoming_alloc_est)
 	{
-		int			sync_state = SyncOneBuffer(next_to_clean, true,
+		int			sync_state = SyncOneBuffer(saved->next_to_clean, true,
 											   wb_context);
 
-		if (++next_to_clean >= NBuffers)
+		if (++saved->next_to_clean >= (first_buffer + num_buffers))
 		{
-			next_to_clean = 0;
-			next_passes++;
+			saved->next_to_clean = first_buffer;
+			saved->next_passes++;
 		}
 		num_to_scan--;
 
@@ -4086,7 +4079,7 @@ BgBufferSync(WritebackContext *wb_context)
 
 #ifdef BGW_DEBUG
@@ -4116,8 +4109,83 @@ BgBufferSync(WritebackContext *wb_context)
 #endif
 	}
 
+	/* allocate space for per-partition information between calls */
+	if (saved_info == NULL)
+		 * XXX Not great it's using malloc(), but how else to allocate a
+		 * variable-length array?
+		saved_info = malloc(sizeof(BufferSyncPartition) * num_partitions);
+										   partition, recent_alloc_partition,
+										   &saved_info[partition]);
 
 /*
diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index 53ef5239e8d..2d56579682e 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -36,17 +36,28 @@

 /*
+	/* range for this clock sweep partition */
+	int32		node;
+	int32		firstBuffer;
+	int32		numBuffers;
 
 	/*
 
@@ -57,11 +68,32 @@ typedef struct
 	uint32		completePasses; /* Complete cycles of the clock-sweep */
 	pg_atomic_uint32 numBufferAllocs;	/* Buffers allocated since last reset */
 
+	/* cached info about freelist partitioning */
 
 /* Pointers to shared state */
@@ -108,6 +140,7 @@ static BufferDesc *GetBufferFromRing(BufferAccessStrategy strategy,
 									 uint64 *buf_state);
 static void AddBufferToRing(BufferAccessStrategy strategy,
 							BufferDesc *buf);
+static ClockSweep *ChooseClockSweep(void);
 
 /*
  * ClockSweepTick - Helper routine for StrategyGetBuffer()
@@ -119,6 +152,7 @@ static inline uint32
 ClockSweepTick(void)
 {
 	uint32		victim;
+	ClockSweep *sweep = ChooseClockSweep();
 
 	/*
 	 * Atomically move hand ahead one buffer - if there's several processes
@@ -126,14 +160,14 @@ ClockSweepTick(void)
 	 * apparent order.
 	 */
 	victim =
-		pg_atomic_fetch_add_u32(&StrategyControl->nextVictimBuffer, 1);
+		pg_atomic_fetch_add_u32(&sweep->nextVictimBuffer, 1);

 
 		/*
@@ -159,19 +193,118 @@ ClockSweepTick(void)

-				success = pg_atomic_compare_exchange_u32(&StrategyControl->nextVictimBuffer,
+				success = pg_atomic_compare_exchange_u32(&sweep->nextVictimBuffer,
 														 &expected, wrapped);
 				if (success)
-					StrategyControl->completePasses++;
-				SpinLockRelease(&StrategyControl->buffer_strategy_lock);
+					sweep->completePasses++;
+				SpinLockRelease(&sweep->clock_sweep_lock);
 			}
 		}
 	}
-	return victim;
+
+	/*
+	 * Make sure we've calculated a buffer in the range of the partition. Buffer
+	 * IDs are 1-based, we're calculating 0-based indexes.
+	 */
+	Assert((victim >= 0) && (victim < sweep->numBuffers));
+	Assert(BufferIsValid(1 + sweep->firstBuffer + victim));
+
+	return sweep->firstBuffer + victim;
+}
+
+/*
+ * ClockSweepPartitionIndex
+ *		pick the clock-sweep partition to use based on PID and NUMA node
+ *
+ * With libnuma, use the NUMA node and PID to pick the partition. Otherwise
+ * use just PID (as if there's a single NUMA node).
+ *
+ * XXX This should also check if buffers are NUMA-partitioned, not just if
+ * compiled with libnuma.
+ */
+static int
+ClockSweepPartitionIndex(void)
+{
+	int		node = 0,
+			index;
+	pid_t	pid = MyProcPid;;
+
+	Assert(StrategyControl->num_partitions ==
+		   (StrategyControl->num_nodes * StrategyControl->num_partitions_per_node));
+
+	/*
+	 * If buffers are NUMA-partitioned, determine the partition using the NUMA
+	 * node and PID. Without NUMA assume everything is a single NUMA node 0, and
+	 * we pick the partition based on PID.
+	 */
+#ifdef USE_LIBNUMA
+	if (shared_buffers_numa)
+	{
+		int		cpu;
+
+		/* XXX do we need to check sched_getcpu is available, somehow? */
+		if ((cpu = sched_getcpu()) < 0)
+			elog(ERROR, "sched_getcpu failed: %m");
+
+		node = numa_node_of_cpu(cpu);
+	}
+#endif
+
+	/*
+	 * We should't get unexpected NUMA nodes, not considered when setting up the
+	 * buffer partitions. It could happen if the allowed NUMA nodes get adjusted
+	 * at runtime, but at this point we just create partitions for all existing
+	 * nodes. We could plan for allowed partitions, but then what if those get
+	 * disabled, and the user allows some other partitions?
+	 */
+	if ((node < 0) || (node > StrategyControl->num_nodes))
+		elog(ERROR, "node out of range: %d > %u", node, StrategyControl->num_nodes);
+
+	/*
+	 * Calculate the partition index. Nodes have the same number of partitions,
+	 * and we use the PID to pick one of those (for a given node). If there's
+	 * only a single partition per node, we can ignore PID and use node directly.
+	 */
+	if (StrategyControl->num_partitions_per_node == 1)
+	{
+		/* fast-path */
+		index = node;
+	}
+	else
+	{
+		/* use PID to pick one of node's partitions */
+		index = (node * StrategyControl->num_partitions_per_node)
+			+ (pid % StrategyControl->num_partitions_per_node);
+	}
+
+	/* should have a valid partition index */
+	Assert((index >= 0) && (index < StrategyControl->num_partitions));
+
+	return index;
+}
+
+/*
+ * ChooseClockSweep
+ *		pick a clocksweep partition based on NUMA node and PID
+ *
+ * Pick a partition mapped to the NUMA node the backend is currently running
+ * on, and use PID if there are multiple partitions per node. Without NUMA
+ * supported/enabled, use just PID.
+ *
+ * XXX Maybe we should do both the total and "per group" counts a power of
+ * two? That'd allow using shifts instead of divisions in the calculation,
+ * and that's cheaper. But how would that deal with odd number of nodes?
+ */
+static ClockSweep *
+ChooseClockSweep(void)
+{
+	int			index = ClockSweepPartitionIndex();
+
+	return &StrategyControl->sweeps[index];
 }
 
 /*
@@ -242,10 +375,37 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint64 *buf_state, bool *from_r
 	 * We count buffer allocation requests so that the bgwriter can estimate
 	 * the rate of buffer consumption.  Note that buffers recycled by a
 	 * strategy object are intentionally not counted here.
+	 *
+	 * XXX It's not quite right we call ChooseClockSweep twice - now, and then
+	 * a couple lines later (through ClockSweepTick). If the process moves
+	 * between CPUs / NUMA nodes in between, these call may pick different
+	 * partitions, confusing the logic a bit.
 	 */
-	pg_atomic_fetch_add_u32(&StrategyControl->numBufferAllocs, 1);
+	pg_atomic_fetch_add_u32(&ChooseClockSweep()->numBufferAllocs, 1);
 
-	/* Use the "clock sweep" algorithm to find a free buffer */
+	/*
+	 * Use the "clock sweep" algorithm to find a free buffer
+	 *
+	 * XXX Note that ClockSweepTick() is NUMA-aware, i.e. it only looks at
+	 * buffers from a single partition, aligned with the NUMA node. That means
+	 * a process "sweeps" only a fraction of buffers, even if the other buffers
+	 * are better candidates for eviction. Maybe there should be some logic to
+	 * "steal" buffers from other partitions or other nodes?
+	 *
+	 * XXX This only searches a single partition, which can result in "no
+	 * unpinned buffers available" even if there are buffers in other
+	 * partitions. Needs to scan partitions if needed, as a fallback.
+	 *
+	 * XXX Would that also mean we should have multiple bgwriters, one for each
+	 * node, or would one bgwriter still handle all nodes?
+	 *
+	 * XXX Also, the trycounter should not be set to NBuffers, but to buffer
+	 * count for that one partition. In fact, this should not call ClockSweepTick
+	 * for every iteration. The call is likely quite expensive (does a lot
+	 * of stuff), and also may return a different partition on each call.
+	 * We should just do it once, and then do the for(;;) loop. And then
+	 * maybe advance to the next partition, until we scan through all of them.
+	 */
 	trycounter = NBuffers;
 	for (;;)
 	{
@@ -325,6 +485,48 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint64 *buf_state, bool *from_r
 	}
 }
 
+/*
+		/* XXX Do we need the lock, if we're only accessing atomics? Surely not. */
+		/* XXX Are we ever calling this without num_buf_alloc? */
@@ -332,37 +534,44 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint64 *buf_state, bool *from_r

 	if (complete_passes)
 	{
 
 		/*
 
-	if (num_buf_alloc)
-	{
 
 /*
@@ -394,8 +603,14 @@ StrategyNotifyBgWriter(int bgwprocno)
 static void
 StrategyCtlShmemRequest(void *arg)
 {
+	int			num_partitions;
+
+	/* get the number of buffer partitions */
+	BufferPartitionsCalculate(NULL, &num_partitions, NULL);
+
 	ShmemRequestStruct(.name = "Buffer Strategy Status",
-					   .size = sizeof(BufferStrategyControl),
+					   .size = offsetof(BufferStrategyControl, sweeps) +
+							mul_size(num_partitions, sizeof(ClockSweep)),
 					   .ptr = (void **) &StrategyControl
 		);
 }
@@ -408,12 +623,42 @@ StrategyCtlShmemInit(void *arg)
 {
 	SpinLockInit(&StrategyControl->buffer_strategy_lock);
 
-	/* Initialize the clock-sweep pointer */
-	pg_atomic_init_u32(&StrategyControl->nextVictimBuffer, 0);
+	/* Remember the number of partitions */
+	BufferPartitionsParams(&StrategyControl->num_nodes,
+						   &StrategyControl->num_partitions,
+						   &StrategyControl->num_partitions_per_node);
+
+	/* Initialize the clock sweep pointers (for all partitions) */
+	for (int i = 0; i < StrategyControl->num_partitions; i++)
+	{
+		int			node,
+					num_buffers,
+					first_buffer,
+					last_buffer;
+
+		SpinLockInit(&StrategyControl->sweeps[i].clock_sweep_lock);
 
-	/* Clear statistics */
+		StrategyControl->sweeps[i].node = node;
 
 	/* No pending notification */
 	StrategyControl->bgwprocno = -1;
@@ -777,3 +1022,23 @@ StrategyRejectBuffer(BufferAccessStrategy strategy, BufferDesc *buf, bool from_r
 
 	return true;
 }
+
+void
index e944cee2e91..5ab0cee4281 100644
@@ -593,7 +593,9 @@ extern BufferDesc *StrategyGetBuffer(BufferAccessStrategy strategy,

 /* buf_table.c */
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index 1cf09e8fb7c..e0bb4cc1df1 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -410,6 +410,11 @@ extern int	GetAccessStrategyBufferCount(BufferAccessStrategy strategy);
 extern int	GetAccessStrategyPinLimit(BufferAccessStrategy strategy);

 /* inline functions */
index ae977297849..f68e08e5697 100644
 
index ea756015249..183570b4d43 100644
@@ -451,6 +451,7 @@ ClientCertName
 ClientConnectionInfo
 ClientData
 ClientSocket
+ClockSweep
 ClonePtrType
 ClosePortalStmt
 ClosePtrType
-- 
2.54.0

From 4672b0b53f68177ca76b9400e7f9ca4172a08ddb Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 2 Jun 2026 23:27:13 +0200
Subject: [PATCH v20260605 3/6] NUMA: shared buffers partitioning

Ensure shared buffers are allocated from all NUMA nodes, in a balanced
way, instead of just using the node where Postgres initially starts, or
where the kernel decides to migrate the page, etc.

In cases like pre-warming a database from a single worker (e.g. using
pg_prewarm), we may end up with severely unbalanced memory distribution
(with most memory located on a single NUMA node). Unbalanced allocation
may put a lot of pressure on the memory system on a small number of NUMA
nodes, limiting the bandwidth etc.

With zone_reclaim, the kernel would eventually move some of the memory
to other nodes, but that tends to take a long time and is unpredictable.

This change forces even distribution of shared buffers on all NUMA
nodes, improving predictability, reducing the time needed for warmup
during benchmarking, etc. It's also less dependent on what the CPU
scheduler decides to do (which cores get used for the warmup.)

The effect is similar to

     numactl --interleave=all

in that the buffers are distributed on the NUMA nodes evenly, but
there's also a number of important differences.

Firstly, it's applied only to shared buffers (and buffer descriptors),
not to the whole shared memory segment. It's possible to enable memory
interleaving using the shmem_interleave GUC, introduced in an earlier
patch in this series.

NUMA works at the granularity of a memory page, which is typically
either 4K or 2MB (hugepage), but other sizes are possible. For systems
where NUMA matters, we expect large amounts of memory (hundreds of
gigabytes) and hugepages enabled. But not necessarily.

The partitioning scheme is best-effort with respect to memory page size.
The shared buffers do not "align" with memory pages (i.e. a partition
may not end at the memory page boundary), in which case we simply locate
just the section of the partition with complete memory pages. This means
there may be ~one unmapped memory page between partitions. Considering
the expected amounts of memory, this is negligible, and the alternative
would be a significant amount of complexity to align the pages and
enforce "allowed" partition sizes.

Buffer descriptors are affected by this too, and the effect may be more
significant, simply because the descriptors are much smaller (~64B). So
the array is smaller, and a single 2MB memory page is worth ~32K buffer
descriptors. But with large systems it's still negligible.

The "buffer partitions" may not be 1:1 with NUMA nodes. We want to allow
clock-sweep partitioning even on non-NUMA systems, or when running only
on a small number of NUMA nodes. There's a minimal number of partitions
(default: 4), and a node may get multiple partitions. Nodes always get
the same number of partitions (e.g. with 3 NUMA nodes there will be 6
partitions in total, as each node gets 2 partitions).

The feature is enabled by dshared_buffers_numa GUC (default: false).
---
 .../pg_buffercache--1.7--1.8.sql              |   1 +
 contrib/pg_buffercache/pg_buffercache_pages.c |  24 +-
 src/backend/storage/buffer/buf_init.c         | 244 ++++++++++++++++--
 src/backend/storage/buffer/freelist.c         |   9 +
 src/backend/utils/misc/guc_parameters.dat     |   6 +
 src/backend/utils/misc/postgresql.conf.sample |   1 +
 src/include/port/pg_numa.h                    |   7 +
 src/include/storage/buf_internals.h           |  16 +-
 src/include/storage/bufmgr.h                  |   8 +
 src/port/pg_numa.c                            | 112 ++++++++
 10 files changed, 392 insertions(+), 36 deletions(-)

diff --git a/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql b/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
index d62b8339bfc..a6e49fd1652 100644
--- a/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
+++ b/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
@@ -11,6 +11,7 @@ LANGUAGE C PARALLEL SAFE;
 CREATE VIEW pg_buffercache_partitions AS
 	SELECT P.* FROM pg_buffercache_partitions() AS P
 	(partition integer,			-- partition index
+	 numa_node integer,			-- NUMA node of the partitioon
 	 num_buffers integer,		-- number of buffers in the partition
 	 first_buffer integer,		-- first buffer of partition
 	 last_buffer integer);		-- last buffer of partition
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index d678cb045ba..e3efeeda675 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -31,7 +31,7 @@
 #define NUM_BUFFERCACHE_MARK_DIRTY_ALL_ELEM 3

 PG_MODULE_MAGIC_EXT(
@@ -904,11 +904,13 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 
 		funcctx->user_fctx = BlessTupleDesc(tupledesc);
@@ -926,28 +928,32 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 	{
 		uint32		i = funcctx->call_cntr;
 
-		int			num_buffers,
+		int			numa_node,
+					num_buffers,
 					first_buffer,
 					last_buffer;

 

 
 
index e593b02e0ca..1f93a31d451 100644
@@ -14,12 +14,20 @@
  */
 #include "postgres.h"
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
+#include "port/pg_numa.h"
 #include "storage/aio.h"
 #include "storage/buf_internals.h"
 #include "storage/bufmgr.h"
 #include "storage/proclist.h"
 #include "storage/shmem.h"
 #include "storage/subsystems.h"
+#include "utils/guc_hooks.h"
+#include "utils/varlena.h"
 
 BufferDescPadded *BufferDescriptors;
 char	   *BufferBlocks;
@@ -71,9 +79,12 @@ const ShmemCallbacks BufferManagerShmemCallbacks = {
  *		multiple times. Check the PrivateRefCount infrastructure in bufmgr.c.
  */
 
-/* number of buffer partitions */
-#define NUM_CLOCK_SWEEP_PARTITIONS	4
+/*
+ * Minimum number of buffer partitions, no matter the number of NUMA nodes.
+ */
+#define MIN_BUFFER_PARTITIONS	4
 
+bool	shared_buffers_numa = false;
 
 /*
  * Register shared memory area for the buffer pool.
@@ -81,6 +92,10 @@ const ShmemCallbacks BufferManagerShmemCallbacks = {
 static void
 BufferManagerShmemRequest(void *arg)
 {
+	int		nparts;
+
+	BufferPartitionsCalculate(NULL, &nparts, NULL);
+
 	ShmemRequestStruct(.name = "Buffer Descriptors",
 					   .size = NBuffers * sizeof(BufferDescPadded),
 	/* Align descriptors to a cacheline boundary. */
@@ -103,7 +118,7 @@ BufferManagerShmemRequest(void *arg)
 		);
 
 	ShmemRequestStruct(.name = "Buffer Partition Registry",
-					   .size = NUM_CLOCK_SWEEP_PARTITIONS * sizeof(BufferPartition),
+					   .size = nparts * sizeof(BufferPartition),
 	/* Align descriptors to a cacheline boundary. */
 					   .alignment = PG_CACHE_LINE_SIZE,
 					   .ptr = (void **) &BufferPartitionsRegistry,
@@ -134,6 +149,10 @@ BufferManagerShmemInit(void *arg)
 	/*
 	 * Initialize the buffer partition registry first, before other parts
 	 * have a chance to touch the memory.
+	 *
+	 * Also moves memory to different NUMA nodes (if enabled by a GUC).
+	 * Do this before the loop that initializes buffer headers etc. which
+	 * may fault some of the memory pages etc.
 	 */
 	BufferPartitionsInit();
 
@@ -231,35 +250,203 @@ BufferPartitionsInit(void)
 {
 	int			buffer = 0;
 
-	/* number of buffers per partition (make sure to not overflow) */
-	int			part_buffers = NBuffers / NUM_CLOCK_SWEEP_PARTITIONS;
-	int			remaining_buffers = NBuffers % NUM_CLOCK_SWEEP_PARTITIONS;
+	int			nnodes,
+				npartitions,
+				npartitions_per_node;
 
-	BufferPartitionsRegistry->npartitions = NUM_CLOCK_SWEEP_PARTITIONS;
+	int			buffers_per_partition,
+				buffers_remaining;
 
-	for (int n = 0; n < BufferPartitionsRegistry->npartitions; n++)
-	{
-		BufferPartition *part = &BufferPartitionsRegistry->partitions[n];
+	/* calculate partitioning parameters */
+	BufferPartitionsCalculate(&nnodes, &npartitions, &npartitions_per_node);
+
+	/* paranoia */
+	Assert(nnodes > 0);
+	Assert(npartitions >= MIN_BUFFER_PARTITIONS);
+	Assert((npartitions % nnodes) == 0);
+	Assert((npartitions_per_node * nnodes) == npartitions);
 
-		int			num_buffers = part_buffers;
-		if (n < remaining_buffers)
-			num_buffers += 1;
+	BufferPartitionsRegistry->nnodes = nnodes;
+	BufferPartitionsRegistry->npartitions = npartitions;
+	BufferPartitionsRegistry->npartitions_per_node = npartitions_per_node;
 
-		remaining_buffers -= num_buffers;
+	/* regular partition size, the first couple get an extra buffer */
+	buffers_per_partition = (NBuffers / npartitions);
+	buffers_remaining = (NBuffers % buffers_per_partition);
 
-		Assert((num_buffers > 0) && (num_buffers <= part_buffers));
-		Assert((buffer >= 0) && (buffer < NBuffers));
+	/* should have all the buffers */
+	Assert((buffers_per_partition * npartitions + buffers_remaining) == NBuffers);
 
-		part->num_buffers = num_buffers;
-		part->first_buffer = buffer;
-		part->last_buffer = buffer + (num_buffers - 1);
+	/*
+	 * Now walk the partitions, and set the buffer range. Optionally, place
+	 * the partitions on a given node (for all partitions at once).
+	 */
+	for (int n = 0; n < nnodes; n++)
+	{
+		for (int p = 0; p < npartitions_per_node; p++)
+		{
+			int			idx = (n * npartitions_per_node) + p;
+			BufferPartition *part = &BufferPartitionsRegistry->partitions[idx];
+
+			/*
+			 * Assign to the NUMA node, but only with shared_buffers_numa=on.
+			 *
+			 * XXX we should get an actual node ID from the mask, in case the
+			 * task is restricted to only some nodes.
+			 */
+			part->numa_node = (shared_buffers_numa) ? n : -1;
+
+			/* The first couple partitions may get an extra buffer. */
+			part->num_buffers = buffers_per_partition;
+			if (idx < buffers_remaining)
+				part->num_buffers += 1;
+
+			/* remember the buffer range */
+			part->first_buffer = buffer;
+			part->last_buffer = buffer + (part->num_buffers - 1);
+
+			/* remember start of the next partition */
+			buffer += part->num_buffers;
+		}
 
-		buffer += num_buffers;
+#ifdef USE_LIBNUMA
+		/*
+		 * Now try to locate buffers and buffer descriptors to the node (all
+		 * partitions for the node at once).
+		 */
+		if (shared_buffers_numa)
+		{
+			Size	numa_page_size = pg_numa_page_size();
+
+			int		part_first,
+					part_last,
+					buff_first,
+					buff_last;
+
+			char   *startptr,
+				   *endptr;
+
+			/* first/last partition for this node */
+			part_first = (n * npartitions_per_node);
+			part_last = part_first + (npartitions_per_node - 1);
+
+			/* buffers (blocks) */
+
+			/* first/last buffer */
+			buff_first = BufferPartitionsRegistry->partitions[part_first].first_buffer;
+			buff_last = BufferPartitionsRegistry->partitions[part_last].last_buffer;
+
+			/* beginning of the first block, end of last block */
+			startptr = BufferBlocks + ((Size) buff_first * BLCKSZ);
+			endptr = BufferBlocks + ((Size) (buff_last + 1) * BLCKSZ);
+
+			/* print some warnings when the partitions are not aligned */
+			if ((startptr != (char *) TYPEALIGN(numa_page_size, startptr)) ||
+				(endptr != (char *) TYPEALIGN_DOWN(numa_page_size, endptr)))
+			{
+				elog(WARNING, "buffers for node %d not well aligned [%p,%p] aligned [%p,%p]",
+					 n, startptr, endptr,
+					 (char *) TYPEALIGN(numa_page_size, startptr),
+					 (char *) TYPEALIGN_DOWN(numa_page_size, endptr));
+			}
+
+			/* best effort: align the pointers, so that the mbind() works */
+			startptr = (char *) TYPEALIGN(numa_page_size, startptr);
+			endptr = (char *) TYPEALIGN_DOWN(numa_page_size, endptr);
+
+			/* XXX or should we use pg_numa_move_to_node? */
+			pg_numa_bind_to_node(startptr, endptr, n);
+
+			/* buffer descriptors */
+
+			/* beginning of the first descriptor, end of last descriptor */
+			startptr = (char *) &BufferDescriptors[buff_first];
+			endptr = (char *) &BufferDescriptors[buff_last] + 1;
+
+			/* print some warnings when the partitions are not aligned */
+			if ((startptr != (char *) TYPEALIGN(numa_page_size, startptr)) ||
+				(endptr != (char *) TYPEALIGN_DOWN(numa_page_size, endptr)))
+			{
+				elog(WARNING, "buffers descriptors for node %d not well aligned [%p,%p] aligned [%p,%p]",
+					 n, startptr, endptr,
+					 (char *) TYPEALIGN(numa_page_size, startptr),
+					 (char *) TYPEALIGN_DOWN(numa_page_size, endptr));
+			}
+
+			/* best effort: align the pointers, so that the mbind() works */
+			startptr = (char *) TYPEALIGN(numa_page_size, startptr);
+			endptr = (char *) TYPEALIGN_DOWN(numa_page_size, endptr);
+
+			/* XXX or should we use pg_numa_move_to_node? */
+			pg_numa_bind_to_node(startptr, endptr, n);
+		}
+#endif
 	}

+/*
+ * BufferPartitionsCalculate
+ *		Pick number of buffer partitions for the number of nodes and
+ *		MIN_BUFFER_PARTITIONS.
+ *
+ * Picks the smallest number of partitions higher thah MIN_BUFFER_PARTITIONS,
+ * such that all nodes have the same number of partitions.
+ *
+ * This is best-effort with respect to size of the partitions. It's possible
+ * the partitions are not a perfect multiple of page size, in which case
+ * we set location only for the part where that is possible. The buffers on
+ * the "boundary" may get located up on arbitrary nodes.
+ *
+ * The extra complexity of figuring out the right "partition size" is not
+ * worth it, and it can lead to some partitions being much smaller. This way
+ * we end up with partitions of almost exactly the same size (one BLCKSZ is
+ * the largest difference).
+ *
+ * We expect shared buffers to be much larger than page size (at least on
+ * system where NUMA is a relevant feature), so the number of "not located"
+ * buffers should be a negligible fraction. This only affects pages between
+ * partitions for different nodes, so (nodes-1) pages. This is certainly
+ * fine with 2MB huge pages, but even with 1GB pages it should be OK (as
+ * such systems should have humongous amounts of memory).
+ *
+ * It also means we don't need to worry about memory page size before knowing
+ * if huge pages got used (which we only learn during allocation).
+ */
+void
+BufferPartitionsCalculate(int *num_nodes, int *num_partitions,
+						  int *num_partitions_per_node)
+{
+	int		nnodes,
+			nparts,
+			nparts_per_node;
+
+#if USE_LIBNUMA
+	nnodes = numa_num_configured_nodes();
+	nparts_per_node = 1;	/* at least one partition per node */
+
+	while ((nparts_per_node * nnodes) < MIN_BUFFER_PARTITIONS)
+		nparts_per_node++;
+
+	nparts = (nnodes * nparts_per_node);
+#else
+	/* without NUMA, assume there's just one node */
+	nnodes = 1;
+	nparts = MIN_BUFFER_PARTITIONS;
+	nparts_per_node = MIN_BUFFER_PARTITIONS;
+#endif
+
+	if (num_nodes)
+		*num_nodes = nnodes;
+
+	if (num_partitions)
+		*num_partitions = nparts;
+
+	if (num_partitions_per_node)
+		*num_partitions_per_node = nparts_per_node;
+}
+
 /*
  * BufferPartitionCount
  *		Returns the number of partitions created.
@@ -277,13 +464,14 @@ BufferPartitionCount(void)
  * The returned information is first/last buffer, number of buffers.
  */
 void
-BufferPartitionGet(int idx, int *num_buffers,
+BufferPartitionGet(int idx, int *node, int *num_buffers,
 				   int *first_buffer, int *last_buffer)
 {
 	if ((idx >= 0) && (idx < BufferPartitionsRegistry->npartitions))
 	{
 		BufferPartition *part = &BufferPartitionsRegistry->partitions[idx];
 
@@ -293,3 +481,17 @@ BufferPartitionGet(int idx, int *num_buffers,
 
 	elog(ERROR, "invalid partition index");
 }
+
+void
+BufferPartitionsParams(int *num_nodes, int *num_partitions,
+					   int *num_partitions_per_node)
+{
+	if (num_nodes)
+		*num_nodes = BufferPartitionsRegistry->nnodes;
+
+	if (num_partitions)
+		*num_partitions = BufferPartitionsRegistry->npartitions;
+
+	if (num_partitions_per_node)
+		*num_partitions_per_node = BufferPartitionsRegistry->npartitions_per_node;
+}
diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index fdb5bad7910..53ef5239e8d 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -15,6 +15,15 @@
  */
 #include "postgres.h"
 
+#ifdef USE_LIBNUMA
+#include <sched.h>
+#endif
+
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
 #include "pgstat.h"
index f15e74198c5..2e71c04282c 100644
@@ -2724,6 +2724,12 @@
   max => 'INT_MAX / 2',
 },
 
+{ name => 'shared_buffers_numa', type => 'bool', context => 'PGC_POSTMASTER', group => 'RESOURCES_MEM',
+  short_desc => 'Locate partitions of shared buffers (and descriptors) to NUMA nodes.',
+  variable => 'shared_buffers_numa',
+  boot_val => 'false',
+},
+
 { name => 'shared_memory_size', type => 'int', context => 'PGC_INTERNAL', group => 'PRESET_OPTIONS',
   short_desc => 'Shows the size of the server\'s main shared memory area (rounded up to the nearest MB).',
   flags => 'GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE | GUC_UNIT_MB | GUC_RUNTIME_COMPUTED',
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index ac38cddaaf9..c0f79c779cc 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -142,6 +142,7 @@
 #temp_buffers = 8MB                     # min 800kB
 #max_prepared_transactions = 0          # zero disables the feature
                                         # (change requires restart)
+#shared_buffers_numa = off              # NUMA-aware partitioning
 # Caution: it is not advisable to set max_prepared_transactions nonzero unless
 # you actively intend to use prepared transactions.
 #work_mem = 4MB                         # min 64kB
diff --git a/src/include/port/pg_numa.h b/src/include/port/pg_numa.h
index 1b668fe1d91..8fe4d4ab7e3 100644
--- a/src/include/port/pg_numa.h
+++ b/src/include/port/pg_numa.h
@@ -17,6 +17,13 @@
 extern PGDLLIMPORT int pg_numa_init(void);
 extern PGDLLIMPORT int pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status);
 extern PGDLLIMPORT int pg_numa_get_max_node(void);
+extern PGDLLIMPORT Size pg_numa_page_size(void);
+extern PGDLLIMPORT void pg_numa_move_to_node(char *startptr, char *endptr, int node);
+extern PGDLLIMPORT int pg_numa_bind_to_node(char *startptr, char *endptr, int node);
+
+extern PGDLLIMPORT int numa_flags;
+
+#define		NUMA_BUFFERS		0x01
 
 #ifdef USE_LIBNUMA
 
index e5a887b9969..e944cee2e91 100644
@@ -365,10 +365,10 @@ typedef struct BufferDesc
@@ -378,7 +378,7 @@ typedef struct BufferDesc
 
 typedef union BufferDescPadded
 {
@@ -416,8 +416,12 @@ extern PGDLLIMPORT ConditionVariableMinimallyPadded *BufferIOCVArray;
 extern PGDLLIMPORT WritebackContext BackendWritebackContext;
 
 extern int	BufferPartitionCount(void);
-extern void BufferPartitionGet(int idx, int *num_buffers,
+extern void BufferPartitionGet(int idx, int *node, int *num_buffers,
 							   int *first_buffer, int *last_buffer);
+extern void BufferPartitionsCalculate(int *num_nodes, int *num_partitions,
+									  int *num_partitions_per_node);
+extern void BufferPartitionsParams(int *num_nodes, int *num_partitions,
+								   int *num_partitions_per_node);
 
 /* in localbuf.c */
 extern PGDLLIMPORT BufferDesc *LocalBufferDescriptors;
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index 79a3f44747a..1cf09e8fb7c 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -158,10 +158,12 @@ typedef struct ReadBuffersOperation ReadBuffersOperation;
 /*
  * information about one partition of shared buffers
  *
+ * numa_nod specifies node for this partition (-1 means allocated on any node)
  * first/last buffer - the values are inclusive
  */
 typedef struct BufferPartition
 {
+	int			numa_node;		/* NUMA node (-1 no node) */
 	int			num_buffers;	/* number of buffers */
 	int			first_buffer;	/* first buffer of partition */
 	int			last_buffer;	/* last buffer of partition */
@@ -170,7 +172,9 @@ typedef struct BufferPartition
 /* an array of information about all partitions */
 typedef struct BufferPartitions
 {
+	int			nnodes;			/* number of NUMA nodes */
 	int			npartitions;	/* number of partitions */
+	int			npartitions_per_node;	/* for convenience */
 	BufferPartition partitions[FLEXIBLE_ARRAY_MEMBER];
 } BufferPartitions;
 
@@ -206,6 +210,7 @@ extern PGDLLIMPORT const PgAioHandleCallbacks aio_local_buffer_readv_cb;
 
 /* in buf_init.c */
 extern PGDLLIMPORT char *BufferBlocks;
+extern PGDLLIMPORT bool shared_buffers_numa;
 
 /* in localbuf.c */
 extern PGDLLIMPORT int NLocBuffer;
@@ -390,6 +395,9 @@ extern void MarkDirtyAllUnpinnedBuffers(int32 *buffers_dirtied,
 										int32 *buffers_already_dirty,
 										int32 *buffers_skipped);
 
+/* in buf_init.c */
 
diff --git a/src/port/pg_numa.c b/src/port/pg_numa.c
index 8954669273a..11c8a4503da 100644
--- a/src/port/pg_numa.c
+++ b/src/port/pg_numa.c
@@ -18,6 +18,9 @@
 
 #include "miscadmin.h"
 
 /*
  * At this point we provide support only for Linux thanks to libnuma, but in
@@ -118,6 +121,94 @@ pg_numa_get_max_node(void)
 	return numa_max_node();
 }
 
+/*
+ * pg_numa_move_to_node
+ *		move memory to different NUMA nodes in larger chunks
+ *
+ * startptr - start of the region (should be aligned to page size)
+ * endptr - end of the region (doesn't need to be aligned)
+ * node - node to move the memory to
+ *
+ * The "startptr" is expected to be a multiple of system memory page size, as
+ * determined by pg_numa_page_size.
+ *
+ * XXX We only expect to do this during startup, when the shared memory is
+ * still being setup.
+ */
+void
+pg_numa_move_to_node(char *startptr, char *endptr, int node)
+{
+	Size		sz = (endptr - startptr);
+
+	Assert((int64) startptr % pg_numa_page_size() == 0);
+
+	/*
+	 * numa_tonode_memory does not actually cause a page fault, and thus does
+	 * not locate the memory on the node. So it's fast, at least compared to
+	 * pg_numa_query_pages, and does not make startup longer. But it also
+	 * means the expensive part happen later, on the first access.
+	 */
+	numa_tonode_memory(startptr, sz, node);
+}
+
+int
+pg_numa_bind_to_node(char *startptr, char *endptr, int node)
+{
+	int				ret;
+	struct bitmask *nodemask;
+
+	if (node < 0)
+	{
+		errno = EINVAL;
+		return -1;
+	}
+
+	nodemask = numa_allocate_nodemask();
+	if (nodemask == NULL)
+	{
+		errno = ENOMEM;
+		return -1;
+	}
+
+	numa_bitmask_setbit(nodemask, node);
+
+	/*
+	 * MPOL_BIND places the pages strictly on the node, and MPOL_MF_MOVE migrates
+	 * pages already faulted in to that node. If mbind() fails, leave the default
+	 * placement in effect, and report the failure.
+	 */
+	ret = mbind(startptr, (endptr - startptr),
+				MPOL_BIND, nodemask->maskp, node, MPOL_MF_MOVE);
+
+	numa_free_nodemask(nodemask);
+
+	return ret;
+}
+
+Size
+pg_numa_page_size(void)
+{
+	Size		os_page_size;
+	Size		huge_page_size;
+
+#ifdef WIN32
+	SYSTEM_INFO sysinfo;
+
+	GetSystemInfo(&sysinfo);
+	os_page_size = sysinfo.dwPageSize;
+#else
+	os_page_size = sysconf(_SC_PAGESIZE);
+#endif
+
+	/* assume huge pages get used, unless HUGE_PAGES_OFF */
+	if (huge_pages_status != HUGE_PAGES_OFF)
+		GetHugePageSize(&huge_page_size, NULL);
+	else
+		huge_page_size = 0;
+
+	return Max(os_page_size, huge_page_size);
+}
+
 #else
 
 /* Empty wrappers */
@@ -140,4 +231,25 @@ pg_numa_get_max_node(void)
 	return 0;
 }
 
+void
+pg_numa_move_to_node(char *startptr, char *endptr, int node)
+{
+	/* we don't expect to ever get here in builds without libnuma */
+	Assert(false);
+}
+
+int
+pg_numa_bind_to_node(char *startptr, char *endptr, int node)
+{
+	/* we don't expect to ever get here in builds without libnuma */
+	Assert(false);
+}
+
+Size
+pg_numa_page_size(void)
+{
+	/* we don't expect to ever get here in builds without libnuma */
+	Assert(false);
+}
+
 #endif
-- 
2.54.0

From 052ac2256afbba3f25f20f683449a0bbb0af4241 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 2 Jun 2026 22:09:33 +0200
Subject: [PATCH v20260605 2/6] Infrastructure for partitioning of shared
 buffers

The patch introduces a simple "registry" of buffer partitions, keeping
track of the first/last buffer, etc. This serves as a source of truth
for later patches (e.g. to partition clock-sweep or to make the
partitioning NUMA-aware).

The registry is a small array of BufferPartition entries in shared
memory, with partitions sized to be a fair share of shared buffers.

Notes:

* The buffers are divided as evenly as possible, with the first couple
  partitions possibly getting an extra buffer.
---
 contrib/pg_buffercache/Makefile               |   3 +-
 .../pg_buffercache--1.7--1.8.sql              |  23 +++
 contrib/pg_buffercache/pg_buffercache.control |   2 +-
 contrib/pg_buffercache/pg_buffercache_pages.c |  86 +++++++++++
 src/backend/storage/buffer/buf_init.c         | 142 ++++++++++++++++++
 src/include/storage/buf_internals.h           |   5 +
 src/include/storage/bufmgr.h                  |  19 +++
 src/tools/pgindent/typedefs.list              |   2 +
 8 files changed, 280 insertions(+), 2 deletions(-)
 create mode 100644 contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql

diff --git a/contrib/pg_buffercache/Makefile b/contrib/pg_buffercache/Makefile
index 0e618f66aec..7fd5cdfc43d 100644
--- a/contrib/pg_buffercache/Makefile
+++ b/contrib/pg_buffercache/Makefile
@@ -9,7 +9,8 @@ EXTENSION = pg_buffercache
 DATA = pg_buffercache--1.2.sql pg_buffercache--1.2--1.3.sql \
 	pg_buffercache--1.1--1.2.sql pg_buffercache--1.0--1.1.sql \
 	pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql \
-	pg_buffercache--1.5--1.6.sql pg_buffercache--1.6--1.7.sql
+	pg_buffercache--1.5--1.6.sql pg_buffercache--1.6--1.7.sql \
+	pg_buffercache--1.7--1.8.sql
 PGFILEDESC = "pg_buffercache - monitoring of shared buffer cache in real-time"
 
index bf2e6c97220..d678cb045ba 100644
@@ -31,6 +31,7 @@
 #define NUM_BUFFERCACHE_MARK_DIRTY_ALL_ELEM 3

 PG_MODULE_MAGIC_EXT(
 					.name = "pg_buffercache",
@@ -75,6 +76,7 @@ PG_FUNCTION_INFO_V1(pg_buffercache_evict_all);
 PG_FUNCTION_INFO_V1(pg_buffercache_mark_dirty);
 PG_FUNCTION_INFO_V1(pg_buffercache_mark_dirty_relation);
 PG_FUNCTION_INFO_V1(pg_buffercache_mark_dirty_all);
+PG_FUNCTION_INFO_V1(pg_buffercache_partitions);

 /* Only need to touch memory once per backend process lifetime */
@@ -871,3 +873,87 @@ pg_buffercache_mark_dirty_all(PG_FUNCTION_ARGS)
 
 	PG_RETURN_DATUM(result);
 }
+
+/*
+ * Inquire about partitioning of shared buffers.
+ */
+Datum
index 1407c930c56..e593b02e0ca 100644
@@ -26,10 +26,12 @@ char	   *BufferBlocks;
 ConditionVariableMinimallyPadded *BufferIOCVArray;
+BufferPartitions *BufferPartitionsRegistry;
 
 static void BufferManagerShmemRequest(void *arg);
 static void BufferManagerShmemInit(void *arg);
 static void BufferManagerShmemAttach(void *arg);
+static void BufferPartitionsInit(void);
 
 const ShmemCallbacks BufferManagerShmemCallbacks = {
 	.request_fn = BufferManagerShmemRequest,
@@ -69,6 +71,9 @@ const ShmemCallbacks BufferManagerShmemCallbacks = {
  *		multiple times. Check the PrivateRefCount infrastructure in bufmgr.c.
  */
 
+/* number of buffer partitions */
+#define NUM_CLOCK_SWEEP_PARTITIONS	4
+
 
 /*
  * Register shared memory area for the buffer pool.
@@ -97,6 +102,13 @@ BufferManagerShmemRequest(void *arg)
 					   .ptr = (void **) &BufferIOCVArray,
 		);
 
+	ShmemRequestStruct(.name = "Buffer Partition Registry",
+					   .size = NUM_CLOCK_SWEEP_PARTITIONS * sizeof(BufferPartition),
+	/* Align descriptors to a cacheline boundary. */
+					   .alignment = PG_CACHE_LINE_SIZE,
+					   .ptr = (void **) &BufferPartitionsRegistry,
+		);
+
 	/*
 	 * The array used to sort to-be-checkpointed buffer ids is located in
 	 * shared memory, to avoid having to allocate significant amounts of
@@ -119,6 +131,12 @@ BufferManagerShmemRequest(void *arg)
 static void
 BufferManagerShmemInit(void *arg)
 {
+	/*
+	 * Initialize the buffer partition registry first, before other parts
+	 * have a chance to touch the memory.
+	 */
+	BufferPartitionsInit();
+
 	/*
 	 * Initialize all the buffer headers.
 	 */
@@ -151,3 +169,127 @@ BufferManagerShmemAttach(void *arg)
 	WritebackContextInit(&BackendWritebackContext,
 						 &backend_flush_after);
 }
+
+/*
+ * Sanity checks of buffers partitions - there must be no gaps, it must cover
+ * the whole range of buffers, etc.
+ */
+static void
+AssertCheckBufferPartitions(void)
+{
+#ifdef USE_ASSERT_CHECKING
+	int			num_buffers = 0;
+
+	Assert(BufferPartitionsRegistry->npartitions > 0);
+
+	for (int i = 0; i < BufferPartitionsRegistry->npartitions; i++)
+	{
+		BufferPartition *part = &BufferPartitionsRegistry->partitions[i];
+
+		/*
+		 * We can get a single-buffer partition, if the sizing forces the last
+		 * partition to be just one buffer. But it's unlikely (and
+		 * undesirable).
+		 */
+		Assert(part->first_buffer <= part->last_buffer);
+		Assert((part->last_buffer - part->first_buffer + 1) == part->num_buffers);
+
+		num_buffers += part->num_buffers;
+
+		/*
+		 * The first partition needs to start on buffer 0. Later partitions
+		 * need to be contiguous, without skipping any buffers.
+		 */
+		if (i == 0)
+		{
+			Assert(part->first_buffer == 0);
+		}
+		else
+		{
+			BufferPartition *prev = &BufferPartitionsRegistry->partitions[i - 1];
+
+			Assert((part->first_buffer - 1) == prev->last_buffer);
+		}
+
+		/* the last partition needs to end on buffer (NBuffers - 1) */
+		if (i == (BufferPartitionsRegistry->npartitions - 1))
+		{
+			Assert(part->last_buffer == (NBuffers - 1));
+		}
+	}
+
+	Assert(num_buffers == NBuffers);
+#endif
+}
+
+/*
+ * BufferPartitionsInit
+ *		Initialize registry of buffer partitions.
+ */
+static void
+BufferPartitionsInit(void)
+{
+	int			buffer = 0;
+
+	/* number of buffers per partition (make sure to not overflow) */
+	int			part_buffers = NBuffers / NUM_CLOCK_SWEEP_PARTITIONS;
+	int			remaining_buffers = NBuffers % NUM_CLOCK_SWEEP_PARTITIONS;
+
+	BufferPartitionsRegistry->npartitions = NUM_CLOCK_SWEEP_PARTITIONS;
+
+	for (int n = 0; n < BufferPartitionsRegistry->npartitions; n++)
+	{
+		BufferPartition *part = &BufferPartitionsRegistry->partitions[n];
+
+		int			num_buffers = part_buffers;
+		if (n < remaining_buffers)
+			num_buffers += 1;
+
+		remaining_buffers -= num_buffers;
+
+		Assert((num_buffers > 0) && (num_buffers <= part_buffers));
+		Assert((buffer >= 0) && (buffer < NBuffers));
+
+		part->num_buffers = num_buffers;
+		part->first_buffer = buffer;
+		part->last_buffer = buffer + (num_buffers - 1);
+
+		buffer += num_buffers;
+	}
+
+	AssertCheckBufferPartitions();
+}
+
+/*
+ * BufferPartitionCount
+ *		Returns the number of partitions created.
+ */
+int
+BufferPartitionCount(void)
+{
+	return BufferPartitionsRegistry->npartitions;
+}
+
+/*
+ * BufferPartitionGet
+ *		Returns information about a partition at the provided index.
+ *
+ * The returned information is first/last buffer, number of buffers.
+ */
+void
+BufferPartitionGet(int idx, int *num_buffers,
+				   int *first_buffer, int *last_buffer)
+{
+	if ((idx >= 0) && (idx < BufferPartitionsRegistry->npartitions))
+	{
+		BufferPartition *part = &BufferPartitionsRegistry->partitions[idx];
+
+		*num_buffers = part->num_buffers;
+		*first_buffer = part->first_buffer;
+		*last_buffer = part->last_buffer;
+
+		return;
+	}
+
+	elog(ERROR, "invalid partition index");
+}
diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h
index 89615a254a3..e5a887b9969 100644
--- a/src/include/storage/buf_internals.h
+++ b/src/include/storage/buf_internals.h
@@ -411,9 +411,14 @@ typedef struct WritebackContext
 
 /* in buf_init.c */
 extern PGDLLIMPORT BufferDescPadded *BufferDescriptors;
+extern PGDLLIMPORT BufferPartitions *BufferPartitionsRegistry;
 extern PGDLLIMPORT ConditionVariableMinimallyPadded *BufferIOCVArray;
 extern PGDLLIMPORT WritebackContext BackendWritebackContext;
 
+extern int	BufferPartitionCount(void);
+extern void BufferPartitionGet(int idx, int *num_buffers,
+							   int *first_buffer, int *last_buffer);
+
 /* in localbuf.c */
 extern PGDLLIMPORT BufferDesc *LocalBufferDescriptors;
 
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index 6837b35fc6d..79a3f44747a 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -155,6 +155,25 @@ struct ReadBuffersOperation

+/*
 
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 8cf40c87043..ea756015249 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -365,6 +365,8 @@ BufferHeapTupleTableSlot
 BufferLockMode
 BufferLookupEnt
 BufferManagerRelation
+BufferPartition
+BufferPartitions
 BufferStrategyControl
 BufferTag
 BufferUsage
-- 
2.54.0

From 238935f06c793dcd0271423af83e039d2ee00120 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 3 Jun 2026 16:30:28 +0200
Subject: [PATCH v20260605 1/6] Add shmem_populate and shmem_interleave GUCs

- shmem_populate - Forces mmap() with MAP_POPULATE, which faults all
  memory pages backing the shared memory segment.

- shmem_interleave - Applies NUMA interleaving on the whole shared
  memory segment, to balance allocations between nodes.
---
 src/backend/port/sysv_shmem.c             | 47 +++++++++++++++++++++++
 src/backend/utils/misc/guc_parameters.dat | 14 +++++++
 src/include/miscadmin.h                   |  4 ++
 3 files changed, 65 insertions(+)

diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 2e3886cf9fe..9eaff838a04 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -27,6 +27,10 @@
 #include <sys/shm.h>
 #include <sys/stat.h>
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#endif
+
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
 #include "portability/mem.h"
@@ -98,6 +102,10 @@ void	   *UsedShmemSegAddr = NULL;
 static Size AnonymousShmemSize;
 static void *AnonymousShmem = NULL;
 
+/* GUCs */
+bool shmem_populate = false;	/* MAP_POPULATE */
+bool shmem_interleave = false;	/* NUMA interleaving */
+
 static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
 static void IpcMemoryDetach(int status, Datum shmaddr);
 static void IpcMemoryDelete(int status, Datum shmId);
@@ -604,6 +612,21 @@ CreateAnonymousSegment(Size *size)
 	int			mmap_errno = 0;
 	int			mmap_flags = MAP_SHARED | MAP_ANONYMOUS | MAP_HASSEMAPHORE;
 
+	/* If requested, populate the shared memory by MAP_POPULATE. */
+	if (shmem_populate)
+		mmap_flags |= MAP_POPULATE;
+
+#ifdef USE_LIBNUMA
+	/*
+	 * If requested, interleave the shared memory by setting a memory policy
+	 * before the mmap() call. This really matters only with MAP_POPULATE,
+	 * because without page faults the memory does not actually get placed
+	 * to the nodes. But without MAP_POPULATE it's virtually free.
+	 */
+	if (shmem_interleave)
+		numa_set_interleave_mask(numa_all_nodes_ptr);
+#endif
+
 #ifndef MAP_HUGETLB
 	/* PGSharedMemoryCreate should have dealt with this case */
 	Assert(huge_pages != HUGE_PAGES_ON);
@@ -665,6 +688,30 @@ CreateAnonymousSegment(Size *size)
 						 allocsize) : 0));
 	}
 
+#ifdef USE_LIBNUMA
+	/*
+	 * If set the policy to interleaving by numa_set_membind(), undo it now by
+	 * setting the policy to localalloc. With MAP_POPULATE, all the pages were
+	 * faulted and are now interleaved on the available nodes.
+	 *
+	 * To handle the case without MAP_POPULATE, apply the interleaving policy to
+	 * the shared memory segment allocated by mmap() before touching it in any
+	 * way, so that it gets placed on the correct node on first access.
+	 *
+	 * This matters especially with huge pages, where it's possible to run out
+	 * of huge pages on some nodes and then crash. By explicitly interleaving
+	 * the whole segment, that's much less likely.
+	 */
+	if (shmem_interleave)
+	{
+		/* undo the policy set by numa_set_membind() earlier */
+		numa_set_localalloc();
+
+		/* set interleaving policy for not yet faulted memory */
+		numa_interleave_memory(ptr, allocsize, numa_all_nodes_ptr);
+	}
+#endif
+
 	*size = allocsize;
 	return ptr;
 }
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index afaa058b046..f15e74198c5 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -740,6 +740,20 @@
   ifdef => 'DEBUG_NODE_TESTS_ENABLED',
 },
 
+{ name => 'debug_shmem_interleave', type => 'bool', context => 'PGC_SUSET', group => 'DEVELOPER_OPTIONS',
+  short_desc => 'Forces interleaving for the whole shared memory segment.',
+  flags => 'GUC_NOT_IN_SAMPLE',
+  variable => 'shmem_interleave',
+  boot_val => 'false'
+},
+
+{ name => 'debug_shmem_populate', type => 'bool', context => 'PGC_SUSET', group => 'DEVELOPER_OPTIONS',
+  short_desc => 'Populates (faults) the whole shared memory segment using MAP_POPULATE.',
+  flags => 'GUC_NOT_IN_SAMPLE',
+  variable => 'shmem_populate',
+  boot_val => 'false'
+},
+
 { name => 'debug_write_read_parse_plan_trees', type => 'bool', context => 'PGC_SUSET', group => 'DEVELOPER_OPTIONS',
   short_desc => 'Set this to force all parse and plan trees to be passed through outfuncs.c/readfuncs.c, to facilitate catching errors and omissions in those modules.',
   flags => 'GUC_NOT_IN_SAMPLE',
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 7de0a115402..13bb29f8702 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -213,6 +213,10 @@ extern PGDLLIMPORT Oid MyDatabaseTableSpace;
 
 extern PGDLLIMPORT bool MyDatabaseHasLoginEventTriggers;
 
+extern PGDLLIMPORT bool shmem_populate;
+extern PGDLLIMPORT bool shmem_interleave;
+
+
 /*
  * Date/Time Configuration
  *
-- 
2.54.0

---

From: Jakub Wartak <[email-masked]>
Date: Tue, 16 Jun 2026 10:16:00 +0200

> Hi,

Hi Tomas, thanks for working on this.

[..]

1. 005 says:

but there isn't direct any check for checking if pg_numa_get_max_node() ->
numa_max_node() is not getting higher than allowed here. In theory this could
happen I think if ClockSweepPartitionIndex() would return
    numa = numa_node_of_cpu()
on some hypothethical very high-end setup (with plenty of sub-NUMA nodes)
and that would cause accesing .balance[] without bounds.

2. If we have in 004 struct ClockSweep with nextVictimBuffer, shouldn't
this be padded/aligned somehow later in BufferStrategyControl which does
    ClockSweep      sweeps[FLEXIBLE_ARRAY_MEMBER];
to avoid contention/false sharing? (comments says it should be but it
doesn't seem so?), maybe the comment should be TODO for now? I have not
quantified any potential benefit

With pahole after some hassle I've got:
 struct ClockSweep {
        slock_t                    clock_sweep_lock;     /*     0     1 */

        /* XXX 3 bytes hole, try to pack */

        int32                      node;                 /*     4     4 */
        int32                      firstBuffer;          /*     8     4 */
        int32                      numBuffers;           /*    12     4 */
        pg_atomic_uint32           nextVictimBuffer;     /*    16     4 */
        uint32                     completePasses;       /*    20     4 */
        pg_atomic_uint32           numBufferAllocs;      /*    24     4 */
        pg_atomic_uint32           numRequestedAllocs;   /*    28     4 */
        pg_atomic_uint64           numTotalAllocs;       /*    32     8 */
        pg_atomic_uint64           numTotalRequestedAllocs; /*    40     8 */
        uint8                      balance[32];          /*    48    32 */

        /* size: 80, cachelines: 2, members: 11 */
        /* sum members: 77, holes: 1, sum holes: 3 */
        /* last cacheline: 16 bytes */
 };
maybe with smaller MAX_BUFFER_PARTITIONS we could pack this into size=64 ?

3. In 004 sched_getcpu() is used and mentioned how to check if it is available

But my $0.02 (maybe not that important), but I've at least saw once where
(on EC2?) some clock_gettime() was very slow and that was because it was not
available in VDSO. It's usually some mix of kernel <-> arch <-> libc (not
always glibc?) compatibility matrix issue. My worry is that StrategyGetBuffer()
-> ChooseClockSweep() -> ClockSweepPartitionIndex() -> sched_getcpu() would be
available, but slow and it would mean real syscall price (and that's not once
there per buffer). I'm also somehow thinking other platforms (FreeBSD comes to
mind, but I haven't checked further). The point is: wouldn't it be cheaper
that to be refreshed from time to time instead otherwise we risk some slow
code on non-x86_64, but I doubt how proliferated is e.g. ARM64 with NUMA..
Or alternative is to have pg_test_numa proggie and this would be measuring
certain things about NUMA including timing of sched_getcpu (just like
pg_test_timing does for time), at least that could explain why somebody's
system/platform is slow.

4. Patch has problem (without fix for #8) that when number of available huge
pages in the OS is greatly higher than shared_memory_size_in_huge_pages it
will use only first NUMA node. This might be a problem when starting mulitple
DBs (they will occupy first available NUMA):

### with s_b=8GB and nr_hugepages=1500 it's OK

# find /sys/devices/system/node/ -name nr_hugepages -exec grep -H . {}
\; | grep 2048 | sort
/sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages:1250
/sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages:1250
/sys/devices/system/node/node2/hugepages/hugepages-2048kB/nr_hugepages:1250
/sys/devices/system/node/node3/hugepages/hugepages-2048kB/nr_hugepages:1250

## note the correct split below for N0/N1..
# grep huge /proc/`pgrep -f /usr/pgsql19/bin/postgres`/numa_maps
7fb1b4400000 default file=/anon_hugepage\040(deleted) huge dirty=4269
mapmax=6 N0=1250 N1=1250 N2=519 N3=1250 kernelpagesize_kB=2048

### still s_b=8GB but nr_hugepages = 19000 (~37GB), it ends all on N0=4269
# find /sys/devices/system/node/ -name nr_hugepages -exec grep -H . {}
\; | grep 2048 | sort
/sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages:4750
/sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages:4750
/sys/devices/system/node/node2/hugepages/hugepages-2048kB/nr_hugepages:4750
/sys/devices/system/node/node3/hugepages/hugepages-2048kB/nr_hugepages:4750
## all on N0...
# grep huge /proc/`pgrep -f /usr/pgsql19/bin/postgres`/numa_maps
7ff3a7a00000 default file=/anon_hugepage\040(deleted) huge dirty=4269
mapmax=6 N0=4269 kernelpagesize_kB=2048

I was even thinking go to lengths and add code for inspecting that /sys on
some later date that the kernel NUMA hugepages are really distributed
on the nodes as they should be (it's easy to end up on just 1 node out of
many; allocating via sysctl -w <higher> and then <lower> allocation is easy
way to force hugepages just to 1 node instead of many :o). I've hit the
problem multiple times, so we should bail out if we want NUMA and the
Buffer Blocks were just put on 1 node (instead of many).

5. In 005 we could mention more clealry what's the difference between
those 3: numRequestedAllocs, numTotalAllocs, numTotalRequestedAllocs
in the defintion to make it easier to read, maybe copy-cat those earlier
descriptions there too as we already have:

6. While at it, it would be helpful if we could reset the
pg_buffercache_partitions stats in some way (pg_buffercache_partitions
is very usefull).. or is there way to plug into main pg_reset functions?

7. If I add basic error checking for mbind() then it complains a lot, like
below with annotated strace -ffe mbind to show the point:

[pid  2856] mbind(0x7fd8d0e00000, 2145386496, MPOL_BIND, [], 0,
MPOL_MF_MOVE) = -1 EINVAL (Invalid argument)
WARNING:  mbind(): Invalid argument
WARNING:  buffers descriptors for node 0 not well aligned
[0x7fd8cccf5000,0x7fd8cdcf4fc1] aligned
[0x7fd8cce00000,0x7fd8cdc00000]

[pid  2856] mbind(0x7fd8cce00000, 14680064, MPOL_BIND, [], 0,
MPOL_MF_MOVE) = -1 EINVAL (Invalid argument)
WARNING:  mbind(): Invalid argument
WARNING:  buffers for node 1 not well aligned
[0x7fd950cf5000,0x7fd9d0cf5000] aligned
[0x7fd950e00000,0x7fd9d0c00000]

[pid  2856] mbind(0x7fd950e00000, 2145386496, MPOL_BIND,
0x5589057ded00, 1, MPOL_MF_MOVE) = -1 EINVAL (Invalid argument)
WARNING:  mbind(): Invalid argument
WARNING:  buffers descriptors for node 1 not well aligned
[0x7fd8cdcf5000,0x7fd8cecf4fc1] aligned
[0x7fd8cde00000,0x7fd8cec00000]
[..]

but with pg_numa.c fixed like below (node should be size):
        ret = mbind(startptr, (endptr - startptr),
-                               MPOL_BIND, nodemask->maskp, node, MPOL_MF_MOVE);
+                               MPOL_BIND, nodemask->maskp,
nodemask->size, MPOL_MF_MOVE);

it doesn't report errors anymore and suprisngly hugepages in numa_maps are
altered from:
7fb1b4400000 default file=/anon_hugepage\040(deleted) huge dirty=4269
mapmax=6 N0=1250 N1=1250 N2=519 N3=1250 kernelpagesize_kB=2048

to explicit "binds":
7f8540000000 default file=/anon_hugepage\040(deleted) huge dirty=25
mapmax=6 N0=25 kernelpagesize_kB=2048
7f8543200000 bind:0 file=/anon_hugepage\040(deleted) huge dirty=7
mapmax=3 N0=7 kernelpagesize_kB=2048
7f8544000000 default file=/anon_hugepage\040(deleted) huge dirty=1
mapmax=2 N0=1 kernelpagesize_kB=2048
7f8544200000 bind:1 file=/anon_hugepage\040(deleted) huge dirty=7
mapmax=2 N1=7 kernelpagesize_kB=2048
7f8545000000 default file=/anon_hugepage\040(deleted) huge dirty=1
mapmax=2 N0=1 kernelpagesize_kB=2048
7f8545200000 bind:2 file=/anon_hugepage\040(deleted) huge dirty=7
mapmax=2 N2=7 kernelpagesize_kB=2048
7f8546000000 default file=/anon_hugepage\040(deleted) huge dirty=1
mapmax=2 N0=1 kernelpagesize_kB=2048
7f8546200000 bind:3 file=/anon_hugepage\040(deleted) huge dirty=7
mapmax=2 N3=7 kernelpagesize_kB=2048
7f8547000000 default file=/anon_hugepage\040(deleted) huge dirty=1
mapmax=3 N0=1 kernelpagesize_kB=2048
7f8547200000 bind:0 file=/anon_hugepage\040(deleted) huge dirty=1023
mapmax=2 N0=1023 kernelpagesize_kB=2048
7f85c7000000 default file=/anon_hugepage\040(deleted) huge dirty=1
N0=1 kernelpagesize_kB=2048
7f85c7200000 bind:1 file=/anon_hugepage\040(deleted) huge dirty=1023
N1=1023 kernelpagesize_kB=2048
7f8647000000 default file=/anon_hugepage\040(deleted) huge dirty=1
N1=1 kernelpagesize_kB=2048
7f8647200000 bind:2 file=/anon_hugepage\040(deleted) huge dirty=1023
N2=1023 kernelpagesize_kB=2048
7f86c7000000 default file=/anon_hugepage\040(deleted) huge dirty=1
N3=1 kernelpagesize_kB=2048
7f86c7200000 bind:3 file=/anon_hugepage\040(deleted) huge dirty=1023
N3=1023 kernelpagesize_kB=2048
7f8747000000 default file=/anon_hugepage\040(deleted) huge dirty=117
mapmax=6 N2=117 kernelpagesize_kB=2048

so lots of VMAs were created (it could affect performance in some way, I think
for sure it would affect for worse fork() rates by postmaster for new conns).

To me it looks like there's plenty of "N[0..3]=1" with "default" indicating
single HP page being somehow left/missed in address calculations, but I
haven't pressed this harder.

NOTE: the patch works even without this fix, but I believe if got non-0 we
cannot reliably trust the optimizer memory layout has been deployed (I suspect
it's some kind luck it sharded the shm based on number of hugepages available)

> balancing algorithm of some sort).

8. The crux of this email and stuff I wanted to further discuss, when server
is started with on this 4-NUMA box with
* numactl --cpunodebind=0 pg_ctl start # so that all backends fork()ing will be
  on node#0
* the shm split onto 4 nodes properly
* s_b still just 8GB (with ideal split),
* DB size ~15GB with 8 pgbench partitions (and fully in VFS cache)
* pgbench -c 8 -j 8 postgres -T 20 -P 1 -f seqconcurrscans.pgb with:
  \set num (:client_id % 8) + 1
  select sum(octet_length(filler)) from pgbench_accounts_:num;
* mpstat repors correctly just node#0 used

a. with the patch for GUCs with numa on and defaults two clocksweep settings
   on, I'm getting:

   latency average = 3252.254 ms
   latency stddev = 72.011 ms

b. with debug_clocksweep_balance=off, I'm realiably getting

   latency average = 2688.742 ms
   latency stddev = 61.738 ms

so IMHO clocksweep partitioning is cool, but if we are discussing the current
balancing logic leaves some juice on the table from the most optimized variant
(~1.2x) with ~90ns:270ns (local vs remote latency). In the picture above it
was 8 backends accessing 8x 1.6GB tables (lower than NBuffers / 4).

Dunno if it should be optimized further, certainly we'll get reports from
quick benchmarks run by people that PG 20 could be *slower* because.. well,
they got (sub)optimal layout during startup (all HP on 1 node and some
query hitting just that one query with local affinity and this is visible
to naked eye). I was re-reading thread and Andres also wrote "We should use
the partitioned clock sweep to default to using local memory as long as
possible."

So two further ideas:

I. BufferAccessStrategy: we could derrive affinity from the BAS strategy
itself, couldn't we? If we are using capped ring buffer, we could indicate
that we want it just from local node as priority disregarding weights (?).
Same goes for BAS_VACUUM (why would one it on remote NUMA?). With BAS_BULKREAD
there would be some potential issue with sync-scan-table code though.
With BAS_BULKWRITE e.g. CTAS/CREATE INDEX it makes lot of sense too.
prewarm could be hacked to use some new special BAS_DISTRIBUTE or something
for ideal distribution amongst all NUMA nodes.

II. what if we could track if the relation is just all-local-access?

Another idea is that if we would know that's it's just us working on some
relation (created by us; or it's not being touched remotley) then we could
ask for local-only memory affinity. So something like this:

a. in case of locally-only access rels =>
   ask for local memory first
   if that fails failback to weighted RR (so to to weights)
b. in case other rels => weighted RR (so to to weights) directly

The tracking of the fact that Buffer was accessed just locally or remotley
itself is not hard to imagine (by using some free "bits" in BufferDesc.
"state" where refcount/usagecount itself are stored, well at least 4 bits
for my 4 nodes, but there's plenty of left there), I have some PoC for
that but that's just per-Buffer tracking of "was this Buffer accessed by
remote nodes", but I'm completley lost how to make transition to the
is-the-relation-being-accessed-accross-NUMA-nodes info to drive such
optimization (we would need some shared infra just for tracking such info;
assuming up to 2^31 or 2^32 relations [OID?] and just using at least >=
2..4 bits, that's already huge number: we are talking GBs of shm mem).

BTW: I've been experimenting with this patchset and added couple of things
(see attached), and with I'm able to get optimal just by forcing affinity
too using that earlier bench:

  latency average = 2512.929 ms
  latency stddev = 97.525 ms

and that was with pure 100% affinity to my local node:

  select pg_buffercache_set_partition(0, '{100,0,0,0}');
  debug_clocksweep_balance_recalc=off
  debug_clocksweep_balance=on
  debug_clocksweep_scan_all_partitions=on

  (so it's another proof that code is fine, it's just algorithm that would
  have to adjusted)

For benchmarks with pgbench -S for 100% local affinity vs 100% remote
(I can do that with that pg_buffercache_set_partition() of mine), I'm
getting just +/- 1-2k TPS (42-43k TPS vs 41-42k TPS), so not much, __but__
I've spotted some another bug in from where we are fetching memory from
unoptimal places if we are not on node#0, I'll need to dig into that more
though. Another thing is that pgbench -S runs are much less demanding in
terms of memory bandwidth used (under <1GB/s here vs 6-8GB/s for
seqconcurrscans.sql using the same amout of cores).

> node and crashed with a segfault (or was it sigbus?).

It was SIGBUS and with this patchset I think we are fine: I have never
witnessed this one crashing with SIGBUS.

> usual duration), let me know.

See above, but I think we would have to fix at at least: mbind() failure,
and those VMAs disconnected regions.

-J.

---

From: Tomas Vondra <[email-masked]>
Date: Tue, 16 Jun 2026 14:39:45 +0200

> and that would cause accesing .balance[] without bounds.

Yes, this should be capped to the MAX_BUFFER_PARTITIONS.

> maybe with smaller MAX_BUFFER_PARTITIONS we could pack this into size=64 ?

Possibly. Im not entirely happy with making the ClockSweep struct so
much larger, but I haven't found a better way to store the counters
needed for balancing. The only thing I can think of is storing it
outside the struct, and maybe that's the right thing to do.

But that assumes the current balancing approach is the right one.

> pg_test_timing does for time), at least that could explain why somebody's

Yes, I think we may need some sort of caching for this / check only
sometimes. I haven't seen it to matter, but that may be luck and on
other systems / platforms it may be worse.

> Buffer Blocks were just put on 1 node (instead of many).

How come the pg_numa_bind_to_node() calls don't move the parts to the
correct node?

If something is already using huge pages on the other nodes, then sure,
it will fail. But I think that's OK - it's a best-effort thing. Maybe we
should exit instead in this case?

> + * - numRequestedAllocs - number of allocatios requested in a partition

I agree the explanation for this is not entirely clear.

> is very usefull).. or is there way to plug into main pg_reset functions?

Hmm, I was afraid it'd interfere with the balancing. I'm not sure it
makes sense to reset just some of the fields - it'd make it much harder
to interpret the counters. I'll think abou this.

> nodemask->size, MPOL_MF_MOVE);

I think this is a silly bug on my side, clearly the nodemask should have
size > 0 even for node 0.

> for sure it would affect for worse fork() rates by postmaster for new conns).

Isn't this 4 VMAs for buffers ad 4 VMAs for buffer descriptors? Or maybe
it's the parts that could not be mapped due to insufficient alignment?

> it's some kind luck it sharded the shm based on number of hugepages available)

I'm not sure I understand what you mean. What non-0?

> was 8 backends accessing 8x 1.6GB tables (lower than NBuffers / 4).

How does this compare to master, i.e. without these NUMA patches?

> for ideal distribution amongst all NUMA nodes.

Yes, I think it might make sense to disable balancing in these cases.

> seqconcurrscans.sql using the same amout of cores).

No opinion. I need to look at this closer.

> witnessed this one crashing with SIGBUS.

Good. But I wonder if allocating just the precise number of huge pages
(per shared_memory_size_in_huge_pages) can prevent moving the partitions
to the correct node.

> and those VMAs disconnected regions.

Yes, the mbind() failure is a bug.

regards

-- 
Tomas Vondra

---

From: Jakub Wartak <[email-masked]>
Date: Wed, 17 Jun 2026 14:13:03 +0200

[..]
[..]
> But that assumes the current balancing approach is the right one.

Yeah, I'm just not sure if there is some wasted performance due to
false-sharing in very heavy benchmark scenarios (in theory the
nextVictimBuffer could bounce rather heavily).

> other systems / platforms it may be worse.

Okay, so maybe an action point for us much later would be to try this on 2s+
ARM or some other much more rare setup just to see if we need to do it at all
(perhaps annotate it with TODO, I have short memory ;))

> How come the pg_numa_bind_to_node() calls don't move the parts to the

Well, if we were not checking error code mbind() it could behave in
non-deterministic way I think (you ask for MPOL_BIND, maybe it does something,
maybe it doesnt work but we continued anyway).

> should exit instead in this case?

Yes, I think we should exit with FATAL.

> to interpret the counters. I'll think abou this.

Well, small hint: I find the the view very, very usefull in explaining
where/why memory is being allocated from. Maybe if we want to include it
in final version, then it might be worth (later on) to do it, if that
won't be included I'm fine just doing \watch 1 in psql to see what's
happening. Another hint is that maybe it doesnt belong to pg_buffercache
and would make it easier to integrate into pg_stat_reset*() -- dunno, how/
if extension can plug into it.

> it's the parts that could not be mapped due to insufficient alignment?

Yes, like with patchset I'm getting WARNINGs:

WARNING:  buffers for node 0 not well aligned
[0x7efc100f5000,0x7efc900f5000] aligned
[0x7efc10200000,0x7efc90000000]
WARNING:  buffers descriptors for node 0 not well aligned
[0x7efc0c0f4f80,0x7efc0d0f4f41] aligned
[0x7efc0c200000,0x7efc0d000000]
WARNING:  buffers for node 1 not well aligned
[0x7efc900f5000,0x7efd100f5000] aligned
[0x7efc90200000,0x7efd10000000]
WARNING:  buffers descriptors for node 1 not well aligned
[0x7efc0d0f4f80,0x7efc0e0f4f41] aligned
[0x7efc0d200000,0x7efc0e000000]
WARNING:  buffers for node 2 not well aligned
[0x7efd100f5000,0x7efd900f5000] aligned
[0x7efd10200000,0x7efd90000000]
WARNING:  buffers descriptors for node 2 not well aligned
[0x7efc0e0f4f80,0x7efc0f0f4f41] aligned
[0x7efc0e200000,0x7efc0f000000]
WARNING:  buffers for node 3 not well aligned
[0x7efd900f5000,0x7efe100f5000] aligned
[0x7efd90200000,0x7efe10000000]
WARNING:  buffers descriptors for node 3 not well aligned
[0x7efc0f0f4f80,0x7efc100f4f41] aligned
[0x7efc0f200000,0x7efc10000000]
LOG:  starting PostgreSQL 19beta1 on x86_64-linux, compiled by
gcc-12.2.0, 64-bit

relevant numa_maps for postmaster:

7efc0c200000 bind:0 file=/anon_hugepage\040(deleted) huge dirty=7
mapmax=2 N0=7 kernelpagesize_kB=2048
7efc0d000000 default file=/anon_hugepage\040(deleted) huge dirty=1
mapmax=2 N3=1 kernelpagesize_kB=2048
7efc0d200000 bind:1 file=/anon_hugepage\040(deleted) huge dirty=7
mapmax=2 N1=7 kernelpagesize_kB=2048
7efc0e000000 default file=/anon_hugepage\040(deleted) huge dirty=1
mapmax=2 N3=1 kernelpagesize_kB=2048
7efc0e200000 bind:2 file=/anon_hugepage\040(deleted) huge dirty=7
mapmax=2 N2=7 kernelpagesize_kB=2048
7efc0f000000 default file=/anon_hugepage\040(deleted) huge dirty=1
mapmax=2 N3=1 kernelpagesize_kB=2048
7efc0f200000 bind:3 file=/anon_hugepage\040(deleted) huge dirty=7
mapmax=2 N3=7 kernelpagesize_kB=2048
7efc10000000 default file=/anon_hugepage\040(deleted) huge dirty=1
mapmax=3 N3=1 kernelpagesize_kB=2048
7efc10200000 bind:0 file=/anon_hugepage\040(deleted) huge dirty=1023
N0=1023 kernelpagesize_kB=2048
7efc90000000 default file=/anon_hugepage\040(deleted) huge dirty=1
N3=1 kernelpagesize_kB=2048
7efc90200000 bind:1 file=/anon_hugepage\040(deleted) huge dirty=1023
N1=1023 kernelpagesize_kB=2048
7efd10000000 default file=/anon_hugepage\040(deleted) huge dirty=1
N0=1 kernelpagesize_kB=2048
7efd10200000 bind:2 file=/anon_hugepage\040(deleted) huge dirty=1023
N2=1023 kernelpagesize_kB=2048
7efd90000000 default file=/anon_hugepage\040(deleted) huge dirty=1
N2=1 kernelpagesize_kB=2048
7efd90200000 bind:3 file=/anon_hugepage\040(deleted) huge dirty=1023
N3=1023 kernelpagesize_kB=2048
7efe10000000 default file=/anon_hugepage\040(deleted) huge dirty=116
mapmax=6 N1=116 kernelpagesize_kB=2048

so if You take just 7efc10200000..7efc90000000 (from buffers@node0, 1st line)
and zoom in/grep You'll get:

7efc10200000 bind:0 file=/anon_hugepage\040(deleted) huge dirty=1023
N0=1023 kernelpagesize_kB=2048
7efc90000000 default file=/anon_hugepage\040(deleted) huge dirty=1
N3=1 kernelpagesize_kB=2048

so @ 7efc90000000 it's Node3=1 hugepage (so it's wrong by 1x 2048kb offset)

so it contains the tail of node 0 buffers and the head of node - and is
excluded from both mbind() calls. At first I've spotted this in 003/
BufferPartitionsInit() with:
  cstartptr = (char *) &BufferDescriptors[buff_first];
  endptr   = (char *) &BufferDescriptors[buff_last] + 1; // <-- BUG?
with
  endptr   = (char *) &BufferDescriptors[buff_last + 1];

but got the same issue, so I've forced the mbind() to use 2x TYPEALIGN_DOWN
to cover with policy everything and got:

7fc134e00000 default file=/anon_hugepage\040(deleted) huge dirty=24
mapmax=6 N3=24 kernelpagesize_kB=2048
7fc137e00000 bind:0 file=/anon_hugepage\040(deleted) huge dirty=8
mapmax=4 N0=8 kernelpagesize_kB=2048
7fc138e00000 bind:1 file=/anon_hugepage\040(deleted) huge dirty=8
mapmax=2 N1=8 kernelpagesize_kB=2048
7fc139e00000 bind:2 file=/anon_hugepage\040(deleted) huge dirty=8
mapmax=3 N2=8 kernelpagesize_kB=2048
7fc13ae00000 bind:3 file=/anon_hugepage\040(deleted) huge dirty=8
mapmax=2 N3=8 kernelpagesize_kB=2048
7fc13be00000 bind:0 file=/anon_hugepage\040(deleted) huge dirty=1024
N0=1024 kernelpagesize_kB=2048
7fc1bbe00000 bind:1 file=/anon_hugepage\040(deleted) huge dirty=1024
N1=1024 kernelpagesize_kB=2048
7fc23be00000 bind:2 file=/anon_hugepage\040(deleted) huge dirty=1024
mapmax=2 N2=1024 kernelpagesize_kB=2048
7fc2bbe00000 bind:3 file=/anon_hugepage\040(deleted) huge dirty=1024
N3=1024 kernelpagesize_kB=2048
7fc33be00000 default file=/anon_hugepage\040(deleted) huge dirty=116
mapmax=6 N1=116 kernelpagesize_kB=2048

that was with:
@@ -351,7 +351,7 @@ BufferPartitionsInit(void)
                        }

                        /* best effort: align the pointers, so that
the mbind() works */
-                       startptr = (char *) TYPEALIGN(numa_page_size, startptr);
+                       startptr = (char *)
TYPEALIGN_DOWN(numa_page_size, startptr);
                        endptr = (char *)
TYPEALIGN_DOWN(numa_page_size, endptr);

@@ -374,7 +374,7 @@ BufferPartitionsInit(void)
                        }

                        /* best effort: align the pointers, so that
the mbind() works */
-                       startptr = (char *) TYPEALIGN(numa_page_size, startptr);
+                       startptr = (char *)
TYPEALIGN_DOWN(numa_page_size, startptr);
                        endptr = (char *)
TYPEALIGN_DOWN(numa_page_size, endptr);

Looks way better, but it was wild guess (TBH I think I've prefered the
previous patchset version where the input shm addresses were already aligned,
but if You and others say it's easier route then sure)

> I'm not sure I understand what you mean. What non-0?

Non-0 return code from mbind() as the mbind() failed with -1, but it did somehow
alter numa pages probably(?), but without creating specific "isolated" VMA
explicit "bind" policy, probably maybe it did not even work at all..

> How does this compare to master, i.e. without these NUMA patches?

I had some problems comparing, but with "perfect setup" that includes the
following:
- pgbench/clients on node#0
- backends running on node#0
- hugepages memory on node#0..3, but with with this patchset and those goodies:
  debug_clocksweep_balance=on
  debug_clocksweep_balance_recalc=off
  debug_clocksweep_scan_all_partitions=on
  (so technically node0 backends accessing just Buffers / Buffers Desc from
  node0, technically node0 weights: "{100,0,0,0}")
- with today's new discovery for me that Linux's kernel VFS cache is also
  having also first-touch (!) NUMA policy and really important here (so
  VFS cached data also alters results of testing wildly!), so I had to
  force unloading VFS cache and force-loading it into node#0, I've was
  getting for seqconcurrscans:
  latency average = 2701.705 ms
  latency stddev = 111.608 ms

vs master, huh, but which scenario? the default one without any affinity?
- assuming you get the split shm split like "N0=1059 N1=1299 N2=879 N3=1031"
  but that's appeneded-only (so not interleaved (?)) and you even risk
  having shm placed on just __one__ node (if it is big enough and free
  enough)
- if you go straight to benchmarking it (CPU hits random nodes)
  latency average = 3439.200 ms
  latency stddev = 580.501 ms
- with backends forked() to node#0 (numactl --cpunodebind=0 pg_ctl start)
  latency average = 4937.543 ms
  latency stddev = 573.841 ms
  because of random VFS cache placement (I imagine it as flow of on node0 CPUs
    a. nextVictimBuffer contention
    b. getBuffer() - fetch random shm memory from random NUMA node
    c. pread() - fetch from VFS cache but from *remote* NUMA node
  )
- with backends forked() to node#0 and pinned VFS cached fully on node#0 too
  latency average = 4518.651 ms
  latency stddev = 797.369 ms
  (but this is still Buffers from other nodes)
- same as above above and numactl interleaved shm:
  latency average = 3792.016 ms
  latency stddev = 825.186 ms
- same as above and interleaved shm, but without pining to CPUs on specifc
  node and ensure random VFS cache vs nodes:
  latency average = 2913.813 ms
  latency stddev = 352.552 ms
- but the moment you read anything reads base/ into VFS cache to particular
  node (imagine pg_prewarm or even just tar) assuming it was not there it
  also pins that to that node memory and you'll get:
  latency average = 3594.180 ms
  latency stddev = 851.949 ms

> Yes, I think it might make sense to disable balancing in these cases.

OK, I did not code anything of that as

> No opinion. I need to look at this closer.

Great !

> (per shared_memory_size_in_huge_pages) can prevent moving the partitions

Not sure I understand (?) how's that related to SIGBUS?

-J.

---

From: Tomas Vondra <[email-masked]>
Date: Wed, 24 Jun 2026 22:26:29 +0200

Hi,

Here's an updated patch series, with only minor changes to fix the mbind
issues:

1) It uses the correct nodemask size, so that the mbind actually binds
the partition to the right node.

2) It aligns the start/end pointers so that there no pages are left with
the default memory policy. So now there should be only the bind:N
entries, not the single-page "default" ones. This means the last page of
a partition can be mapped to a different node, but that seems fine (in
the end it could have happened with the old approach too).

I've also included Jakub's "goodies" patch with the additional GUCs.
Those seem potentially useful to development.

I have some results from a new round of benchmarks, and it's a bit
disappointing. Or rather, there seem to be some issues that I can't
figure out, causing regressions.

Consider a very simple test, doing a lot of sequential scans to put a
fair amount of pressure on the clocksweep / buffer replacement. There's
a .tgz with the benchmark script attached, but it does about this:

* Initialize a pgbench database with scale 2000 (so ~30GB, about twice
the shared buffers).

* Uses --partitions=100, so that the partitions are small enough not to
trigger the 1/4 threshold (i.e. not use circular buffers).

* Does runs with custom script, forcing sequential scans of the table,
with two queries:

select count(1) from pgbench_accounts;

select * from pgbench_accounts offset 1000000000;

Those are called "count" and "offset" in the results. The script forces
serial sequential scans (no index scans, no parallelism), and does runs
with 1, 8 and 32 clients (this is an old-ish xeon with 44 physical cores
on two sockets, 2 NUMA nodes).

I did runs with "master" and the all the 7 patches, with the NUMA stuff
enabled/disabled since 0003 (which adds it). See the two PDFs with more
complete results, but here's the "count" query for a subset of the
patches (the omitted ones behave similarly to what's shown here).

This chart is for median latency (in milliseconds):

  clients       master     0003      0004    0003/on    0004/on
  -------------------------------------------------------------
        1        12767    12582     14509      12807      15307
        8        14383    14355     14149      14069      16165
       32        14756    15198     14836      14984      17128
       --------------------------------------------------------
        1                  103%      114%       100%       120%
        8                  101%       98%        98%       112%
       32                  102%      101%       102%       116%

The percentages are compared to "master", the columns with "/on" are
with shared_buffers_numa=on.

Clearly, there's no chance with 0003 (which binds shared buffer
partitions to NUMA nodes, even if that's enabled). The differences are
within noise, pretty much, for all client counts.

Then 0004 gets applied, which partitions the clock sweep. And well, that
doesn't go particularly well. There is a bit of a regression even with
numa=off, but it kinda recovers with the following patches. But with
numa=on, there's a consistent ~10% regression (give or take).

I've spent a fair bit of time investigating what's causing this, but so
far I have nothing. I assume it's something silly in the patches
partitioning the clocksweep, or maybe the approach is flawed in some
way. Not sure :-(

regards

-- 
Tomas Vondra
From e6d192ae573d63b5cb5c71773047b97c97eb7961 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 3 Jun 2026 16:30:28 +0200
Subject: [PATCH v20260624 1/7] Add shmem_populate and shmem_interleave GUCs

 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#endif
+
 #include "miscadmin.h"
 
+/* GUCs */

+#ifdef USE_LIBNUMA
+	/*
 
index 7170a4bff98..bf38aa6baa2 100644

From 1f3fe6ec274f459ca4de440cf9da4eb43818e48c Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 2 Jun 2026 22:09:33 +0200
Subject: [PATCH v20260624 2/7] Infrastructure for partitioning of shared
 buffers

Notes:

 
index 510455998aa..6c9838b4efc 100644

 PG_MODULE_MAGIC_EXT(
@@ -77,6 +78,7 @@ PG_FUNCTION_INFO_V1(pg_buffercache_evict_all);

 /* Only need to touch memory once per backend process lifetime */
@@ -922,3 +924,87 @@ pg_buffercache_mark_dirty_all(PG_FUNCTION_ARGS)
 
 	PG_RETURN_DATUM(result);
 }
+
+/*

 
 
 /*

 /* in buf_init.c */

 
 
+/*
 
index c5db6ca6705..45fc91fcb97 100644

From 5a8f757fea9a2755d1c4fadad067352f4ab307be Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 2 Jun 2026 23:27:13 +0200
Subject: [PATCH v20260624 3/7] NUMA: shared buffers partitioning

The effect is similar to

     numactl --interleave=all

 src/backend/storage/buffer/buf_init.c         | 251 ++++++++++++++++--
 10 files changed, 399 insertions(+), 36 deletions(-)

index 6c9838b4efc..46b6b85a2e3 100644

 PG_MODULE_MAGIC_EXT(
@@ -955,11 +955,13 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 
 		funcctx->user_fctx = BlessTupleDesc(tupledesc);
@@ -977,28 +979,32 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 	{
 		uint32		i = funcctx->call_cntr;
 
-		int			num_buffers,
+		int			numa_node,
+					num_buffers,
 					first_buffer,
 					last_buffer;

 

 
 
index e593b02e0ca..4ed79db7b49 100644
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+

 
 
 /*

@@ -231,35 +250,210 @@ BufferPartitionsInit(void)
 {
 	int			buffer = 0;

 

 

-		buffer += num_buffers;
+#ifdef USE_LIBNUMA
+		/*
+		 * Now try to locate buffers and buffer descriptors to the node (all
+		 * partitions for the node at once).
+		 */
+		if (shared_buffers_numa)
+		{
+			Size	numa_page_size = pg_numa_page_size();
+
+			int		part_first,
+					part_last,
+					buff_first,
+					buff_last;
+
+			char   *startptr,
+				   *endptr;
+
+			/* first/last partition for this node */
+			part_first = (n * npartitions_per_node);
+			part_last = part_first + (npartitions_per_node - 1);
+
+			/* buffers (blocks) */
+
+			/* first/last buffer */
+			buff_first = BufferPartitionsRegistry->partitions[part_first].first_buffer;
+			buff_last = BufferPartitionsRegistry->partitions[part_last].last_buffer;
+
+			/* beginning of the first block, end of last block */
+			startptr = BufferBlocks + ((Size) buff_first * BLCKSZ);
+			endptr = BufferBlocks + ((Size) (buff_last + 1) * BLCKSZ);
+
+			/* print some warnings when the partitions are not aligned */
+			if ((startptr != (char *) TYPEALIGN(numa_page_size, startptr)) ||
+				(endptr != (char *) TYPEALIGN(numa_page_size, endptr)))
+			{
+				elog(WARNING, "buffers for node %d not well aligned [%p,%p]",
+					 n, startptr, endptr);
+			}
+
+			/* best effort: align the pointers, so that the mbind() works */
+			startptr = (char *) TYPEALIGN_DOWN(numa_page_size, startptr);
+
+			/* the last partition aligns to the end of the buffer */
+			if (n == (nnodes - 1))
+				endptr = (char *) TYPEALIGN(numa_page_size, endptr);
+			else
+				endptr = (char *) TYPEALIGN_DOWN(numa_page_size, endptr);
+
+			/* XXX or should we use pg_numa_move_to_node? */
+			if (pg_numa_bind_to_node(startptr, endptr, n) != 0)
+				elog(WARNING, "failed to bind shared buffers partition to node %d", n);
+
+			/* buffer descriptors */
+
+			/* beginning of the first descriptor, end of last descriptor */
+			startptr = (char *) &BufferDescriptors[buff_first];
+			endptr = (char *) &BufferDescriptors[buff_last] + 1;
+
+			/* print some warnings when the partitions are not aligned */
+			if ((startptr != (char *) TYPEALIGN(numa_page_size, startptr)) ||
+				(endptr != (char *) TYPEALIGN(numa_page_size, endptr)))
+			{
+				elog(WARNING, "buffers descriptors for node %d not well aligned [%p,%p]",
+					 n, startptr, endptr);
+			}
+
+			/* best effort: align the pointers, so that the mbind() works */
+			startptr = (char *) TYPEALIGN_DOWN(numa_page_size, startptr);
+
+			if (n == (nnodes - 1))
+				endptr = (char *) TYPEALIGN(numa_page_size, endptr);
+			else
+				endptr = (char *) TYPEALIGN_DOWN(numa_page_size, endptr);
+
+			/* XXX or should we use pg_numa_move_to_node? */
+			if (pg_numa_bind_to_node(startptr, endptr, n) != 0)
+				elog(WARNING, "failed to bind shared buffer descriptors partition to node %d", n);
+		}
+#endif
 	}

+/*
@@ -277,13 +471,14 @@ BufferPartitionCount(void)
 
@@ -293,3 +488,17 @@ BufferPartitionGet(int idx, int *num_buffers,

+#ifdef USE_LIBNUMA
+#include <sched.h>
+#endif
+
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
 #include "pgstat.h"

 #ifdef USE_LIBNUMA

 
 
 /* in localbuf.c */

 /* in buf_init.c */
 
 /* in localbuf.c */
 
+/* in buf_init.c */
 
diff --git a/src/port/pg_numa.c b/src/port/pg_numa.c
index 8954669273a..66985f32db3 100644
--- a/src/port/pg_numa.c
+++ b/src/port/pg_numa.c
@@ -18,6 +18,9 @@
 
 #include "miscadmin.h"
 
 /*
 
+/*
+ * pg_numa_move_to_node
+				MPOL_BIND, nodemask->maskp, nodemask->size, MPOL_MF_MOVE);
 
 /* Empty wrappers */
 
+void

From 8265b556de61802983a3fefc31e0e2e071b456ed Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 2 Jun 2026 22:16:55 +0200
Subject: [PATCH v20260624 4/7] clock-sweep: basic partitioning

 
index 46b6b85a2e3..b07fafda0d9 100644

 PG_MODULE_MAGIC_EXT(
@@ -963,6 +963,14 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)

@@ -984,12 +992,22 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 					first_buffer,
 					last_buffer;

 
 
@@ -1005,6 +1023,18 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		values[4] = Int32GetDatum(last_buffer);
 		nulls[4] = false;
 
index d6c0cc1f6d4..7bedaf8987a 100644
 
 /*

 
 	/*

 

 	/*
 
 	/*

 

 #ifdef BGW_DEBUG

 /*

 /*
 
 	/*

 
 
 /*
 
 	/*

 
 		/*

 
 /*

+/*

 	if (complete_passes)
 	{
 
 		/*
 
-	if (num_buf_alloc)
-	{
 
 /*

-	/* Clear statistics */

 	return true;
 }
+
+void

 /* buf_table.c */

 /* inline functions */
 
index 45fc91fcb97..a4617481f7c 100644

From 4ed4cf8669a6f379ebbf39a2283704931d4b3279 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 2 Jun 2026 22:28:10 +0200
Subject: [PATCH v20260624 5/7] clock-sweep: balancing of allocations

 
index b07fafda0d9..f26b2332c1d 100644
 
@@ -31,7 +33,7 @@

 PG_MODULE_MAGIC_EXT(
 					.name = "pg_buffercache",
@@ -940,6 +942,8 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 
 	if (SRF_IS_FIRSTCALL())
 	{
 
@@ -971,6 +975,12 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)

@@ -992,11 +1002,17 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 					first_buffer,
 					last_buffer;

 		uint32		complete_passes,
 					next_victim_buffer,
-					buffer_allocs;
+					buffer_allocs,
+					buffer_req_allocs;
+
+		int		   *weights;
+		Datum	   *dweights;
+		ArrayType  *array;
 
 		Datum		values[NUM_BUFFERCACHE_PARTITIONS_ELEM];
 		bool		nulls[NUM_BUFFERCACHE_PARTITIONS_ELEM];
@@ -1005,8 +1021,16 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 						   &first_buffer, &last_buffer);

 		values[0] = Int32GetDatum(i);
 		nulls[0] = false;
@@ -1035,6 +1059,15 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		values[8] = Int64GetDatum(buffer_allocs);
 		nulls[8] = false;
 
index 7bedaf8987a..61be05a68d4 100644

+/*
 
+	/*
 
+	/*
 
 /*
 
 /*
 
 	/*

 /*
 
 	/*
 
+/*
+ * StrategySyncBalance

 void

 

 
 
 /* inline functions */
 
-- 
2.54.0

From fafa9ba54c0201cec1084f558cdad285b5fc903b Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 2 Jun 2026 22:39:38 +0200
Subject: [PATCH v20260624 6/7] clock-sweep: scan all partitions

 
 /*
 
 	/*
 
 	*from_ring = false;

 
 		/*

From f5b7b6733419186e56b06ed1d8b6da7e34984c2f Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email-masked]>
Date: Thu, 11 Jun 2026 12:38:43 +0200
Subject: [PATCH v20260624 7/7] Add parttioned clocksweep and NUMA goodies.

1. Add three clocksweep GUCs to allow manipulation of partitoned clocksweep
   in runtime.
2. Add pg_buffercache_set_weights(int, int[]) to alter partition allocs (with
   clocksweep_balance_recalc=off).
3. Add debug_numa_node GUC to pin to NUMA node.
---
 .../pg_buffercache--1.7--1.8.sql              |  7 ++
 contrib/pg_buffercache/pg_buffercache_pages.c | 37 ++++++++
 src/backend/storage/buffer/freelist.c         | 86 ++++++++++++++++++-
 src/backend/tcop/postgres.c                   | 57 ++++++++++++
 src/backend/utils/misc/guc_parameters.dat     | 31 +++++++
 src/include/miscadmin.h                       |  1 +
 src/include/storage/bufmgr.h                  |  6 ++
 7 files changed, 224 insertions(+), 1 deletion(-)

diff --git a/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql b/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
index 43d2e84f9d2..9d8f4969555 100644
--- a/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
+++ b/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
@@ -25,9 +25,16 @@ CREATE VIEW pg_buffercache_partitions AS
 	 num_req_allocs bigint,		-- handled allocs (current cycle)
 	 weights int[]);			-- balancing weights
 
+-- Register the function to set clock-sweep balance weights.
+CREATE FUNCTION pg_buffercache_set_partition(IN partition int, IN weights int[])
+RETURNS void
+AS 'MODULE_PATHNAME', 'pg_buffercache_set_partition'
+LANGUAGE C VOLATILE PARALLEL UNSAFE;
+
 -- Don't want these to be available to public.
 REVOKE ALL ON FUNCTION pg_buffercache_partitions() FROM PUBLIC;
 REVOKE ALL ON pg_buffercache_partitions FROM PUBLIC;
+REVOKE ALL ON FUNCTION pg_buffercache_set_partition(int, int[]) FROM PUBLIC;
 
 GRANT EXECUTE ON FUNCTION pg_buffercache_partitions() TO pg_monitor;
 GRANT SELECT ON pg_buffercache_partitions TO pg_monitor;
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index f26b2332c1d..2a1b7cb7549 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -81,6 +81,7 @@ PG_FUNCTION_INFO_V1(pg_buffercache_mark_dirty);
 PG_FUNCTION_INFO_V1(pg_buffercache_mark_dirty_relation);
 PG_FUNCTION_INFO_V1(pg_buffercache_mark_dirty_all);
 PG_FUNCTION_INFO_V1(pg_buffercache_partitions);
+PG_FUNCTION_INFO_V1(pg_buffercache_set_partition);

 /* Only need to touch memory once per backend process lifetime */
@@ -1077,3 +1078,39 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 	else
 		SRF_RETURN_DONE(funcctx);
 }
+
+/*
+ * Set the clock-sweep balance weights for a single partition.
+ */
+Datum
+pg_buffercache_set_partition(PG_FUNCTION_ARGS)
+{
+	int			partition = PG_GETARG_INT32(0);
+	ArrayType  *array = PG_GETARG_ARRAYTYPE_P(1);
+	Datum	   *elems;
+	bool	   *nulls;
+	int			nelems;
+	int		   *weights;
+
+	if (ARR_NDIM(array) > 1)
+		ereport(ERROR,
+				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+				 errmsg("weights must be a one-dimensional array")));
+
+	deconstruct_array_builtin(array, INT4OID, &elems, &nulls, &nelems);
+
+	weights = palloc_array(int, nelems);
+	for (int i = 0; i < nelems; i++)
+	{
+		if (nulls[i])
+			ereport(ERROR,
+					(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+					 errmsg("weights must not contain NULL values")));
+
+		weights[i] = DatumGetInt32(elems[i]);
+	}
+
+	ClockSweepSetWeights(partition, weights, nelems);
+
+	PG_RETURN_VOID();
+}
diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index 1ac1e3e3490..e677c71e0b3 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -55,6 +55,26 @@
  */
 #define CLOCKSWEEP_HISTORY_COEFF	0.5
 
+/*
+ * GUCs controlling the NUMA-aware clock-sweep behavior.
+ *
+ * clocksweep_balance - when enabled, allocations may get redirected between
+ * clock-sweep partitions to keep them balanced (see StrategySyncBalance).
+ *
+ * clocksweep_balance_recalc - when enabled, the balance weights are
+ * periodically recalculated (see StrategySyncBalance). Disabling this keeps
+ * the current weights, e.g. ones configured manually using
+ * pg_buffercache_set_partition(), while still using them to balance
+ * allocations (if clocksweep_balance is enabled).
+ *
+ * clocksweep_scan_all_partitions - when enabled, looking for a free buffer
+ * scans all clock-sweep partitions (in a round-robin way), not just the
+ * backend's "home" partition.
+ */
+bool		clocksweep_balance = true;
+bool		clocksweep_balance_recalc = true;
+bool		clocksweep_scan_all_partitions = true;
+
 /*
  * Information about one partition of the ClockSweep (on a subset of buffers).
  *
@@ -436,8 +456,11 @@ ChooseClockSweep(bool balance)
 	 * When balancing allocations, redirect the allocations to other partitions
 	 * according to the budgets. We move through partitions in a round-robin way,
 	 * after allocating the "budget" of allocations from the current one.
+	 *
+	 * Balancing can be disabled at runtime using the clocksweep_balance GUC, in
+	 * which case allocations always stay in the backend's "home" partition.
 	 */
-	if (balance)
+	if (balance && clocksweep_balance)
 	{
 		/*
 		 * Ran out of budget from the current partition? Move to the next one
@@ -551,6 +574,10 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint64 *buf_state, bool *from_r
 	 * Start with the "preferred" partition, and then proceed in a round-robin
 	 * manner. If we cycle back to the starting partition, it means none of the
 	 * partitions has unpinned buffers.
+	 *
+	 * Scanning of the other partitions can be disabled at runtime using the
+	 * clocksweep_scan_all_partitions GUC. In that case we only scan the
+	 * backend's "home" partition, and fail if it has no unpinned buffers.
 	 */
 	sweep = ChooseClockSweep(true);
 	sweep_start = sweep;
@@ -562,6 +589,10 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint64 *buf_state, bool *from_r
 		if (buf != NULL)
 			return buf;
 
+		/* don't look at other partitions unless allowed to */
+		if (!clocksweep_scan_all_partitions)
+			break;
+
 		/*
 		 * Try advancing to the next partition, round-robin (if last partition,
 		 * wrap around to the beginning).
@@ -739,6 +770,9 @@ StrategySyncBalance(void)
 			avg_allocs,			/* average allocations (per partition) */
 			delta_allocs = 0;	/* sum of allocs above average */
 
+	if (!clocksweep_balance || !clocksweep_balance_recalc)
+		return;
+
 	/*
 	 * Collect the number of allocations requested in the past interval.
 	 * While at it, reset the counter to start the new interval.
@@ -1481,3 +1515,53 @@ ClockSweepPartitionGetInfo(int idx,
 		(*weights)[i] = (int) sweep->balance[i];
 	}
 }
+
+/*
+ * ClockSweepSetWeights override the clock-sweep balance weights of a single
+ * partition.
+ */
+void
+ClockSweepSetWeights(int partition, int *weights, int nweights)
+{
+	ClockSweep *sweep;
+
+	/*
+	 * Disallow manual weights while the automatic recalculation is enabled, as
+	 * StrategySyncBalance would just recompute (and overwrite) them.
+	 */
+	if (clocksweep_balance_recalc)
+		ereport(ERROR,
+				(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+				 errmsg("cannot set clock-sweep weights while debug_clocksweep_balance_recalc is enabled"),
+				 errhint("Set debug_clocksweep_balance_recalc to off before setting the weights manually.")));
+
+	if ((partition < 0) || (partition >= StrategyControl->num_partitions))
+		ereport(ERROR,
+				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+				 errmsg("clock-sweep partition %d out of range", partition),
+				 errhint("There are %d clock-sweep partitions, numbered from 0.",
+						 StrategyControl->num_partitions)));
+
+	if (nweights != StrategyControl->num_partitions)
+		ereport(ERROR,
+				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+				 errmsg("number of weights (%d) does not match number of clock-sweep partitions (%d)",
+						nweights, StrategyControl->num_partitions)));
+
+	for (int i = 0; i < nweights; i++)
+	{
+		if ((weights[i] < 0) || (weights[i] > 100))
+			ereport(ERROR,
+					(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+					 errmsg("clock-sweep weight %d out of range",
+							weights[i]),
+					 errhint("Each weight must be between 0 and 100.")));
+	}
+
+	sweep = &StrategyControl->sweeps[partition];
+
+	SpinLockAcquire(&sweep->clock_sweep_lock);
+	for (int j = 0; j < nweights; j++)
+		sweep->balance[j] = (uint8) weights[j];
+	SpinLockRelease(&sweep->clock_sweep_lock);
+}
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index dbef734a93f..02787062c83 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -86,6 +86,7 @@
 #include "utils/timeout.h"
 #include "utils/timestamp.h"
 #include "utils/varlena.h"
+#include <numa.h>
 
 /* ----------------
  *		global variables
@@ -110,6 +111,9 @@ int			client_connection_check_interval = 0;
 /* flags for non-system relation kinds to restrict use */
 int			restrict_nonsystem_relation_kind;
 
+/* NUMA node to pin the backend to at query start; -1 disables pinning */
+int			debug_numa_node = -1;
+
 /*
  * Include signal sender PID/UID in the server log when available
  * (SA_SIGINFO). The caller must supply the already-captured pid and uid
@@ -1020,6 +1024,53 @@ pg_plan_queries(List *querytrees, const char *query_string, int cursorOptions,
 }

+/*
+ * process_debug_numa_node
+ *
+ * If the debug_numa_node GUC is set (>= 0), pin this backend to run on the
+ * CPUs of the requested NUMA node.  -1 disables it (default)
+ */
+static void
+process_debug_numa_node(void)
+{
+#ifdef USE_LIBNUMA
+	static int	applied_numa_node = -1;
+
+	if (debug_numa_node == applied_numa_node)
+		return;
+
+	/* Nothing we can do if the kernel/library has no NUMA support. */
+	if (numa_available() < 0)
+	{
+		applied_numa_node = debug_numa_node;
+		return;
+	}
+
+	if (debug_numa_node < 0)
+	{
+		/* Pinning disabled: allow running on all nodes again. */
+		numa_run_on_node_mask(numa_all_nodes_ptr);
+	}
+	else if (debug_numa_node > numa_max_node())
+	{
+		ereport(WARNING,
+				(errmsg("debug_numa_node %d exceeds the highest available NUMA node %d, ignoring",
+						debug_numa_node, numa_max_node())));
+	}
+	else if (numa_run_on_node(debug_numa_node) != 0)
+	{
+		ereport(WARNING,
+				(errmsg("could not pin backend to NUMA node %d: %m",
+						debug_numa_node)));
+	}
+	else
+		elog(DEBUG1, "pinned backend to NUMA node %d", debug_numa_node);
+
+	applied_numa_node = debug_numa_node;
+#endif
+}
+
+
 /*
  * exec_simple_query
  *
@@ -1044,6 +1095,9 @@ exec_simple_query(const char *query_string)
 
 	pgstat_report_activity(STATE_RUNNING, query_string);
 
+	/* Pin the backend to the configured NUMA node, if requested. */
+	process_debug_numa_node();
+
 	TRACE_POSTGRESQL_QUERY_START(query_string);
 
 	/*
@@ -2185,6 +2239,9 @@ exec_execute_message(const char *portal_name, long max_rows)
 
 	pgstat_report_activity(STATE_RUNNING, sourceText);
 
+	/* Pin the backend to the configured NUMA node, if requested. */
+	process_debug_numa_node();
+
 	foreach(lc, portal->stmts)
 	{
 		PlannedStmt *stmt = lfirst_node(PlannedStmt, lc);
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 2e71c04282c..b72fce4b92b 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -641,6 +641,27 @@
   boot_val => 'DEFAULT_ASSERT_ENABLED',
 },
 
+{ name => 'debug_clocksweep_balance', type => 'bool', context => 'PGC_USERSET', group => 'DEVELOPER_OPTIONS',
+  short_desc => 'Enables balancing of buffer allocations between clock-sweep partitions.',
+  flags => 'GUC_NOT_IN_SAMPLE',
+  variable => 'clocksweep_balance',
+  boot_val => 'true'
+},
+
+{ name => 'debug_clocksweep_balance_recalc', type => 'bool', context => 'PGC_USERSET', group => 'DEVELOPER_OPTIONS',
+  short_desc => 'Enables periodic recalculation of clock-sweep partition balance weights.',
+  flags => 'GUC_NOT_IN_SAMPLE',
+  variable => 'clocksweep_balance_recalc',
+  boot_val => 'true'
+},
+
+{ name => 'debug_clocksweep_scan_all_partitions', type => 'bool', context => 'PGC_USERSET', group => 'DEVELOPER_OPTIONS',
+  short_desc => 'Enables scanning all clock-sweep partitions when looking for a free buffer.',
+  flags => 'GUC_NOT_IN_SAMPLE',
+  variable => 'clocksweep_scan_all_partitions',
+  boot_val => 'true'
+},
+
 { name => 'debug_copy_parse_plan_trees', type => 'bool', context => 'PGC_SUSET', group => 'DEVELOPER_OPTIONS',
   short_desc => 'Set this to force all parse and plan trees to be passed through copyObject(), to facilitate catching errors and omissions in copyObject().',
   flags => 'GUC_NOT_IN_SAMPLE',
@@ -693,6 +714,16 @@
   options => 'debug_logical_replication_streaming_options',
 },
 
+{ name => 'debug_numa_node', type => 'int', context => 'PGC_USERSET', group => 'DEVELOPER_OPTIONS',
+  short_desc => 'Pins the backend to the given NUMA node at query start.',
+  long_desc => '-1 (the default) disables pinning.',
+  flags => 'GUC_NOT_IN_SAMPLE',
+  variable => 'debug_numa_node',
+  boot_val => '-1',
+  min => '-1',
+  max => 'INT_MAX',
+},
+
 { name => 'debug_parallel_query', type => 'enum', context => 'PGC_USERSET', group => 'DEVELOPER_OPTIONS',
   short_desc => 'Forces the planner\'s use parallel query nodes.',
   long_desc => 'This can be useful for testing the parallel query infrastructure by forcing the planner to generate plans that contain nodes that perform tuple communication between workers and the main process.',
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index bf38aa6baa2..9590f7ac467 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -215,6 +215,7 @@ extern PGDLLIMPORT bool MyDatabaseHasLoginEventTriggers;
 
 extern PGDLLIMPORT bool shmem_populate;
 extern PGDLLIMPORT bool shmem_interleave;
+extern PGDLLIMPORT int debug_numa_node;

 /*
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index 02833b19b0c..b5c3873acae 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -190,6 +190,11 @@ extern PGDLLIMPORT int bgwriter_lru_maxpages;
 extern PGDLLIMPORT double bgwriter_lru_multiplier;
 extern PGDLLIMPORT bool track_io_timing;
 
+/* in freelist.c */
+extern PGDLLIMPORT bool clocksweep_balance;
+extern PGDLLIMPORT bool clocksweep_balance_recalc;
+extern PGDLLIMPORT bool clocksweep_scan_all_partitions;
+
 #define DEFAULT_EFFECTIVE_IO_CONCURRENCY 16
 #define DEFAULT_MAINTENANCE_IO_CONCURRENCY 16
 extern PGDLLIMPORT int effective_io_concurrency;
@@ -418,6 +423,7 @@ extern void ClockSweepPartitionGetInfo(int idx,
 									 uint64 *buffer_total_req_allocs,
 									 uint32 *buffer_req_allocs,
 									 int **weights);
+extern void ClockSweepSetWeights(int partition, int *weights, int nweights);
 
 /* inline functions */
 
-- 
2.54.0

---

From: Jakub Wartak <[email-masked]>
Date: Thu, 25 Jun 2026 14:19:36 +0200

On Wed, Jun 24, 2026 at 10:29 PM Tomas Vondra <[email-masked]> wrote:

Hi,

[..]
> Those seem potentially useful to development.

Cool!

[..]
>   clients       master     0003      0004    0003/on    0004/on

I haven't tried it yet, however I can spot some things:

No crystal clear idea why, but in the script I can see that you have
io_method = io_uring and are not dropping_caches, so IMHO it is too complex
interaction at this stage.

One hint: such setup is going to be problematic for proving numbers.
On the meeting I've tried to describe that I've been using io_method = sync
instead of 'worker' to get more predicitable results (together with echo 3
> drop_caches), because then it is that backend's CPU/$NODE doing that
pread()/pwrite() -- or any other operating performing the load --
it is going to put that file onto that_specific_$NODE --
so even if you have sequence like:
    pgbench -i
    pg_ctl restart
    pgbench -c XX

then pgbench -i even with shared_buffers_numa=on will spread into many
nodes the Buffers, yet after the restart the VFS cache portion of the data
will still reside on single specific $NODE that wrote it to the filesystem
(due to local-first-tocuh-affinity even for VFS cache), so any further reading:
VFS cache --pread()--> s_b will take the hit of remote interconnect with
some probablity depending on where the new backends are running. Also
with worker it is even worse as we have those memory queue in between. I
think we even can have this:

file in VFS cache @ node0   --because of first touch policy (pgbench -i/prewarm)
io worker @ node1           --hits latency from node0 and node2
shm io worker queue @ node2 --well
client backend @ node 3     --puts into shm io worker on node2

Therefore I'm sticking to 'sync' to ease the pain... but with uring, I suspect
the situation is kind of similiar as we call io_uring_submit(), and we
may endup using io-wq kernel threads, and we have those submission/receive
(memory) queues that are located somewhere (that is on some node) too.

I think, we simply lack affinity for IO/NUMA for all io modes except sync, but
it's too early I suspect and way outside of scope for this $thread. I've
started thinking about it just last week, so... (but hopefully I'll be able
to ship helper fscachenuma.c to show layout of file across VFS caches on nodes
next week I hope)

Maybe some other suggestions:

Q1) Maybe some crosschecks first?
       # balance should be equal between nodes even for baseline
       # linux kernel has tendency to fit shm into one if it fits
       find /sys/devices/system/node*/ -name 'free_hugepages' -exec
grep -H . {} \;

       # check N0 and N1 even for default policy, might also reveal imbalance
       # lots of RAM and too big huge_pages allows fitting whole shm
into just N0
       # see point 4 from [1]
       grep /anon_h /proc/$SOMEREALBACKENDPID/numa_maps

       # then during pgbench -c run maybe those:
       mpstat -N ALL 1
       perf stat -a -e uncore_imc/cas_count_read/,uncore_imc/cas_count_write/ \
          --per-socket -I 1000  # or -M
memory_bandwidth_read,memory_bandwidth_write

    (it might reveal that problem I've described above about io_method:
    even with pgbench -c 1 you might be reading from all sockets/wrong sockets
    instead of the correct one with affinity)

    I like to pin CPUs to just one node for pgbench -c
<NUMBER_OF_CPUS/NUMBER_OF_NODES>
    [to saturate one node only] and start server also with CPU pining
    [or use this debug_numa_node to force] to that one node and cross-check
    what's being read (using perf) and usually I have to disarm clock balancing
    and override weights using pg_buffercache_set_partition() to also force
    weight to stay local only - only then I'm able to outrun master. That's
    how this idea was born that if we are only working on node $N with
some relations
    then let's use only node $N's Buffers. But I have 90us:~280us
local vs remote
    latency, so it's probably way easier for me to see results even without
    disabling CPU-idle-states/turboboost/etc.

Q2) Dunno, but 0007 is not changing anything in runtime and you get huge
    discrepeancy results when going 0006 -> 0007 for clients=1 (see
128% -> 112%).
    Literally, as the same code but different rebuild (ELF image)
would be having
    vastly different layout enough to cause perf issues?

Hopefully next week I'll try to repro those numbers to see if I can
help more.

-J.

[1] - https://www.postgresql.org/message-id/CAKZiRmzo9xnJSgO4b26DTZqPuObcQ-6ncay%2BmOEKs9rzCkegUA%40mail.gmail.com

---

From: Tomas Vondra <[email-masked]>
Date: Thu, 25 Jun 2026 15:49:00 +0200

> interaction at this stage.

By caches I assume you mean page cache? The test is meant so simulate a
cached system, copying data between shared buffers and page cache. My
expectation is that once we start hitting I/O, it'll completely hide
most differences due to NUMA.

> so even if you have sequence like:

Hmm, I missed that point during the meeting. I wonder if "worker" is
interacting with the NUMA somehow (I mean, does it load it into the
right node?). But I'm using io_uring, and it's not clear to me why sync
would be better for benchmarking?

Ultimately, we need to make sure it works well with io_uring anyway,
right? Even if "sync" happens to be better for benchmarking (or even for
NUMA stuff), we have to make it work with worker/io_uring. Because
that's what practical systems use.

> to ship helper fscachenuma.c to show layout of file across VFS caches on nodes

Ah, you're suggesting the page cache stuff will be placed on a single
NUMA node? That may be true, it's a good point. And maybe it could skew
the results in a bad way. Still, that would be the case even without the
NUMA partitioning, no?

>     instead of the correct one with affinity)

I'll try, but if you could try running some experiments on your own,
that might be helpful.

>     discrepeancy results when going 0006 -> 0007 for clients=1 (see

Thank you! That'd be great.

regards

-- 
Tomas Vondra

---

From: Jakub Wartak <[email-masked]>
Date: Mon, 29 Jun 2026 09:42:49 +0200

> most differences due to NUMA.

No, it wont completley hide it, those differences at least here still matter
(AFAIR right now like +/- 10% here)

> that's what practical systems use.

Yes, we need to make work with more advanced, but I don't think we are there
yet (we'll need some more patches in orde rto demonstrate it reliably).

> > (due to local-first-tocuh-affinity even for VFS cache),
> > [.. blabla , use io_method=sync ]
> >
>
> the results in a bad way.

I've just published [0], see for yourself:

This happens especiall after pgbench -i, so:
     pgbench -i # pagecache placement on one NUMA node
     pg_ctl restart
     pgbench -c XX

is day and night different than let's say:
     pgbench -i
     echo 3 > drop_caches
     pg_ctl restart
     pgbench -c XX # pagecache placement happens by many backends
                   # potentially many NUMA nodes

> Still, that would be the case even without the NUMA partitioning, no?

Right, in my experience we should not benchmark against master started
with the default pg_ctl (that's is without numactl --interleave=all) because
it is confusing to reason about it due how the s_b could laid out without
that interleaving. I mean later we can switch to that default, but IMHO not
yet.

[..]
> Thank you! That'd be great.

Yeah, I'll try my best, we'll see how it goes. Right now I've just dropped
that fscachenuma proggie to aid us in troubleshooting.

-J.

[0] - https://github.com/jakubwartakEDB/fscachenuma

---

From: Jakub Wartak <[email-masked]>
Date: Tue, 30 Jun 2026 14:51:18 +0200

> > >>   clients       master     0003      0004    0003/on    0004/on

[..lots of variables..]

Hi Tomas,

OK, so I've run couple of tests and modified run.sh and also tried to fix
some inefficiencies spotted while testing this. Note the attached
performance matrix is in TPS (so more is better). Raw results/CSV and
scripts are attached too.

* run2 = 2 workloads, partitioned pgbench_accounts
* run3 = just pgbenchS w/o partitioning + warmup
* run4 = semi-like pgbenchS w/o partitioning but 100k rows + warmup

One important modification in those run shell scripts is that they
clean page-cache (drop_cached) as mentioned earlier to avoid false results
where everything would on node#N after pgbench -i ran. Probably I did
not get any regressions you've got, because of this. Or better diff -u
run*.sh scripts.

The "inst-optimized" is just the same patchset (so "inst-patchset") + crude
attempt in 0008 to make further smooth out things and avoid regressions while
I've been working on this. 0008 does couple of things:

a. implements CPU/node caching instead quering it every single buffer. Even
   if on x86_64 that is optimized by vdso/kernel to avoid the real syscall,
   the semi-syscall tax seems to be visible when fetching lots of buffers.
   128 is arbitrary and still kind of low (128*8kB=1MB, and we are doing
   hundreths of MB/s; while rescheduling happened only every couple of
   seconds).

b1. minimize the attempt to use other partittions till some threshold (
   and then it relies on the scan-all-partitions)

b2. avoids selecting idle partitions (defined as avg_allocs/2) - if there
   are low allocations there it is debatable if cache utilization is better
   or sticking to lower latency is better (e.g. in some workloads buffer
   reuse is close to 0, so lower latency is clearly better)

Results are attached, some observations:

0.There were vast differences in how pg_ctl is started (interleaved or not),
  so I've decided in the end to show relative to both situations.

1.In run2/seqconcurrscans I've saturated my interconnect and that's why
  it's giving 129-155% there. I don't have access physiscal hw, but I suspect
  that modern 2socket EPYC5 has like ~614GB/s per socket RAM bandwidth,
  but the max oneway bandwith of the interconnect is around ~220GB/s (
  no way to provie it), so *IF* with hundreths of cores we would be able
  fetch at this rate we could saturte modern hardware too that way (and
  we birefly touched related topic: batched executor, accelerating it
  so fast those effects could be more easily achieveable)

2.run3 has no partiitioning because according to perf and my eyes, it
  spent time not on the buffers itself (thus it was way heavier on CPU
  [partitioning] than on memory...), so that's how run3 was born without
  partitions :D

3.The warmup is critical for run3/pgbenchS, as I've noticed that depending
  on ${luck} if you start the "master" (baseline w/o interleaving) and pgbench
  it right away everything might land on node0 (s_b, pagecache), so "master"
  was basically cheating in benchmarks vs especially Your's patchset where
  it was spreading way too soon. Having drop_caches, additional warump and
  only then proper pgbench kind of reduces that luck-factor. In general I
  think all runs with c=1 seem to have kind of low singal-to-noise ratio. I
  was thinking about pinning to always stick to the same NUMA node from start
  to win against master just for this c=1 scenarios, but "meh".

3b. in short for pgbench -S we can gain like 2-5%

4.run4 was made just to prove that workload fetching more buffers, than
  the standard pgbench -S (1 row?), seems to be the key to prove
  optimizations in 0008 (other than showing good benefits for seqconcurrscans
  of course). So run4 just shows benefit compared to 0001-0007 alone.

Stil on the table:

1. maybe even better balancing is possible (?), but this one is seems enough?
   I'm out of other ideas, well other than the
   "shared-relation-use-by-foreign-node" idea described much earlier (but
   I won't be able to pull that off), so I'm not entering this rabbit hole
   any deeper.

2. Digging into io_method=worker optimizations (answering question: are they
   necessary?) Maybe I'll throw in run5 quite soon, this is going to be
   crucial to answer.

3. Potentially mentioned earlier BAS strategies (forcing just use of local
   partitions for known-to-be-only-local-users: CTAS/VACCUM/etc), but I'm
   afarid that's not for me as I would certainly break/violate some
   invisible to me boundary.

Maybe You could run those run*.sh with master vs inst-patchset/optimized?
(I'm not sure, maybe there's even different factor at play too...)

-J.
From 8513d188ed5ed999e72fc3a58046bbc1ff9f5688 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email-masked]>
Date: Tue, 30 Jun 2026 14:22:02 +0200
Subject: [PATCH v20260630-0008] clock-sweep: cached CPU/NUMA node and more
 locality-aware balancing

Enhancements on top of 0001-0007, to have sligthly better NUMA locality
and perfromance.

1. Cache numa_node_of_cpu()/sched_getcpu() per backend in
   ClockSweepPartitionIndex(), refreshing every CLOCKSWEEP_CPU_NODE_REFRESH
   allocations rather than on every call (visible hot buffer path in perf)

2. CLOCKSWEEP_BALANCE_THRESHOLD - make it less likely to redirect on any
   surplus of allocations (so scatter buffers LESS onto remote nodes).
   With this, it redirects its allocations to other (remote?) partitions
   when the allocation exceeds the per-partition average allocation rate
   by this percentage factor .

3. Avoid redirects to "idle" partitions: a redirect partition target
   must have some traffic which is at least 2x our demand. This elimnates
   cold partitions, but we can still reach them using scan-all-partitions
   fallback.
---
 src/backend/storage/buffer/freelist.c | 85 +++++++++++++++++++++++----
 1 file changed, 74 insertions(+), 11 deletions(-)

index e677c71e0b3..d64c2c67eb6 100644
@@ -55,6 +55,9 @@
 
+/* How often backend should re-fetch the CPU/node on which it is running on? */
+#define CLOCKSWEEP_CPU_NODE_REFRESH	128
+
 /*
  * GUCs controlling the NUMA-aware clock-sweep behavior.
  *
@@ -70,6 +73,7 @@
  * clocksweep_scan_all_partitions - when enabled, looking for a free buffer
  * scans all clock-sweep partitions (in a round-robin way), not just the
  * backend's "home" partition.
+ *
  */
 bool		clocksweep_balance = true;
 bool		clocksweep_balance_recalc = true;
@@ -368,13 +372,29 @@ ClockSweepPartitionIndex(void)
 #ifdef USE_LIBNUMA
 	if (shared_buffers_numa)
 	{
-		int		cpu;
+		/*
+		 * Cache the CPU/NUMA node, refreshing only every CLOCKSWEEP_CPU_NODE_REFRESH
+		 * allocations. It appears that sched_getcpu()/numa_node_of_cpu() are not free.
+		 * On some platforms it take price of full system call, or the rest (x86_64?)
+		 * is can be use VDSO optimization. The backend rarely migrates between NUMA
+		 * nodes, and the balance logic only needs to notice migration after some time,
+		 * so an occasional refresh is good enough.
+		 */
+		static int		cached_node = -1;
+		static uint32	refresh_counter = 0;
+
+		if (cached_node < 0 || (refresh_counter++ % CLOCKSWEEP_CPU_NODE_REFRESH) == 0)
+		{
+		  int cpu;
 
-		/* XXX do we need to check sched_getcpu is available, somehow? */
-		if ((cpu = sched_getcpu()) < 0)
+		  /* XXX do we need to check sched_getcpu is available, somehow? */
+		  if ((cpu = sched_getcpu()) < 0)
 			elog(ERROR, "sched_getcpu failed: %m");
 
-		node = numa_node_of_cpu(cpu);
+		  /* XXX/JW: use libnuma wrapper for this */
+		  cached_node = numa_node_of_cpu(cpu);
+		}
+		node = cached_node;
 	}
 #endif
 
@@ -768,7 +788,8 @@ StrategySyncBalance(void)
 
 	uint32	total_allocs = 0,	/* total number of allocations */
 			avg_allocs,			/* average allocations (per partition) */
-			delta_allocs = 0;	/* sum of allocs above average */
+			delta_allocs = 0,	/* sum of allocs above average */
+			redirect_cutoff;	/* redirect only above this many allocs */
 
 	if (!clocksweep_balance || !clocksweep_balance_recalc)
 		return;
@@ -852,6 +873,20 @@ StrategySyncBalance(void)
 		return;
 	}
 
+	/*
+	 * A partition only redirects allocations to other partitions when it
+	 * exceeds the average by more than some threshold percent.
+	 * Below this cutoff we keep allocations local, to preserve NUMA locality.
+	 *
+	 * TODO: maybe better value is possible. On 4s with 25 I've got good results,
+	 *       but with value of 50 I've got slight degradation. Maybe it should 
+	 *       be equal to 100/numa_nodes ?
+	 *
+	 */
+#define CLOCKSWEEP_CUTOFF_THRESHOLD 25
+	redirect_cutoff = avg_allocs +
+		(uint32) ((uint64) avg_allocs * CLOCKSWEEP_CUTOFF_THRESHOLD / 100);
+
 	/*
 	 * The actual rebalancing
 	 *
@@ -884,10 +919,15 @@ StrategySyncBalance(void)
 		/* reset the weights to start from scratch */
 		memset(balance, 0, sizeof(uint8) * MAX_BUFFER_PARTITIONS);
 
-		/* does this partition has fewer or more than avg_allocs? */
-		if (allocs[i] < avg_allocs)
+		/*
+		 * Does this partition exceed its fair share by more than the
+		 * threshold? If not, keep all allocations local - redirecting them
+		 * would push memory onto remote NUMA nodes for no real benefit when
+		 * the load is already close to balanced.
+		 */
+		if (allocs[i] <= redirect_cutoff)
 		{
-			/* fewer - don't redirect any allocations elsewhere */
+			/* near fair share (or below) - keep allocations local */
 			balance[i] = 100;
 		}
 		else
@@ -902,22 +942,45 @@ StrategySyncBalance(void)
 			/* fraction of the "total" delta */
 			double	delta_frac = (allocs[i] - avg_allocs) * 1.0 / delta_allocs;
 
-			/* keep just enough allocations to meet the target */
-			balance[i] = (100.0 * avg_allocs / allocs[i]);
+			/* how much we keep local; we hand out the rest below */
+			int		kept = 100;
 
 			/* redirect the extra allocations */
 			for (int j = 0; j < StrategyControl->num_partitions; j++)
 			{
 				/* How many allocations to receive from i-th partition? */
 				uint32	receive_allocs = delta_frac * (avg_allocs - allocs[j]);
+				int		w;
+
+				/* do not redirect to ourselves */
+				if (j == i)
+					continue;
 
 				/* ignore partitions that don't need additional allocations */
 				if (allocs[j] > avg_allocs)
 					continue;
 
+				/*
+				 * Only use other partitions that actually have demand of
+				 * their own (avoid idle). If we fail, there's always the
+				 * scan-all-partitions fallback.
+				 *
+				 * TODO:: just guessing,heuristics
+				 */
+				if (allocs[j] < (avg_allocs / 2))
+					continue;
+
 				/* fraction to redirect */
-				balance[j] = (100.0 * receive_allocs / allocs[i]) + 0.5;
+				w = (int) ((100.0 * receive_allocs / allocs[i]) + 0.5);
+				balance[j] = w;
+				kept -= w;
 			}
+
+			/* avoid negative balances */
+			if (kept > 0)
+				balance[i] = kept;
+			else
+				balance[i] = 1;
 		}
 
 		/* combine the old and new weights (hysteresis) */
-- 
2.43.0

---

From: Jakub Wartak <[email-masked]>
Date: Thu, 2 Jul 2026 11:24:21 +0200

[..]
>    I won't be able to pull that off), so I'm not entering this rabbit hole

See below, seems like not needed (?)

>    necessary?) Maybe I'll throw in run5 quite soon, this is going to be

OK, I'm attaching are results from mine runs 5 and 6:
- only seqconcurrscans was tested, well because for other workloads io_worker
  method was not getting load for those workers (only seq scans were offloaded)
- checksums were disabled, because IMHO that would be unfair comparision
  (AFAIR there are offloaded)
- those optimizations for 0008 "optimized (numa=on, bal=on)" easily beat
  "patched (numa=on, bal=on)" and seem to be crucial. We get like 1.2x-1.4x
  across every io_method, but only with 0008.
- even when then doing just those logical fully cached reads from fully VFS
  cached case, io_urings shines (I've added raw TPS number to show this,
  compare across tables e.g. io_uring vs sync  13.378/8.993=1.487x for
  io_uring with NUMA, but for master's for io_uring:sync it was just 8.79/7.389
  = 1.189x without NUMA); seems like io_uring is more lightweight to show
  more benefits of remote memory latencies
- there's some more juice to get out of the balancer for 0-reuse workloads
  (but IMHO it's pointless to squeeze more, it's hard already)
- I was probably wrong when expecting that io_worker's worker processes/queues
  should get NUMA affinity. They don't need to be apparently for me to see
  benefits (maybe they could  be and it would even better, but meh).
  So with ruling io_method impact (I speculated earlier that his could be it),
  this means that you were either hitting lack of opimizations needed from
  0008 or were impacted by lack of drop_caches before the runs

> (I'm not sure, maybe there's even different factor at play too...)

This is seems to be crucial now, to double confirm the results / loaded-tested
on your hw with 0008. (but that hardware really needs to have effective latency
difference between at least 2 NUMA nodes -- Intel's mlc is good for this);
maybe also tweak those 125% inside 0008 to some other values, I've got 4 nodes,
so 100/4=25%)

>    invisible to me boundary.

And this one is still potentially on the table as nice thing to have.

-J.