Subject: Re: Adding basic NUMA awareness

From: Andres Freund <[email-masked]>
Date: Mon, 12 Jan 2026 19:10:00 -0500

Hi,

On 2026-01-13 00:58:49 +0100, Tomas Vondra wrote:
> On 1/10/26 02:42, Andres Freund wrote:
> > psql -Xq -c 'SELECT pg_buffercache_evict_all();' -c 'SELECT numa_node, sum(size) FROM pg_shmem_allocations_numa GROUP BY 1;' && perf stat --per-socket  -M memory_bandwidth_read,memory_bandwidth_write -a psql -c 'SELECT sum(abalance) FROM pgbench_accounts;'

> And then I initialized pgbench with scale that is much larger than
> shared buffers, but fits into RAM. So cached, but definitely > NB/4. And
> then I ran
> 
>   select * from pgbench_accounts offset 1000000000;
> 
> which does a sequential scan with the circular buffer you mention abobe

Did you try it with the query I suggested? One plausible reason why you did
not see an effect with your query is that with a huge offset you actually
never deform the tuple, which is an important and rather latency sensitive
path.

Greetings,

Andres Freund

---

From: Andres Freund <[email-masked]>
Date: Mon, 12 Jan 2026 19:24:34 -0500

Hi,

> never deform the tuple, which is an important and rather latency sensitive

Btw, this doesn't need anywhere close to as much data, it should be visible as
soon as you're >> L3.

To show why
  SELECT * FROM pgbench_accounts OFFSET 100000000
doesn't show an effect but
  SELECT sum(abalance) FROM pgbench_accounts;

does, just look at the difference using the perf command I posted. Here on a
scale 200.

numactl --membind 0 --cpunodebind 0

offset:

S0        6         47,138,135      UNC_M_CAS_COUNT.WR               #   3884.1 MB/s  memory_bandwidth_write
S0       20        780,343,577      duration_time
S0        6         61,685,331      UNC_M_CAS_COUNT.RD               #   5082.8 MB/s  memory_bandwidth_read
S0       20        780,353,818      duration_time
S1        6          1,238,568      UNC_M_CAS_COUNT.WR               #    102.1 MB/s  memory_bandwidth_write
S1        6          1,475,224      UNC_M_CAS_COUNT.RD               #    121.6 MB/s  memory_bandwidth_read

       0.776715450 seconds time elapsed

agg:

S0        6         53,145,706      UNC_M_CAS_COUNT.WR               #   2000.8 MB/s  memory_bandwidth_write
S0       20      1,706,046,493      duration_time
S0        6        111,390,488      UNC_M_CAS_COUNT.RD               #   4193.5 MB/s  memory_bandwidth_read
S0       20      1,706,057,341      duration_time
S1        6          3,968,454      UNC_M_CAS_COUNT.WR               #    149.4 MB/s  memory_bandwidth_write
S1        6          4,026,212      UNC_M_CAS_COUNT.RD               #    151.6 MB/s  memory_bandwidth_read

numactl --membind 0 --cpunodebind 1

offset:

S0        6         91,982,003      UNC_M_CAS_COUNT.WR               #   7036.4 MB/s  memory_bandwidth_write
S0       20        842,785,290      duration_time
S0        6        113,076,316      UNC_M_CAS_COUNT.RD               #   8650.1 MB/s  memory_bandwidth_read
S0       20        842,797,430      duration_time
S1        6          1,545,612      UNC_M_CAS_COUNT.WR               #    118.2 MB/s  memory_bandwidth_write
S1        6          2,354,087      UNC_M_CAS_COUNT.RD               #    180.1 MB/s  memory_bandwidth_read

       0.836623794 seconds time elapsed

agg:

S0        6        133,267,754      UNC_M_CAS_COUNT.WR               #   3980.9 MB/s  memory_bandwidth_write
S0       20      2,146,221,284      duration_time
S0        6        159,951,549      UNC_M_CAS_COUNT.RD               #   4777.9 MB/s  memory_bandwidth_read
S0       20      2,146,233,675      duration_time
S1        6         71,543,708      UNC_M_CAS_COUNT.WR               #   2137.1 MB/s  memory_bandwidth_write
S1        6         49,584,957      UNC_M_CAS_COUNT.RD               #   1481.2 MB/s  memory_bandwidth_read

       2.142535432 seconds time elapsed

Note how much bigger the absolute numbers of reads and writes are for the
aggregate compared to the offset.

Interestingly I do see a performance difference, albeit a smaller one, even
with OFFSET. I see similar numbers on two different 2 socket machines.

Greetings,

Andres Freund

---

From: Tomas Vondra <[email-masked]>
Date: Tue, 13 Jan 2026 01:51:09 +0100

> never deform the tuple, which is an important and rather latency sensitive

I did try with the agg query too, and there's still no difference on
either machine.

I can't do the perf on the Azure VM, because the Ubuntu is image is
borked and does not allow installing the package. But on my xeon I can
do the perf, and that gives me this:

numactl --membind=0 --cpunodebind=0 ~/builds/master-test/bin/pg_ctl
-----------------------------------------------------------------------
S0 1     24,677,226    UNC_M_CAS_COUNT.WR #    79.0 MB/s ... idth_write
S0 1 20,001,829,522 ns duration_time                     ...
S0 1    972,631,426    UNC_M_CAS_COUNT.RD #  3112.2 MB/s ... idth_read
S0 1 20,001,822,807 ns duration_time                     ...
S1 1     15,602,233    UNC_M_CAS_COUNT.WR #    49.9 MB/s ... idth_write
S1 1    712,431,146    UNC_M_CAS_COUNT.RD #  2279.6 MB/s ... idth_read

numactl --membind=0 --cpunodebind=1 ~/builds/master-test/bin/pg_ctl
-----------------------------------------------------------------------
S0 1     47,931,019    UNC_M_CAS_COUNT.WR #    153.4 MB/s ... idth_write
S0 1 20,002,933,380 ns duration_time                      ...
S0 1  1,007,386,994    UNC_M_CAS_COUNT.RD #   3223.2 MB/s ... idth_read
S0 1 20,002,927,341 ns duration_time                      ...
S1 1     10,310,201    UNC_M_CAS_COUNT.WR #     33.0 MB/s ... idth_write
S1 1    714,826,668    UNC_M_CAS_COUNT.RD #   2287.2 MB/s ... idth_read

so there is a little bit of a difference for some stats, but not much.

FWIW this is from

perf stat --per-socket  -M memory_bandwidth_read,memory_bandwidth_write
-a -- sleep 20

while the agg query runs in a loop.

cheers
-- 
Tomas Vondra

---

From: Andres Freund <[email-masked]>
Date: Mon, 12 Jan 2026 20:08:31 -0500

Hi,

> I did try with the agg query too, and there's still no difference on

Could you provide numactl --hardware for both?  There may be more than two
numa nodes on a system with 2 sockets, due to one socket being split into two
- in which case the latency between 0,1 might be a lot lower than say 0 and 3.

> numactl --membind=0 --cpunodebind=0 ~/builds/master-test/bin/pg_ctl

FWIW doing one perf stat for each execution is preferrable for comparison,
because otherwise you can hide large differences in total number of memory
accesses if the runtimes for the queries in the two "numa configurations" are
different.

Greetings,

Andres Freund

---

From: Tomas Vondra <[email-masked]>
Date: Tue, 13 Jan 2026 02:13:40 +0100

> does, just look at the difference using the perf command I posted. Here on a

OK, I tried with smaller scale (and larger shared buffers, to make the
data set smaller than NBuffers/4).

On the azure VM (scale 200, 32GB sb), there's still no difference:

297.770 ms

297.924 ms

and on xeon (scale 100, 8GB sb), there's a bit of a difference:

236.451 ms

298.418 ms

So roughly 20%. There's also a bigger difference in the perf, about
5944.3 MB/s vs. 5202.3 MB/s.

> with OFFSET. I see similar numbers on two different 2 socket machines.

I wonder how significant is the number of sockets. The Azure is a single
socket with 2 NUMA nodes, so maybe the latency differences are not
significant enough to affect this kind of tests.

The xeon is a 2-socket machine, but it's also older (~10y).

regards

-- 
Tomas Vondra

---

From: Tomas Vondra <[email-masked]>
Date: Tue, 13 Jan 2026 02:25:57 +0100

> - in which case the latency between 0,1 might be a lot lower than say 0 and 3.

xeon:

available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 44
45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
node 0 size: 32066 MB
node 0 free: 13081 MB
node 1 cpus: 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41
42 43 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87
node 1 size: 32210 MB
node 1 free: 17764 MB
node distances:
node     0    1
   0:   10   21
   1:   21   10

azure/epyc:

available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47
node 0 size: 193412 MB
node 0 free: 147949 MB
node 1 cpus: 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91
92 93 94 95
node 1 size: 193513 MB
node 1 free: 151577 MB
node distances:
node   0   1
  0:  10  11
  1:  11  10

> accesses if the runtimes for the queries in the two "numa configurations" are

Good point, I'll do that next time. But in this case they are not all
that different, I think.

regards

-- 
Tomas Vondra

---

From: Jakub Wartak <[email-masked]>
Date: Tue, 13 Jan 2026 14:26:37 +0100

> On the azure VM (scale 200, 32GB sb), there's still no difference:
[..]
> and on xeon (scale 100, 8GB sb), there's a bit of a difference:
[..]
> The xeon is a 2-socket machine, but it's also older (~10y).

My 0.02$ from intentionally having even more slow hardware with even
more sockets where some effects are even more visible (maybe this
helps the $thread)

There are two ways we could benefit from NUMA multi-socket systems:
a) potentially getting lower latency
b) avoid hitting interconnect max bandwidth limitations

- so it started with offline discussion yesterday that earlier we have
seen some yield from pgbench -S results, however I could not reproduce
the yield from [1] NUMAv2 -- I was trying to replicate that result "I
did some quick perf testing on my old xeon machine (2 NUMA nodes), and
the results are encouraging. For a read-only pgbench (2x shared
buffers, within RAM), I saw an increase from 1.1M tps to 1.3M." [1]. I
was using numa_buffers_interleave=on,numa_localalloc=on,numa_partition_freelist=node,numa_procs_interleave=on,numa_procs_pin=off,
no HPs (due to "Bad address" bug in that patchset), two -i pgbench
scales (2000, 500), with and without checksums, with/without -M
prepared. More or less it's always like that:
  jul17__64__off_pgbench.log:tps = 564153.278183 (without initial
connection time)
  numav2__64__off_pgbench.log:tps = 562068.655263 (without initial
connection time)
 so that probably rules out that we have recently introduced a bug
into the patchset (but that may mean that 1.1->1.3M boost was
something else?)

- so per above and in my opinion, both on master or all patchsets
here, classic OLTP pgbench (-S) is way too CPU computation heavy even
with -M prepared to see NUMA latency effects. Affects *both* NUMAv2
(Aug2025)[1] and Nov 2025 patchset versions. Even getting 0.5M tps on
pgbench scale -i 500 ends up using way less traffic before hitting the
QPI (interconnect) max link bandwidth (just 3.8-4.2 GB/s in my case,
but pgbench -S can consume just max 1.2GB/s assuming proper
interleaving).

- the single seqscan "SELECT sum(abalance) FROM pgbench_accounts;"
problem (or lack of it -- with single session) is that with standard
master, you may end up having data on just a single NUMA node. If that
system is idle and running just 1 instance of this query, I'm getting
like ~750MB/s of memory reads from the socket where the data is
located (again , much below the limit of the interconnect
[3.8-4.2GB/s])

- so yes when I do "synchronize_seqscans=off", and I start throwing
those seqscans from *other* sockets [--cpubindnodes 1,2,3 (while that
relation is hosted on node 0's DRAM)], of course I can see choke point
on the interconnect, (so it can feed data @ ~4.2 GB/s and even more
for short periods of time, but that's it). That's the
problem1(optimization1) realistically we can solve here I think, with
interleaving and that has been shown here multiple times: I can get
simply much more juice in the above scenario, because I have access to
better aggregated bandwidth for seqscans like that and simply put more
CPU cores (from CPUs on nodes 1,2,3) to feed that data from RAM on
node #0. It also helps achieve less deviation in latency in such
conditions. To add more fun Linux kernel likes to put randomly that
shm segments (e.g. if it fits single NUMA free hugepages, it will be
put there OR it uses some spanning across nodes , but not interleaving
and depending on relation you have it here or there). So far, so good
as far interleaving is concerned.

- however the above might be simply not true on single-socket NUMA
systems (EPYC) or just more modern multi-socket (but still same
chassis NUMA systems) - so EPYC again?(~500GB/s wild interconnect)? -
as the interconnect there might be not present (1s, BIOS/UEFI may have
setting to expose group of cores [CCD] as NUMA) OR have the bandwidth
and there's no realistic chokepoint for us there, so it wouldn't make
sense to benchmark using such hardware to spot the effects.

- So the remaining question remains, what about having better
locality/latency of let's say ~100ns on local DRAM vs 3x as much as
the remote [3][4] - how much we can gain?  (BTW this seems to be a
pretty universal rule of thumb -> 3x factor for most hardware, see
[5). If just shm is remote to the CPU for this query I'm getting:
remote: Time: 1925.820 ms
local:  Time: 1796.678 ms
so 1.071x.

and in some different situations if both shm and heap memory are
remote to the CPU for this query I was getting:
local:  Time: 9773.179 ms (00:09.773)
remote: Time: 12154.636 ms (00:12.155) +/- 300ms (+ I can see more
traffic on sockets).

So technically I should be getting this ~7%..22% profit due to lower
latency if I would be fetching just ONLY local memory (but with NUMA
we are not doing it right? we are interleaving - so we hit all sockets
most of the time to fetch data). So think about benchmarking: maybe we
would have to be using multiple pgbench_accounts_N (not just one, but
each per backend/CPU pgbench_accounts_$CPU?), together larger than s_b
to cause natural evictions, as this would then cause the partitioned
clocksweep to make it more likely that data is on the local NUMA node
and keep it hot for some time while avoiding any access to VFS cache
(so stick to reading only s_b)? (and we could measure local vs remote
access ratio too along the way). Also just for research, we should
disable clocksweep balancing. It should be enough to demonstrate that
the patch is working, right? (so 1 backend should be just reading from
1 NUMA node mostly and that should be producing some yield and it
should be having high local access ratio and just build up from there
if necessary rather than enabling all of the v20251126* patches)

XXX but eviction would be followed up by something like pread() from
VFS, and what about if that VFS cache is also on another node?

XXX those BAS strategies are just pain and yet another thing to care
about, couldn't we have some debug GUC to turn them off
(debug_benchmarking=true)?

-J.

[1] - https://www.postgresql.org/message-id/3223cdcd-6d16-4e90-a3a6-b957f762dc5a%40vondra.me

[2] - mlc --bandwidth_matrix: //that's sequential memory reads, but
with -U it is pretty close (90%) to the below ones
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
                Numa node
Numa node            0       1       2       3
       0        22677.4  3784.2  3720.6  4199.1
       1         3855.7 22463.1  4226.8  3732.0
       2         3713.7  4228.3 21816.6  3886.3
       3         4190.7  3692.9  3796.3 22673.0

[3] mlc --latency_matrix:
Measuring idle latencies (in ns)...
                Numa node
Numa node            0       1       2       3
       0          96.7   296.0   300.7   292.5
       1         289.7    96.9   289.7   303.8
       2         309.9   289.3    97.0   291.0
       3         300.0   303.7   296.5    97.1

[4] numactl --hardware lies a little bit (real latency vs below numbers):
node distances:
node   0   1   2   3
  0:  10  20  30  20
  1:  20  10  20  30
  2:  30  20  10  20
  3:  20  30  20  10

[5] https://github.com/nviennot/core-to-core-latency?tab=readme-ov-file#dual-sockets-results

---

From: Andres Freund <[email-masked]>
Date: Tue, 13 Jan 2026 09:14:17 -0500

Hi,

> On the azure VM (scale 200, 32GB sb), there's still no difference:

One possibility is that the host is configured with memory interleaving. That
configures the memory so that physical memory addresses interleave between the
different NUMA nodes, instead of really being node local. That can help avoid
bad performance characteristics for NUMA naive applications.

I don't quite know how to figure that out though, particularly from within a
VM :(.  Even something like https://github.com/nviennot/core-to-core-latency
or intel's mlc will not necessarily be helpful, because it depends on which
node the measured cacheline ends up on.

But I'd probably still test it, just to see whether you're observing very
different latencies between the systems.

> significant enough to affect this kind of tests.

Ah, yes, a single socket machine might not show that much of an increase, at
least in simpler cases.  One of my workstations has two sockets, but each
socket has two numa nodes, the latency difference between the same numa node
and the other numa node in the same socket is small, but the difference to the
other socket is ~1.5x.

Using intel's mlc:

Measuring idle latencies for sequential access (in ns)...
		Numa node
Numa node	     0	     1	     2	     3
       0	  98.6	 106.9	 157.6	 167.9
       1	 105.8	  99.4	 158.4	 170.5
       2	 157.2	 167.4	 103.6	 105.6
       3	 158.4	 171.2	 104.5	 104.3

So there's a about a 2-10ns latency difference between 0,1 and 2,3, but about
a 50-60ns diffence across sockets...

> The xeon is a 2-socket machine, but it's also older (~10y).

It's perhaps worth noting that memory access latency has been *in*creasing in
the last generation or two of hardware...

Greetings,

Andres Freund

---

From: Andres Freund <[email-masked]>
Date: Tue, 13 Jan 2026 09:37:15 -0500

Hi,

On 2026-01-13 14:26:37 +0100, Jakub Wartak wrote:
> with -M prepared to see NUMA latency effects.

I don't think it's that it's too CPU computation heavy, it's that it's very
latency sensitive to a small number of cachelines (ProcArrayLock, buffer
mapping table locks, btree inner pages), which will fundamentally have to
reside on one of the nodes. For pgbench -S to benefit we'd first need to
address at the very least the btree root page contention.

> located (again , much below the limit of the interconnect

The limit of the interconnect should be pretty much irrelevant for a single
query, you're pretty much never going to hit that query.

On my ~8 year old workstation (2x Xeon Gold 5215), with slow-ish RAM:

./mlc --bandwidth_matrix
Intel(R) Memory Latency Checker - v3.11a
Command line parameters: --bandwidth_matrix

Using buffer size of 100.000MiB/thread for reads and an additional 100.000MiB/thread for writes
Measuring Memory Bandwidths between nodes within system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
		Numa node
Numa node	     0	     1
       0	65173.3	34092.5
       1	33726.3	71244.1

If you're seeing ~3.5GB/s, as your [2] indicates, something either is wrong
with that system, or it's so old that it's useless for benchmarking. That's
worse than single core node-node numbers I've gotten on 10yo hardware.

The reason you're just getting 750MB is presumably because you're *latency*
limited, not because you're bandwidth limited. The problem is that our
deforming code has a, currently, unpredictable memory access at the start that
cannot meaningfully be hidden by out of order execution (because it determines
the address of the first column to actually deform, which cannot be hidden by
speculative execution).

> systems (EPYC)

EPYC supports both single and dual socket systems. And intel has
numa-within-a-socket too...

> or just more modern multi-socket (but still same chassis NUMA systems) - so
> EPYC again?(~500GB/s wild interconnect)?

I don't think EPYC, even in the newer iterations, has anywhere near a 500GB/s
interconnect. But it's really irrelevant, latency is the main factor, not
bandwidth.

> most of the time to fetch data)

We should *not* be interleaving unnecessarily, precisely because of this. We
should use the partitioned clock sweep to default to using local memory as
long as possible.

Greetings,

Andres Freund

---

From: Tomas Vondra <[email-masked]>
Date: Thu, 15 Jan 2026 00:26:47 +0100

> different latencies between the systems.

I did this on the two Azure instances I've been using for testing (D96
and HB176), and I got this:

D96 (v6):

                Numa node
Numa node            0       1
       0         129.9   129.9
       1         128.3   128.1

HB176 (v4):

                Numa node
Numa node            0       1       2       3
       0         107.3   116.8   207.3   207.0
       1         120.5   110.6   207.5   207.1
       2         207.0   207.2   107.8   116.8
       3         204.4   204.7   117.7   107.9

I guess this confirms that D96 is mostly useless for evaluation of the
NUMA patches. This is a single-socket machine, with one NUMA node per
chiplet (I assume), and there's about no difference in latency.

For HB176 there clearly seems to be a difference of ~90ns between the
sockets, i.e. the latency about doubles in some cases. Each socket has
two chiplets - and there the story is about the same as on D96.

I did this on my old-ish Xeon too, and it's somewhere in between. There
clearly is difference between the sockets, but it's smaller than on
HB176. Which matches with your observation that the latency is really
increasing over time.

I doubt the interleaving mode is enabled. It clearly is not enabled on
the HB176 machine (otherwise we wouldn't see the difference, I think),
and the smaller instance can be explained by having a single socket.

I've attached the complete mlc results, for completeness.

I've also done bigger SQL test with pinning the memory/backends to
different nodes, for a range of scales and the two queries (agg and
offset). I'm attaching results for scale 100 and 10000 from D96 and
HB176 instances.

The numbers are timings per query (avg latency reported by pgbench). I
think this mostly aligns with the mlc results - the D96 shows no
difference, while HB176 shows clear differences when memory/cpu get
pinned to different sockets (but not chiplets in the same socket).

But there are some interesting details too, particularly when it comes
to behavior of the two queries. The "offset" query is affected by
latency even with no parallelism (max_parallel_workers_per_gather=0),
and it shows ~30% hit for cross-socket runs. But for "agg" there's no
difference in that case, and the hit is visible only with 4 or 8
workers. That's interesting.

Anyway, my plan at this point is to revive the old patch (before
changing direction to the simple patch), and see if we can observe a
difference on the "right" hardware. Maybe some of the results with no
improvements were due to this. This workload seems much more realistic.

regards

-- 
Tomas Vondra

---

From: Andres Freund <[email-masked]>
Date: Wed, 14 Jan 2026 19:01:38 -0500

Hi,

> Numa node            0       1

I wonder if D96 has turned on memory interleaving... These numbers are so
close to each other that they're a tad hard to believe.

> two chiplets - and there the story is about the same as on D96.

It looks to me like within a socket there is a latency difference of about
10ns? Only when going between sockets there's no difference between which of
the remote nodes is accessed - which makes sense to me.

For newer single-node EPYC
https://chipsandcheese.com/p/amds-epyc-9355p-inside-a-32-core
has some numbers for within socket latencies. They also see about 10ns between
inside-socket-local and inside-socket-remote.

> HB176. Which matches with your observation that the latency is really

FWIW https://chipsandcheese.com/p/a-look-into-intel-xeon-6s-memory has some
numbers in the "NUMA/Chiplet Characteristics" too.

One aspect in it caught my eye:
> Thus accesses to a remote NUMA node are only cached by the remote die’s
> L3. Accessing the L3 on an adjacent die increases latency by about 24
> ns. Crossing two die boundaries adds a similar penalty, increasing latency
> to nearly 80 ns for a L3 hit

Afaict that translates to an L3 hit consistently taking 80ns when accessing
remote memory, that's quite something.

> and the smaller instance can be explained by having a single socket.

As you say, there obviously is no interleaving on the HB176. I do wonder about
the D96, but ...

I wonder if the configuration is somehow visible in MSRs...

> pinned to different sockets (but not chiplets in the same socket).

Yea, that makes sense.

> workers. That's interesting.

Huh, that *is* interesting. I guess the hardware prefetchers are good enough
to prefetch of the tuple headers in this case, possibly because the tuples are
small and regular enough that the hardware prefetchers manage to prefetch
everything in time?

E.g. https://docs.amd.com/api/khub/documents/goX~9ubv8i5r60A_Qrp3Rw/content
documents "L1 Stride Prefetcher" as
> The prefetcher uses the L1 cache memory access history of individual
> instructions to fetch additional lines when each access is a constant
> distance from the previous.

Greetings,

Andres Freund

---

From: Jakub Wartak <[email-masked]>
Date: Wed, 21 Jan 2026 11:30:03 +0100

On Thu, Jan 15, 2026 at 12:26 AM Tomas Vondra <[email-masked]> wrote:
[..]
> improvements were due to this. This workload seems much more realistic.

I think I have an answer why Your patch is misbehaving, but I might be wrong.
First my 4s16c64t hw numbers from master back from ~21st Nov 2025 (so here
without Your's patch), s_b=8GB, huge_pages, pgbench -i scale 150 (so it flies
under the radar NBuffers/4), which gave me ~1922MB pgbench_accounts, and then
I do the select sum(abalance):

--membind=0 --cpunodebind=0 latency average = 2468.321 ms, stddev = 0.479 ms
S0 @ 825MB/s (uncore_imc/cas_count_read/)

--membind=0 --cpunodebind=1 latency average = 2780.653 ms, stddev = 2.080 ms
S0 @ 729MB/s (uncore_imc/cas_count_read/)

(2 socket hops as old UPI/QPI had max 2 interlinks)
--membind=0 --cpunodebind=2 latency average = 2811.500 ms stddev = 1.958 ms
--membind=0 --cpunodebind=3 latency average = 2777.305 ms stddev = 1.314 ms

So in ideal conditions I should be getting a 13-14% boost if all is
well. However
with the patchset (v20251121, debug_numa='buffers,procs'), it gets somewhat
worse numbers (worse than above than on master).

--membind=0 --cpunodebind=0 (we know that patchset code will
"interleave" anyway)
latency average = 2885.806 ms stddev = 20.349 ms
and we are reading RAM from four sockets, each @ 170-180 MB/s (total ~720MB/s)

The patch spread the 1922MB relation (which would fit into one node) into many:

postgres# select numa_node, count(*) from pg_buffercache_numa where bufferid in
  (select bufferid from pg_buffercache  where relfilenode =
  (select relfilenode from pg_class where relname = 'pgbench_accounts'))
group by 1 order by 2;
 numa_node | count
-----------+-------
         2 | 55404
         3 | 55405
         1 | 55415
         0 | 79678

Also the pg_buffercache_partitions.weights indicates "{24,24,24,24}" (%) split.

Now if I do this command-trick `migratepages(1) <pid> 1-3 0`, it
really disarms our
patchset semi-manual-interleave (numa_maps bind:1-3 to real numa 0) and it
*does* restore performance from 2900ms to 2600ms, therefore it just means it is
non optimal memory placement (by clocksweep balancing).

So the question is why such a table (1922MB with 245902 relpages) would end up
spreading so quickly to the other sockets? I've assumed that hitting 4 sockets
instead of 1 will be too slow and disarm the patchset partitioned clocksweep
balancing like below:
@@ -497,7 +497,9 @@ StrategyGetBuffer(BufferAccessStrategy strategy, [...]
-       sweep = ChooseClockSweep(true);
+       sweep = ChooseClockSweep(false);

and this in next tries gets me perfect (thanks to no balancing) weight there:
postgres=# select partition, numa_node, total_allocs, num_allocs, weights
from pg_buffercache_partitions;
 partition | numa_node | total_allocs | num_allocs |   weights
-----------+-----------+--------------+------------+-------------
         0 |         0 |       246186 |          0 | {100,0,0,0}
         1 |         1 |            0 |          0 | {0,100,0,0}
         2 |         2 |            0 |          0 | {0,0,100,0}
         3 |         3 |            0 |          0 | {0,0,0,100}

which yields (compare to initial measurements of 2468 .. 2780.. 2811ms):
latency average = 2737.440 ms
latency stddev = 5.715 ms
socket 0 reliably outputs 725MB/s constant (other are idle as expected)

I think we may simply need to (re?)think of strategy on how/when to distribute,
because even If I fill just 112 MB of data fresh after startup, weight will
change from 100,0,0,0...

postgres=# create table tmp1 as select id, repeat('A', 1024) t
from generate_series(1, 100000) as id;
SELECT 100000
-- 112 MB
postgres=# \dt+ tmp1
[..]

and then scan it, I'm already having just 62% of local affinity (again
my whole s_b is 8GB, and per-NUMA-node is like 2GB, so we are under
radar of NBuffers/4 too):

postgres=# select partition, numa_node, total_allocs, num_allocs, weights
from pg_buffercache_partitions;
 partition | numa_node | total_allocs | num_allocs |    weights
-----------+-----------+--------------+------------+---------------
         0 |         0 |         2587 |          1 | {62,12,12,12}
         1 |         1 |            0 |          0 | {0,100,0,0}
         2 |         2 |            0 |          0 | {0,0,100,0}
         3 |         3 |            0 |          0 | {0,0,0,100}

-- double-confirmation:
postgres=# select numa_node, count(*), count(*) * 100.0 /
sum(count(*)) OVER() AS pct
from pg_buffercache_numa where bufferid in (
        select bufferid from pg_buffercache where relfilenode =
            (select relfilenode from pg_class where relname = 'tmp1')
) group by 1 order by 2;
 numa_node | count |         pct
-----------+-------+---------------------
         3 |  1680 | 11.7130307467057101
         2 |  1683 | 11.7339468730391132
         1 |  1690 | 11.7827511678170536
         0 |  9290 | 64.7702712124381231

so of course during the scanning in loop, we are stressing all the sockets here
pretty much. It gets even worse from there, as if I use multiple tmp* tables
like that from a single backend (but total size << 1GB), I end up with
"{24,24,24,24}" split (but all of them would fit my node).

Of course all of the above was written with assumption for getting
most of the latency ,
single backend and not having backends from different nodes.

But now If I would hypothetically benchmark pgbench -S on master(from Nov)
against Yourpatchset from back then, with low number of backends I would
be comparing single-node-hugepage allocation (on random node, because
it would fit) vspatchset doing interleaving memory. But if kernel would migrate
all those backends (in case of master) toward the node where most of
s_b is located,
Your patchset simply couldn't win this.

This brings me to a point where I'm suspicious of this clock-sweep balancing
idea (partitioning is fine, it's just the balancing which seems to be kicking
too prematurely). BTW: much earlier You seem to have benchmarked

My thoughts for today are like following how it should work if You want to have
"demonstrate any benefits on other workloads":

1. We should do not pin backends to specific CPU/numa nodes, as the kernel
should be free to move the processes closer to the data more requested (it
knows the state of memory transfers and CPU util% across nodes better than
we do).
- we query for sched_getcpu() and get node
- we stick to PgProc and Buffers from that node and that's all
(we seem to be doing just that in the patchset, great!)

2. We then should try to stick to the local node as much as possible till it's
almost full and maybe only then try to start using remote memory as a
last resort. Or maybe
even try to avoid it at all costs.
- wouldn't eviction (as the first baby step) be more preferable rather
than using
  remote memory?
- we get local affinity boost, so no distribution to the other partitions
  (unless absolutely necessary?)
- ring buffers should protect somehow with one backend filling all RAM memory
  on the node  (well except pg_prewarm? maybe we should adjust it so
it intentionally
  interleaves from start?)
  -- related: NBuffers / 4 magic number drives me nuts, but maybe we should
     tweak to take into account the number of nodes too (NBuffers / nodes / 4?)
     to avoid filling whole node
- we would get more I/O as there would be potentially lower chance to find data
  in memory on that node(?), so we would need somehow to counter this
- maybe that's a little bit sci-fi, or I'm going to be flamed here for it,
  but we could potentially track local vs remote usagecount (Buffer state
  seems to be using 54 bits, so we could add 4 bits more to track "remote"
  access, AKA usagecount_remote? but we seem to not have space to track the
  exact origin/remote node). If we would track local vs remote, we would have
  have some input as if interleaving makes sense or not, wouldn't we?
  (that would somehow be tracked on per relation/"blockset", just food
for thought,
  dunno if we even have infra for that)

-J.

---

From: Tomas Vondra <[email-masked]>
Date: Fri, 5 Jun 2026 14:52:35 +0200

Hi,

Here's an updated version of the NUMA patch series, based on some recent
discussions about this (some at pgconf.dev, but not only that),

The main change is I significantly simplified some of the parts. Whe
patch from 20251126 was ~190K, the new version is maybe 100K, so about
half. Some of that is thanks to dropping the PGPROC partitioning
entirely, but the remaining parts are smaller too. I realize it's not a
great metric, of course.

In this message I'll explain the changes since 20251126. I'm yet to do a
thorough performance evaluation and see if it helps, I'll post that in
the next couple days.

The current patch series has these parts:

v20260605-0001-Add-shmem_populate-and-shmem_interleave-GU.patch
-------------

Somewhat unrelated, I find this useful for benchmarking and as a
baseline (what would happen if we just interleaved the shared segment).

v20260605-0002-Infrastructure-for-partitioning-of-shared-.patch
-------------

Just adds a small registry of partitions (ranges of shared buffers),
stored in shared memory, and pg_buffercache interface to inspect it.
Merely a foundation for the following patches.

v20260605-0003-NUMA-shared-buffers-partitioning.patch
-------------

The interesting part, that places some of the partitions to NUMA nodes.

v20260605-0004-clock-sweep-basic-partitioning.patch
v20260605-0005-clock-sweep-balancing-of-allocations.patch
v20260605-0006-clock-sweep-scan-all-partitions.patch
-------------

Patches that gradually partition clock-sweep. Ultimately, it should
probably be squashed into a single commit (each commit fixes some sort
of issue in the naive partitioning in 0004). But I kept them separate
because it's easier to review / understand what the issue is.

what changed?
-------------

First, I dropped the PGPROC partitioning. We may revisit that in the
future (not sure), but for now it was just a distraction and I see it as
less impactful than shared buffers / clock-sweep.

I also simplified the GUC to use a single on/off parameter (instead of
the debug_io_direct-like approach). We can revisit that, but for now
this seems more convenient.

The most significant change in the remaining parts is simplification of
the shared buffer partitioning. In particular, the partitioning is now
"best-effort" when it maps memory to shared buffers. Let me remind that
NUMA works at memory page granularity - we can't map arbitrary ranges of
memory to a node, it needs to be whole memory pages.

The 20251126 patch went into great lengths to (a) make sure BufferBlocks
and BufferDescriptors start at memory page boundary, are the partitions
are also properly aligned (both for blocks and descriptors). That was a
lot of code, it needs to happen even before we know if huge pages are
used, partitions might have been of (very) different sizes, and so on.

The new patch abandons this "perfect" partitioning, and instead does a
best-effort. It splits the buffers as evenly as possible, i.e. all
partitions have (NBuffers/npartitions) buffers, and then locates as much
memory as possible to a selected NUMA node.

With 4K pages, that's always the whole partition. With huge pages (which
is expected of relevant NUMA systems), there may be a couple buffers at
the beginning/end of a partition. But it's less than one memory page,
per partition, and we expect the systems to have 10s or 100s of GBs, so
in the bigger scheme of things it's negligible (fractions of a percent).

For buffer descriptors the math is a bit worse - descriptors need much
less memory, but even there it should not be more than ~1%.

Seems perfectly fine to me. Or rather, the extra complexity does not
seem worth the possible benefit.

This also allowed dropping a part of the "clock-sweep partitioning"
patches, dealing with cases when the partitions are of different sizes.
With this new best-effort scheme the difference is at most 1 buffer, and
we can just ignore that.

questions
---------

At this point, my main question is whether there's a better way to
partition clock-sweep and/or do the balancing of allocations between
partitions. I believe it does work, but I have a feeling there might be
a more elegant way to do this kind of stuff (like an established
balancing algorithm of some sort).

The other thing I need to verify is how this behaves with
kernel.nr_hugepages. With some earlier versions it was easy to end in a
situation where everything seemed to work, but then much later the
kernel realized it does not have enough huge pages on a particular NUMA
node and crashed with a segfault (or was it sigbus?).

Of course, the other question is performance validation - does it even
help? I plan to repeat the various experiments mentioned in this thread
(by Andres and others) on available NUMA machines. But if someone has an
idea for another benchmark (and/or what metric to measure, not just the
usual duration), let me know.

regards

-- 
Tomas Vondra
From 977831d85199b4b1b8d65de1131e4001459878fb Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 2 Jun 2026 22:39:38 +0200
Subject: [PATCH v20260605 6/6] clock-sweep: scan all partitions

When looking for a free buffer, scan all clock-sweep partitions, not
just the "home" one. All buffers in the home partition may be pinned, in
which case we should not fail. Instead, advance to the next partition,
in a round-robin way, and only fail after scanning through all of them.
---
 src/backend/storage/buffer/freelist.c     | 83 ++++++++++++++++-------
 src/test/recovery/t/027_stream_regress.pl |  5 --
 2 files changed, 57 insertions(+), 31 deletions(-)

diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index a543fb12b21..1ac1e3e3490 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -184,6 +184,9 @@ static BufferDesc *GetBufferFromRing(BufferAccessStrategy strategy,
 static void AddBufferToRing(BufferAccessStrategy strategy,
 							BufferDesc *buf);
 static ClockSweep *ChooseClockSweep(bool balance);
+static BufferDesc *StrategyGetBufferPartition(ClockSweep *sweep,
+											  BufferAccessStrategy strategy,
+											  uint64 *buf_state);
 
 /*
  * clocksweep allocation balancing
@@ -251,10 +254,9 @@ static int clocksweep_partition_budget = 0;
  * id of the buffer now under the hand.
  */
 static inline uint32
-ClockSweepTick(void)
+ClockSweepTick(ClockSweep *sweep)
 {
 	uint32		victim;
-	ClockSweep *sweep = ChooseClockSweep(true);
 
 	/*
 	 * Atomically move hand ahead one buffer - if there's several processes
@@ -486,7 +488,8 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint64 *buf_state, bool *from_r
 {
 	BufferDesc *buf;
 	int			bgwprocno;
-	int			trycounter;
+	ClockSweep *sweep,
+			   *sweep_start;		/* starting clock-sweep partition */
 
 	*from_ring = false;
 
@@ -545,33 +548,61 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint64 *buf_state, bool *from_r
 	/*
 	 * Use the "clock sweep" algorithm to find a free buffer
 	 *
-	 * XXX Note that ClockSweepTick() is NUMA-aware, i.e. it only looks at
-	 * buffers from a single partition, aligned with the NUMA node. That means
-	 * a process "sweeps" only a fraction of buffers, even if the other buffers
-	 * are better candidates for eviction. Maybe there should be some logic to
-	 * "steal" buffers from other partitions or other nodes?
-	 *
-	 * XXX This only searches a single partition, which can result in "no
-	 * unpinned buffers available" even if there are buffers in other
-	 * partitions. Needs to scan partitions if needed, as a fallback.
-	 *
-	 * XXX Would that also mean we should have multiple bgwriters, one for each
-	 * node, or would one bgwriter still handle all nodes?
-	 *
-	 * XXX Also, the trycounter should not be set to NBuffers, but to buffer
-	 * count for that one partition. In fact, this should not call ClockSweepTick
-	 * for every iteration. The call is likely quite expensive (does a lot
-	 * of stuff), and also may return a different partition on each call.
-	 * We should just do it once, and then do the for(;;) loop. And then
-	 * maybe advance to the next partition, until we scan through all of them.
+	 * Start with the "preferred" partition, and then proceed in a round-robin
+	 * manner. If we cycle back to the starting partition, it means none of the
+	 * partitions has unpinned buffers.
 	 */
-	trycounter = NBuffers;
+	sweep = ChooseClockSweep(true);
+	sweep_start = sweep;
+	for (;;)
+	{
+		buf = StrategyGetBufferPartition(sweep, strategy, buf_state);
+
+		/* found a buffer in the "sweep" partition, we're done */
+		if (buf != NULL)
+			return buf;
+
+		/*
+		 * Try advancing to the next partition, round-robin (if last partition,
+		 * wrap around to the beginning).
+		 *
+		 * XXX This is a bit ugly, there must be a better way to advance to the
+		 * next partition.
+		 */
+		if (sweep == &StrategyControl->sweeps[StrategyControl->num_partitions - 1])
+			sweep = StrategyControl->sweeps;
+		else
+			sweep++;
+
+		/* we've scanned all partitions */
+		if (sweep == sweep_start)
+			break;
+	}
+
+	/* we shouldn't get here if there are unpinned buffers */
+	elog(ERROR, "no unpinned buffers available");
+}
+
+/*
+ * StrategyGetBufferPartition
+ *		get a free buffer from a single clock-sweep partition
+ *
+ * Returns NULL if there are no free (unpinned) buffers in the partition.
+*/
+static BufferDesc *
+StrategyGetBufferPartition(ClockSweep *sweep, BufferAccessStrategy strategy,
+						   uint64 *buf_state)
+{
+	BufferDesc *buf;
+	int			trycounter;
+
+	trycounter = sweep->numBuffers;
 	for (;;)
 	{
 		uint64		old_buf_state;
 		uint64		local_buf_state;
 
-		buf = GetBufferDescriptor(ClockSweepTick());
+		buf = GetBufferDescriptor(ClockSweepTick(sweep));
 
 		/*
 		 * Check whether the buffer can be used and pin it if so. Do this
@@ -599,7 +630,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint64 *buf_state, bool *from_r
 					 * one eventually, but it's probably better to fail than
 					 * to risk getting stuck in an infinite loop.
 					 */
-					elog(ERROR, "no unpinned buffers available");
+					return NULL;
 				}
 				break;
 			}
@@ -618,7 +649,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint64 *buf_state, bool *from_r
 				if (pg_atomic_compare_exchange_u64(&buf->state, &old_buf_state,
 												   local_buf_state))
 				{
-					trycounter = NBuffers;
+					trycounter = sweep->numBuffers;
 					break;
 				}
 			}
diff --git a/src/test/recovery/t/027_stream_regress.pl b/src/test/recovery/t/027_stream_regress.pl
index f68e08e5697..ae977297849 100644
--- a/src/test/recovery/t/027_stream_regress.pl
+++ b/src/test/recovery/t/027_stream_regress.pl
@@ -18,11 +18,6 @@ $node_primary->adjust_conf('postgresql.conf', 'max_connections', '25');
 $node_primary->append_conf('postgresql.conf',
 	'max_prepared_transactions = 10');
 
-# The default is 1MB, which is not enough with clock-sweep partitioning.
-# Increase to 32MB, so that we don't get "no unpinned buffers".
-$node_primary->append_conf('postgresql.conf',
-	'shared_buffers = 32MB');
-
 # Enable pg_stat_statements to force tests to do query jumbling.
 # pg_stat_statements.max should be large enough to hold all the entries
 # of the regression database.
-- 
2.54.0

From ad46b08a3eaa018b562fdd4ca786c4c23d358ba5 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 2 Jun 2026 22:28:10 +0200
Subject: [PATCH v20260605 5/6] clock-sweep: balancing of allocations

If backends only allocate buffers from the "home" partition, that may
cause significant misbalance. Some partitions might be overused, while
other partitions would be left unused. In other words, shared buffers
would not be used efficiently.

We want all partitions to be used about the same, i.e. serve about the
same number of allocations. To achieve that, allocations from partitions
that are "too busy" may get redirected to other partitions. The system
counts allocations requested from each partition, calculates the "fair
share" (average per partition), and then redirectsexcess allocations to
other partitions.

Each partition gets a set of coefficients determining the fraction of
allocations to redirect to other partitions. The coefficients may be
interpreted as a "budget" for each of the partition, i.e. the number of
allocations to serve from that partition, before moving to the next
partition (in a round-robin manner).

All of this is tied to the partition where the allocation was requested.
Each partition has a separate set of coefficients.

We might also treat the coefficients as probabilities, and use PRNG to
determine where to direct individual requests. But a PRNG seems fairly
expensive, and the budget approach works well.

We intentionally keep the "budget" fairly low, with the sum for a given
partition 100. That means we get to the same partition after only 100
allocations, keeping it more balanced. It wouldn't be hard to make the
budgets higher (e.g. matching the number of allocations per round), but
it might also make the behavior less smooth (long period of allocations
from each partition).

This is very simple/cheap, and over many allocations it has the same
effect. For periods of low activity it may diverge, but that does not
matter much (we care about high-activity periods much more).
---
 .../pg_buffercache--1.7--1.8.sql              |   5 +-
 contrib/pg_buffercache/pg_buffercache_pages.c |  43 +-
 src/backend/storage/buffer/bufmgr.c           |   3 +
 src/backend/storage/buffer/freelist.c         | 428 +++++++++++++++++-
 src/include/storage/buf_internals.h           |   1 +
 src/include/storage/bufmgr.h                  |  12 +-
 6 files changed, 471 insertions(+), 21 deletions(-)

diff --git a/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql b/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
index 92176fed7f8..43d2e84f9d2 100644
--- a/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
+++ b/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
@@ -20,7 +20,10 @@ CREATE VIEW pg_buffercache_partitions AS
 	 num_passes bigint,			-- clocksweep passes
 	 next_buffer integer,		-- next victim buffer for clocksweep
 	 total_allocs bigint,		-- handled allocs (running total)
-	 num_allocs bigint);		-- handled allocs (current cycle)
+	 num_allocs bigint,			-- handled allocs (current cycle)
+	 total_req_allocs bigint,	-- requested allocs (running total)
+	 num_req_allocs bigint,		-- handled allocs (current cycle)
+	 weights int[]);			-- balancing weights
 
 -- Don't want these to be available to public.
 REVOKE ALL ON FUNCTION pg_buffercache_partitions() FROM PUBLIC;
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 739c63b0cfc..c91f2bc5b4a 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -15,6 +15,8 @@
 #include "port/pg_numa.h"
 #include "storage/buf_internals.h"
 #include "storage/bufmgr.h"
+#include "utils/array.h"
+#include "utils/builtins.h"
 #include "utils/rel.h"
 #include "utils/tuplestore.h"
 
@@ -31,7 +33,7 @@
 #define NUM_BUFFERCACHE_MARK_DIRTY_ALL_ELEM 3
 
 #define NUM_BUFFERCACHE_OS_PAGES_ELEM	3
-#define NUM_BUFFERCACHE_PARTITIONS_ELEM	9
+#define NUM_BUFFERCACHE_PARTITIONS_ELEM	12
 
 PG_MODULE_MAGIC_EXT(
 					.name = "pg_buffercache",
@@ -889,6 +891,8 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 
 	if (SRF_IS_FIRSTCALL())
 	{
+		TypeCacheEntry *typentry = lookup_type_cache(INT4OID, 0);
+
 		funcctx = SRF_FIRSTCALL_INIT();
 
 		/* Switch context when allocating stuff to be used in later calls */
@@ -920,6 +924,12 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 						   INT8OID, -1, 0);
 		TupleDescInitEntry(tupledesc, (AttrNumber) 9, "num_allocs",
 						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 10, "total_req_allocs",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 11, "num_req_allocs",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 12, "weigths",
+						   typentry->typarray, -1, 0);
 
 		funcctx->user_fctx = BlessTupleDesc(tupledesc);
 
@@ -941,11 +951,17 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 					first_buffer,
 					last_buffer;
 
-		uint64		buffer_total_allocs;
+		uint64		buffer_total_allocs,
+					buffer_total_req_allocs;
 
 		uint32		complete_passes,
 					next_victim_buffer,
-					buffer_allocs;
+					buffer_allocs,
+					buffer_req_allocs;
+
+		int		   *weights;
+		Datum	   *dweights;
+		ArrayType  *array;
 
 		Datum		values[NUM_BUFFERCACHE_PARTITIONS_ELEM];
 		bool		nulls[NUM_BUFFERCACHE_PARTITIONS_ELEM];
@@ -954,8 +970,16 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 						   &first_buffer, &last_buffer);
 
 		ClockSweepPartitionGetInfo(i,
-								   &complete_passes, &next_victim_buffer,
-								   &buffer_total_allocs, &buffer_allocs);
+								 &complete_passes, &next_victim_buffer,
+								 &buffer_total_allocs, &buffer_allocs,
+								 &buffer_total_req_allocs, &buffer_req_allocs,
+								 &weights);
+
+		dweights = palloc_array(Datum, funcctx->max_calls);
+		for (int i = 0; i < funcctx->max_calls; i++)
+			dweights[i] = Int32GetDatum(weights[i]);
+
+		array = construct_array_builtin(dweights, funcctx->max_calls, INT4OID);
 
 		values[0] = Int32GetDatum(i);
 		nulls[0] = false;
@@ -984,6 +1008,15 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		values[8] = Int64GetDatum(buffer_allocs);
 		nulls[8] = false;
 
+		values[9] = Int64GetDatum(buffer_total_req_allocs);
+		nulls[9] = false;
+
+		values[10] = Int64GetDatum(buffer_req_allocs);
+		nulls[10] = false;
+
+		values[11] = PointerGetDatum(array);
+		nulls[11] = false;
+
 		/* Build and return the tuple. */
 		tuple = heap_form_tuple((TupleDesc) funcctx->user_fctx, values, nulls);
 		result = HeapTupleGetDatum(tuple);
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 02c75f82e5b..62e541abebd 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -4135,6 +4135,9 @@ BgBufferSync(WritebackContext *wb_context)
 	/* assume we can hibernate, any partition can set to false */
 	bool		hibernate = true;
 
+	/* trigger partition rebalancing first */
+	StrategySyncBalance();
+
 	/* get the number of clocksweep partitions, and total alloc count */
 	StrategySyncPrepare(&num_partitions, &recent_alloc);
 
diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index 2d56579682e..a543fb12b21 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -35,6 +35,26 @@
 #define INT_ACCESS_ONCE(var)	((int)(*((volatile int *)&(var))))

+/*
+ * XXX We need to make ClockSweep fixed-size, so that we can have an array
+ * in shared memory. The easiest way is to pick a sufficiently high value
+ * that no system will actually need. 32 seems high enough.
+ *
+ * XXX We should enforce this in bufmgr.c, when initializing the partitions.
+ */
+#define MAX_BUFFER_PARTITIONS		32
+
+/*
+ * Coefficient used to combine the old and new balance coefficients, using
+ * weighted average, so that we don't flap too much. The higher the value, the
+ * more the old value affects the result.
+ *
+ * XXX Doesn't this obscure the interpretation of weights as probabilities to
+ * allocate from a given partition? Does it still sum to 100%? I don't think
+ * so, it's just a fraction of allocations to go from a given partition.
+ */
+#define CLOCKSWEEP_HISTORY_COEFF	0.5
+
 /*
  * Information about one partition of the ClockSweep (on a subset of buffers).
  *
@@ -68,9 +88,32 @@ typedef struct
 	uint32		completePasses; /* Complete cycles of the clock-sweep */
 	pg_atomic_uint32 numBufferAllocs;	/* Buffers allocated since last reset */
 
+	/*
+	 * Buffers that should have been allocated in this partition (but might
+	 * have been redirected to keep allocations balanced).
+	 */
+	pg_atomic_uint32 numRequestedAllocs;
+
 	/* running total of allocs */
 	pg_atomic_uint64 numTotalAllocs;
+	pg_atomic_uint64 numTotalRequestedAllocs;
 
+	/*
+	 * Weights to balance buffer allocations for all the partitions. Each
+	 * partition gets a vector of weights 0-100, determining what fraction
+	 * of buffers to allocate from that partition. So [75, 15, 5, 5] would
+	 * mean 75% allocations should go from partition 0, 15% from partition
+	 * 1, and 5% from partitions 2&3. Each partition gets a different vector
+	 * of weights.
+	 *
+	 * Backends use the budget from it's "home" partition, so that a busy
+	 * partitions (with a lot of processes on that NUMA node etc.) spread
+	 * the allocations evenly.
+	 *
+	 * XXX Allocate a fixed-length array, to simplify working with array of
+	 * the structs, etc.
+	 */
+	uint8		balance[MAX_BUFFER_PARTITIONS];
 } ClockSweep;
 
 /*
@@ -140,7 +183,66 @@ static BufferDesc *GetBufferFromRing(BufferAccessStrategy strategy,
 									 uint64 *buf_state);
 static void AddBufferToRing(BufferAccessStrategy strategy,
 							BufferDesc *buf);
-static ClockSweep *ChooseClockSweep(void);
+static ClockSweep *ChooseClockSweep(bool balance);
+
+/*
+ * clocksweep allocation balancing
+ *
+ * To balance allocations from clocksweep partitions, each partition gets a
+ * budget for allocating buffers from other partitions. A process that
+ * "exhausts" a budget in it's home partition gets redirected to the other
+ * partitions, driven by the budgets.
+ *
+ * For example, a partition may have budget [25, 25, 25, 25], which means
+ * each of the 4 partitions should get 1/4 of allocations. Or the buget
+ * can be [50, 50, 0, 0], which means all allocations will go to the first
+ * two partitions (one of them being the "home" one);
+ *
+ * We could do that based on a random number generator, but for now we
+ * simply treat the values as a budget, i.e. a number of allocations to
+ * serve from other partitions, and move in round-robin way.
+ *
+ * This is very simple/cheap, and over many allocations it has the same
+ * effect. For periods of low activity it may diverge, but that does not
+ * matter much (we care about high-activity periods much more).
+ *
+ * We intentionally keep the "budget" fairly low, with the sum for a given
+ * partition 100. That means we get to the same partition after only 100
+ * allocations, keeping it more balanced. We can make the budgets higher
+ * (say, to match the expected number of allocations, i.e. bout the average
+ * number of allocations from the past interval). Or maybe configurable.
+ *
+ * XXX We should always start allocating from the "home" partition, i.e.
+ * from from it, and only then redirect to other partitions.
+ *
+ * XXX It probably is not great all the processes from that "home"
+ * partition are coordinated, and move to between partitions at about the
+ * same time. Not sure what to do about this.
+ *
+ * XXX We should also prefer other partitions from the same NUMA node (if
+ * there are some). Probably by setting the budgets.
+ *
+ * FIXME Explain at which point are the budgets recalculated, by which
+ * process, and how that affects other processes allocating buffers.
+ */
+
+/*
+ * The "optimal" clock-sweep partition. After a backend gets moved to a
+ * different NUMA node, we restart the balancing so that it uses the
+ * correct "budget" from the new home partition.
+ */
+static int clocksweep_partition_home = -1;
+
+/*
+ * The partition the backend is currently allocating from (either the
+ * home one, or one of the redirected ones).
+ */
+static int clocksweep_partition_current = -1;
+
+/*
+ * The number of buffers to allocate from the current partition.
+ */
+static int clocksweep_partition_budget = 0;
 
 /*
  * ClockSweepTick - Helper routine for StrategyGetBuffer()
@@ -152,7 +254,7 @@ static inline uint32
 ClockSweepTick(void)
 {
 	uint32		victim;
-	ClockSweep *sweep = ChooseClockSweep();
+	ClockSweep *sweep = ChooseClockSweep(true);
 
 	/*
 	 * Atomically move hand ahead one buffer - if there's several processes
@@ -300,11 +402,68 @@ ClockSweepPartitionIndex(void)
  * and that's cheaper. But how would that deal with odd number of nodes?
  */
 static ClockSweep *
-ChooseClockSweep(void)
+ChooseClockSweep(bool balance)
 {
+	/* What's the "optimal" partition for this backend? */
 	int			index = ClockSweepPartitionIndex();
+	ClockSweep *sweep = &StrategyControl->sweeps[index];
+
+	/*
+	 * Was the process migrated to a different NUMA node? If the home partition
+	 * changed, we need to reset the budget and start over, so that we correctly
+	 * prefer "nearby" partitions etc.
+	 *
+	 * XXX Could this be a problem when processes move all the time? I don't
+	 * think so - if a process moves between many partitions, that alone will
+	 * spread the allocations over partitions. Similarly, if there are many
+	 * processes, that should make it even more even.
+	 */
+	if (clocksweep_partition_home != index)
+	{
+		clocksweep_partition_home = index;
+		clocksweep_partition_current = index;
+		clocksweep_partition_budget = sweep->balance[index];
+	}
+
+	/* we should have a valid partition */
+	Assert(clocksweep_partition_home != -1);
+	Assert(clocksweep_partition_current != -1);
+	Assert(clocksweep_partition_budget >= 0);
+
+	/*
+	 * When balancing allocations, redirect the allocations to other partitions
+	 * according to the budgets. We move through partitions in a round-robin way,
+	 * after allocating the "budget" of allocations from the current one.
+	 */
+	if (balance)
+	{
+		/*
+		 * Ran out of budget from the current partition? Move to the next one
+		 * with non-zero budget.
+		 */
+		while (clocksweep_partition_budget == 0)
+		{
+			/* wrap around at the end */
+			clocksweep_partition_current++;
+			if (clocksweep_partition_current >= StrategyControl->num_partitions)
+				clocksweep_partition_current = 0;
+
+			clocksweep_partition_budget
+				= sweep->balance[clocksweep_partition_current];
+		}
 
-	return &StrategyControl->sweeps[index];
+		/* account for the current allocation */
+		--clocksweep_partition_budget;
+
+		/*
+		 * Account for the allocation in the "home" partition, so that the next
+		 * round of rebalancing (recalculating the budgets) knows about the
+		 * allocation traffic in various partitions.
+		 */
+		pg_atomic_fetch_add_u32(&sweep->numRequestedAllocs, 1);
+	}
+
+	return &StrategyControl->sweeps[clocksweep_partition_current];
 }
 
 /*
@@ -381,7 +540,7 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint64 *buf_state, bool *from_r
 	 * between CPUs / NUMA nodes in between, these call may pick different
 	 * partitions, confusing the logic a bit.
 	 */
-	pg_atomic_fetch_add_u32(&ChooseClockSweep()->numBufferAllocs, 1);
+	pg_atomic_fetch_add_u32(&ChooseClockSweep(false)->numBufferAllocs, 1);
 
 	/*
 	 * Use the "clock sweep" algorithm to find a free buffer
@@ -485,6 +644,229 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint64 *buf_state, bool *from_r
 	}
 }
 
+/*
+ * StrategySyncBalance
+ *		update partition budgets, to balance the buffer allocations
+ *
+ * We want to give preference to allocating buffers on the same NUMA node,
+ * but that might lead to imbalance - a single process would only use a
+ * fraction of shared buffers. We don't want that, we want to utilize the
+ * whole shared buffers. The number of allocations in each partition may
+ * also change over time, so we need to adapt to that.
+ *
+ * To allow this "adaptive balancing", each partition has a set of weights,
+ * determining what fraction of allocations to direct to other partitions.
+ * For simplicity the coefficients are integers 0-100, expressing the
+ * percentage of allocations redirected to that partition.
+ *
+ * Consider for example weights [50, 25, 25, 0] for one of 4 partitions.
+ * This means 50% of allocations will be redirected to partition 0, 25%
+ * to partitions 1 and 2, and no allocations will go to partition 3.
+ *
+ * This means an allocation may be requested in partition A (i.e. the
+ * home partition of the process requesting it), but end up allocating
+ * the buffer in partition B. We have a counter for both - the number of
+ * allocations requested in a partition, and the number of allocations
+ * actually handled by that partition. The former is used for calculating
+ * weights, the latter is used only for monitoring.
+ *
+ * The balancing happens in intervals - it adjusts future allocations
+ * based on stats about recent allocations, namely:
+ *
+ * - numBufferAllocs - number of allocations served by a partition
+ *
+ * - numRequestedAllocs - number of allocatios requested in a partition
+ *
+ * We're trying to smooth numBufferAllocs in the next interval, based on
+ * numRequestedAllocs measured in the last interval.
+ *
+ * The balancing algorithm works like this:
+ *
+ * - the target (average number of allocations per partition) is calculated
+ *   from total number of allocations requested in the last intervaal
+ *
+ * - partitions get divided into two groups - those with more allocation
+ *   requests than the target, and those with fewer requests
+ *
+ * - we "distribute" the delta (which is the same between the groups)
+ *   between the groups (one has more, the other fewer)
+ *
+ * Partitions with (nallocs > avg_nallocs) redirect the extra allocations,
+ * with each target allocation getting a proportional part (with respect
+ * to the total delta).
+ *
+ * XXX Currently this does not give preference to other partitions on the
+ * same NUMA node (redirect to it first), but it could.
+ */
+void
+StrategySyncBalance(void)
+{
+	/* snapshot of allocation requests for partitions */
+	uint32	allocs[MAX_BUFFER_PARTITIONS];
+
+	uint32	total_allocs = 0,	/* total number of allocations */
+			avg_allocs,			/* average allocations (per partition) */
+			delta_allocs = 0;	/* sum of allocs above average */
+
+	/*
+	 * Collect the number of allocations requested in the past interval.
+	 * While at it, reset the counter to start the new interval.
+	 *
+	 * XXX We lock the partitions one by one, so this is not a perfectly
+	 * consistent snapshot of the counts, and the resets happen before we
+	 * update the weights too. But we're only looking for heuristics, so
+	 * this should be good enough.
+	 *
+	 * XXX A similar issue applies to the counter reset later - we haven't
+	 * updated the weights yet, so some of the requests counted for the next
+	 * interval will be redirected per current weights. Should be fine, it's
+	 * just an approximate heuristics, and there should be very few requests in
+	 * between. Alternatively, we could reset the request counters when setting
+	 * the new weights, and just ignore the couple requests in between.
+	 *
+	 * XXX Does this need to worry about the completePasses too?
+	 */
+	for (int i = 0; i < StrategyControl->num_partitions; i++)
+	{
+		ClockSweep *sweep = &StrategyControl->sweeps[i];
+
+		/* no need for a spinlock */
+		allocs[i] = pg_atomic_exchange_u32(&sweep->numRequestedAllocs, 0);
+
+		/* add the allocs to running total */
+		pg_atomic_fetch_add_u64(&sweep->numTotalRequestedAllocs, allocs[i]);
+
+		total_allocs += allocs[i];
+	}
+
+	/* Calculate the "fair share" of allocations per partition. */
+	avg_allocs = (total_allocs / StrategyControl->num_partitions);
+
+	/*
+	 * Calculate the "delta" from balanced state for each partition, i.e. how
+	 * many more/fewer allocations it handled relative to the average.
+	 */
+	for (int i = 0; i < StrategyControl->num_partitions; i++)
+	{
+		if (allocs[i] > avg_allocs)
+			delta_allocs += (allocs[i] - avg_allocs);
+	}
+
+	/*
+	 * Skip rebalancing when there's not enough activity, and just keep the
+	 * current weights.
+	 *
+	 * XXX The threshold of 100 allocation is pretty arbitrary.
+	 *
+	 * XXX Maybe a better strategy would be to slowly return to the default
+	 * weights, with each partition allocation only from itself?
+	 *
+	 * XXX Maybe we shouldn't even reset the counters in this case? But it
+	 * should not matter, if the activity is low.
+	 */
+	if (avg_allocs < 100)
+	{
+		elog(DEBUG1, "rebalance skipped: not enough allocations (allocs: %u)",
+			 avg_allocs);
+		return;
+	}
+
+	/*
+	 * Likewise, skip rebalancing if the misbalance is not significant. We
+	 * consider it acceptable if the amount of allocations we'd need to
+	 * redistribute is less than 10% of the average.
+	 *
+	 * XXX Again, these threshold are rather arbitrary. And maybe we should
+	 * do the rabalancing in this case anyway, it's likely cheap and on a big
+	 * system 10% can be quite a lot.
+	 */
+	if (delta_allocs < (avg_allocs * 0.1))
+	{
+		elog(DEBUG1, "rebalance skipped: delta within limit (delta: %u, threshold: %u)",
+			 delta_allocs, (uint32) (avg_allocs * 0.1));
+		return;
+	}
+
+	/*
+	 * The actual rebalancing
+	 *
+	 * Partition with fewer than average allocations, should not redirect any
+	 * allocations to other partitions. So just use weights with a single
+	 * non-zero weight for the partition itself.
+	 *
+	 * Partition with more than average allocations, should not receive any
+	 * redirected allocations, and instead it should redirect excess allocations
+	 * to other partitions.
+	 *
+	 * The redistribution is "proportional" - if the excess allocations of a
+	 * partition represent 10% of the "delta", then each partition that
+	 * needs more allocations will get 10% of the gap from it.
+	 *
+	 * XXX We should add hysteresis, so that it does not oscillate or something
+	 * like that. Maybe CLOCKSWEEP_HISTORY_COEFF already does that?
+	 *
+	 * XXX Ideally, the alternative partitions to use first would be the other
+	 * partitions for the same node (if any).
+	 */
+	for (int i = 0; i < StrategyControl->num_partitions; i++)
+	{
+		ClockSweep *sweep = &StrategyControl->sweeps[i];
+		uint8		balance[MAX_BUFFER_PARTITIONS];
+
+		/* lock, we're going to modify the balance weights */
+		SpinLockAcquire(&sweep->clock_sweep_lock);
+
+		/* reset the weights to start from scratch */
+		memset(balance, 0, sizeof(uint8) * MAX_BUFFER_PARTITIONS);
+
+		/* does this partition has fewer or more than avg_allocs? */
+		if (allocs[i] < avg_allocs)
+		{
+			/* fewer - don't redirect any allocations elsewhere */
+			balance[i] = 100;
+		}
+		else
+		{
+			/*
+			 * more - redistribute the excess allocations
+			 *
+			 * Each "target" partition (with less than avg_allocs) should get
+			 * a fraction proportional to (excess/delta) from this one.
+			 */
+
+			/* fraction of the "total" delta */
+			double	delta_frac = (allocs[i] - avg_allocs) * 1.0 / delta_allocs;
+
+			/* keep just enough allocations to meet the target */
+			balance[i] = (100.0 * avg_allocs / allocs[i]);
+
+			/* redirect the extra allocations */
+			for (int j = 0; j < StrategyControl->num_partitions; j++)
+			{
+				/* How many allocations to receive from i-th partition? */
+				uint32	receive_allocs = delta_frac * (avg_allocs - allocs[j]);
+
+				/* ignore partitions that don't need additional allocations */
+				if (allocs[j] > avg_allocs)
+					continue;
+
+				/* fraction to redirect */
+				balance[j] = (100.0 * receive_allocs / allocs[i]) + 0.5;
+			}
+		}
+
+		/* combine the old and new weights (hysteresis) */
+		for (int j = 0; j < MAX_BUFFER_PARTITIONS; j++)
+		{
+			sweep->balance[j]
+				= CLOCKSWEEP_HISTORY_COEFF * sweep->balance[j] +
+				  (1.0 - CLOCKSWEEP_HISTORY_COEFF) * balance[j];
+		}
+
+		SpinLockRelease(&sweep->clock_sweep_lock);
+	}
+}
+
 /*
  * StrategySyncPrepare -- prepare for sync of all partitions
  *
@@ -657,7 +1039,21 @@ StrategyCtlShmemInit(void *arg)
 		/* Clear statistics */
 		StrategyControl->sweeps[i].completePasses = 0;
 		pg_atomic_init_u32(&StrategyControl->sweeps[i].numBufferAllocs, 0);
+		pg_atomic_init_u32(&StrategyControl->sweeps[i].numRequestedAllocs, 0);
 		pg_atomic_init_u64(&StrategyControl->sweeps[i].numTotalAllocs, 0);
+		pg_atomic_init_u64(&StrategyControl->sweeps[i].numTotalRequestedAllocs, 0);
+
+		/*
+		 * Initialize the weights - start by allocating 100% buffers from
+		 * the current node / partition.
+		 */
+		for (int j = 0; j < MAX_BUFFER_PARTITIONS; j++)
+		{
+			if (i == j)
+				StrategyControl->sweeps[i].balance[i] = 100;
+			else
+				StrategyControl->sweeps[i].balance[j] = 0;
+		}
 	}
 
 	/* No pending notification */
@@ -1025,8 +1421,10 @@ StrategyRejectBuffer(BufferAccessStrategy strategy, BufferDesc *buf, bool from_r
 
 void
 ClockSweepPartitionGetInfo(int idx,
-						   uint32 *complete_passes, uint32 *next_victim_buffer,
-						   uint64 *buffer_total_allocs, uint32 *buffer_allocs)
+						 uint32 *complete_passes, uint32 *next_victim_buffer,
+						 uint64 *buffer_total_allocs, uint32 *buffer_allocs,
+						 uint64 *buffer_total_req_allocs, uint32 *buffer_req_allocs,
+						 int **weights)
 {
 	ClockSweep *sweep = &StrategyControl->sweeps[idx];
 
@@ -1034,11 +1432,21 @@ ClockSweepPartitionGetInfo(int idx,
 
 	/* get the clocksweep stats */
 	*complete_passes = sweep->completePasses;
+
+	/* calculate the actual buffer ID */
 	*next_victim_buffer = pg_atomic_read_u32(&sweep->nextVictimBuffer);
+	*next_victim_buffer = sweep->firstBuffer + (*next_victim_buffer % sweep->numBuffers);
 
-	*buffer_allocs = pg_atomic_read_u32(&sweep->numBufferAllocs);
 	*buffer_total_allocs = pg_atomic_read_u64(&sweep->numTotalAllocs);
+	*buffer_allocs = pg_atomic_read_u32(&sweep->numBufferAllocs);
 
-	/* calculate the actual buffer ID */
-	*next_victim_buffer = sweep->firstBuffer + (*next_victim_buffer % sweep->numBuffers);
+	*buffer_total_req_allocs = pg_atomic_read_u64(&sweep->numTotalRequestedAllocs);
+	*buffer_req_allocs = pg_atomic_read_u32(&sweep->numRequestedAllocs);
+
+	/* return the weights in a newly allocated array */
+	*weights = palloc_array(int, StrategyControl->num_partitions);
+	for (int i = 0; i < StrategyControl->num_partitions; i++)
+	{
+		(*weights)[i] = (int) sweep->balance[i];
+	}
 }
diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h
index 5ab0cee4281..887314d43f4 100644
--- a/src/include/storage/buf_internals.h
+++ b/src/include/storage/buf_internals.h
@@ -593,6 +593,7 @@ extern BufferDesc *StrategyGetBuffer(BufferAccessStrategy strategy,
 extern bool StrategyRejectBuffer(BufferAccessStrategy strategy,
 								 BufferDesc *buf, bool from_ring);
 
+extern void StrategySyncBalance(void);
 extern void StrategySyncPrepare(int *num_parts, uint32 *num_buf_alloc);
 extern int	StrategySyncStart(int partition, uint32 *complete_passes,
 							  int *first_buffer, int *num_buffers);
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index e0bb4cc1df1..02833b19b0c 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -411,11 +411,13 @@ extern int	GetAccessStrategyPinLimit(BufferAccessStrategy strategy);
 
 extern void FreeAccessStrategy(BufferAccessStrategy strategy);
 extern void ClockSweepPartitionGetInfo(int idx,
-									   uint32 *complete_passes,
-									   uint32 *next_victim_buffer,
-									   uint64 *buffer_total_allocs,
-									   uint32 *buffer_allocs);
-
+									 uint32 *complete_passes,
+									 uint32 *next_victim_buffer,
+									 uint64 *buffer_total_allocs,
+									 uint32 *buffer_allocs,
+									 uint64 *buffer_total_req_allocs,
+									 uint32 *buffer_req_allocs,
+									 int **weights);
 
 /* inline functions */
 
-- 
2.54.0

From 5ab7aba5f8af84b748180eb6caa2b165aceb2590 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 2 Jun 2026 22:16:55 +0200
Subject: [PATCH v20260605 4/6] clock-sweep: basic partitioning

Partitions the "clock-sweep" algorithm to work on individual partitions,
one by one. Each backend process is mapped to one "home" partition, with
an independent clock hand. This reduces contention for workloads with
significant buffer pressure.

The patch extends the "pg_buffercache_partitions" view to include
information about the clock-sweep activity.

Note: This needs some sort of "balancing" when one of the partitions is
much busier than the rest (e.g. because there's a single backend consuming
a lot of buffers from it).

Note: There's a problem with some tests running out of unpinned buffers,
due to (intentionally) setting shared buffers very low. That happens
because StrategyGetBuffer() only searches a single partition, and it
has a couple more issues.
---
 .../pg_buffercache--1.7--1.8.sql              |   8 +-
 contrib/pg_buffercache/pg_buffercache_pages.c |  32 +-
 src/backend/storage/buffer/bufmgr.c           | 202 +++++++----
 src/backend/storage/buffer/freelist.c         | 333 ++++++++++++++++--
 src/include/storage/buf_internals.h           |   4 +-
 src/include/storage/bufmgr.h                  |   5 +
 src/test/recovery/t/027_stream_regress.pl     |   5 +
 src/tools/pgindent/typedefs.list              |   1 +
 8 files changed, 486 insertions(+), 104 deletions(-)

diff --git a/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql b/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
index a6e49fd1652..92176fed7f8 100644
--- a/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
+++ b/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
@@ -14,7 +14,13 @@ CREATE VIEW pg_buffercache_partitions AS
 	 numa_node integer,			-- NUMA node of the partitioon
 	 num_buffers integer,		-- number of buffers in the partition
 	 first_buffer integer,		-- first buffer of partition
-	 last_buffer integer);		-- last buffer of partition
+	 last_buffer integer,		-- last buffer of partition
+
+	 -- clocksweep counters
+	 num_passes bigint,			-- clocksweep passes
+	 next_buffer integer,		-- next victim buffer for clocksweep
+	 total_allocs bigint,		-- handled allocs (running total)
+	 num_allocs bigint);		-- handled allocs (current cycle)
 
 -- Don't want these to be available to public.
 REVOKE ALL ON FUNCTION pg_buffercache_partitions() FROM PUBLIC;
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index e3efeeda675..739c63b0cfc 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -31,7 +31,7 @@
 #define NUM_BUFFERCACHE_MARK_DIRTY_ALL_ELEM 3
 
 #define NUM_BUFFERCACHE_OS_PAGES_ELEM	3
-#define NUM_BUFFERCACHE_PARTITIONS_ELEM	5
+#define NUM_BUFFERCACHE_PARTITIONS_ELEM	9
 
 PG_MODULE_MAGIC_EXT(
 					.name = "pg_buffercache",
@@ -912,6 +912,14 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 						   INT4OID, -1, 0);
 		TupleDescInitEntry(tupledesc, (AttrNumber) 5, "last_buffer",
 						   INT4OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 6, "num_passes",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 7, "next_buffer",
+						   INT4OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 8, "total_allocs",
+						   INT8OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 9, "num_allocs",
+						   INT8OID, -1, 0);
 
 		funcctx->user_fctx = BlessTupleDesc(tupledesc);
 
@@ -933,12 +941,22 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 					first_buffer,
 					last_buffer;
 
+		uint64		buffer_total_allocs;
+
+		uint32		complete_passes,
+					next_victim_buffer,
+					buffer_allocs;
+
 		Datum		values[NUM_BUFFERCACHE_PARTITIONS_ELEM];
 		bool		nulls[NUM_BUFFERCACHE_PARTITIONS_ELEM];
 
 		BufferPartitionGet(i, &numa_node, &num_buffers,
 						   &first_buffer, &last_buffer);
 
+		ClockSweepPartitionGetInfo(i,
+								   &complete_passes, &next_victim_buffer,
+								   &buffer_total_allocs, &buffer_allocs);
+
 		values[0] = Int32GetDatum(i);
 		nulls[0] = false;
 
@@ -954,6 +972,18 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		values[4] = Int32GetDatum(last_buffer);
 		nulls[4] = false;
 
+		values[5] = Int64GetDatum(complete_passes);
+		nulls[5] = false;
+
+		values[6] = Int32GetDatum(next_victim_buffer);
+		nulls[6] = false;
+
+		values[7] = Int64GetDatum(buffer_total_allocs);
+		nulls[7] = false;
+
+		values[8] = Int64GetDatum(buffer_allocs);
+		nulls[8] = false;
+
 		/* Build and return the tuple. */
 		tuple = heap_form_tuple((TupleDesc) funcctx->user_fctx, values, nulls);
 		result = HeapTupleGetDatum(tuple);
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index cc398db124d..02c75f82e5b 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3826,33 +3826,34 @@ BufferSync(int flags)
 }
 
 /*
- * BgBufferSync -- Write out some dirty buffers in the pool.
- *
- * This is called periodically by the background writer process.
+ * Information saved between calls so we can determine the strategy point's
+ * advance rate and avoid scanning already-cleaned buffers.
  *
- * Returns true if it's appropriate for the bgwriter process to go into
- * low-power hibernation mode.  (This happens if the strategy clock-sweep
- * has been "lapped" and no buffer allocations have occurred recently,
- * or if the bgwriter has been effectively disabled by setting
- * bgwriter_lru_maxpages to 0.)
+ * XXX Does it actually make sense to split all of this information per
+ * partition? For example, does per-partition advance rate mean anything?
+ * Maybe we should have a global advance rate? Although, if we want to
+ * keep enough clean buffers in each partition, maybe having per-partition
+ * rates makes sense.
  */
-bool
-BgBufferSync(WritebackContext *wb_context)
+typedef struct BufferSyncPartition
+{
+	int	prev_strategy_buf_id;
+	uint32 prev_strategy_passes;
+	int	next_to_clean;
+	uint32 next_passes;
+} BufferSyncPartition;
+
+static BufferSyncPartition *saved_info = NULL;
+static bool saved_info_valid = false;
+
+static bool
+BgBufferSyncPartition(WritebackContext *wb_context, int num_partitions,
+					  int partition, int recent_alloc_partition,
+					  BufferSyncPartition *saved)
 {
 	/* info obtained from freelist.c */
 	int			strategy_buf_id;
 	uint32		strategy_passes;
-	uint32		recent_alloc;
-
-	/*
-	 * Information saved between calls so we can determine the strategy
-	 * point's advance rate and avoid scanning already-cleaned buffers.
-	 */
-	static bool saved_info_valid = false;
-	static int	prev_strategy_buf_id;
-	static uint32 prev_strategy_passes;
-	static int	next_to_clean;
-	static uint32 next_passes;
 
 	/* Moving averages of allocation rate and clean-buffer density */
 	static float smoothed_alloc = 0;
@@ -3880,25 +3881,16 @@ BgBufferSync(WritebackContext *wb_context)
 	long		new_strategy_delta;
 	uint32		new_recent_alloc;
 
+	/* buffer range for the clocksweep partition */
+	int			first_buffer;
+	int			num_buffers;
+
 	/*
 	 * Find out where the clock-sweep currently is, and how many buffer
 	 * allocations have happened since our last call.
 	 */
-	strategy_buf_id = StrategySyncStart(&strategy_passes, &recent_alloc);
-
-	/* Report buffer alloc counts to pgstat */
-	PendingBgWriterStats.buf_alloc += recent_alloc;
-
-	/*
-	 * If we're not running the LRU scan, just stop after doing the stats
-	 * stuff.  We mark the saved state invalid so that we can recover sanely
-	 * if LRU scan is turned back on later.
-	 */
-	if (bgwriter_lru_maxpages <= 0)
-	{
-		saved_info_valid = false;
-		return true;
-	}
+	strategy_buf_id = StrategySyncStart(partition, &strategy_passes,
+										&first_buffer, &num_buffers);
 
 	/*
 	 * Compute strategy_delta = how many buffers have been scanned by the
@@ -3910,17 +3902,17 @@ BgBufferSync(WritebackContext *wb_context)
 	 */
 	if (saved_info_valid)
 	{
-		int32		passes_delta = strategy_passes - prev_strategy_passes;
+		int32		passes_delta = strategy_passes - saved->prev_strategy_passes;
 
-		strategy_delta = strategy_buf_id - prev_strategy_buf_id;
-		strategy_delta += (long) passes_delta * NBuffers;
+		strategy_delta = strategy_buf_id - saved->prev_strategy_buf_id;
+		strategy_delta += (long) passes_delta * num_buffers;
 
 		Assert(strategy_delta >= 0);
 
-		if ((int32) (next_passes - strategy_passes) > 0)
+		if ((int32) (saved->next_passes - strategy_passes) > 0)
 		{
 			/* we're one pass ahead of the strategy point */
-			bufs_to_lap = strategy_buf_id - next_to_clean;
+			bufs_to_lap = strategy_buf_id - saved->next_to_clean;
 #ifdef BGW_DEBUG
 			elog(DEBUG2, "bgwriter ahead: bgw %u-%u strategy %u-%u delta=%ld lap=%d",
 				 next_passes, next_to_clean,
@@ -3928,11 +3920,11 @@ BgBufferSync(WritebackContext *wb_context)
 				 strategy_delta, bufs_to_lap);
 #endif
 		}
-		else if (next_passes == strategy_passes &&
-				 next_to_clean >= strategy_buf_id)
+		else if (saved->next_passes == strategy_passes &&
+				 saved->next_to_clean >= strategy_buf_id)
 		{
 			/* on same pass, but ahead or at least not behind */
-			bufs_to_lap = NBuffers - (next_to_clean - strategy_buf_id);
+			bufs_to_lap = num_buffers - (saved->next_to_clean - strategy_buf_id);
 #ifdef BGW_DEBUG
 			elog(DEBUG2, "bgwriter ahead: bgw %u-%u strategy %u-%u delta=%ld lap=%d",
 				 next_passes, next_to_clean,
@@ -3952,9 +3944,9 @@ BgBufferSync(WritebackContext *wb_context)
 				 strategy_passes, strategy_buf_id,
 				 strategy_delta);
 #endif
-			next_to_clean = strategy_buf_id;
-			next_passes = strategy_passes;
-			bufs_to_lap = NBuffers;
+			saved->next_to_clean = strategy_buf_id;
+			saved->next_passes = strategy_passes;
+			bufs_to_lap = num_buffers;
 		}
 	}
 	else
@@ -3968,15 +3960,16 @@ BgBufferSync(WritebackContext *wb_context)
 			 strategy_passes, strategy_buf_id);
 #endif
 		strategy_delta = 0;
-		next_to_clean = strategy_buf_id;
-		next_passes = strategy_passes;
-		bufs_to_lap = NBuffers;
+		saved->next_to_clean = strategy_buf_id;
+		saved->next_passes = strategy_passes;
+		bufs_to_lap = num_buffers;
 	}
 
 	/* Update saved info for next time */
-	prev_strategy_buf_id = strategy_buf_id;
-	prev_strategy_passes = strategy_passes;
-	saved_info_valid = true;
+	saved->prev_strategy_buf_id = strategy_buf_id;
+	saved->prev_strategy_passes = strategy_passes;
+	/* XXX this needs to happen only after all partitions */
+	/* saved_info_valid = true; */
 
 	/*
 	 * Compute how many buffers had to be scanned for each new allocation, ie,
@@ -3984,9 +3977,9 @@ BgBufferSync(WritebackContext *wb_context)
 	 *
 	 * If the strategy point didn't move, we don't update the density estimate
 	 */
-	if (strategy_delta > 0 && recent_alloc > 0)
+	if (strategy_delta > 0 && recent_alloc_partition > 0)
 	{
-		scans_per_alloc = (float) strategy_delta / (float) recent_alloc;
+		scans_per_alloc = (float) strategy_delta / (float) recent_alloc_partition;
 		smoothed_density += (scans_per_alloc - smoothed_density) /
 			smoothing_samples;
 	}
@@ -3996,7 +3989,7 @@ BgBufferSync(WritebackContext *wb_context)
 	 * strategy point and where we've scanned ahead to, based on the smoothed
 	 * density estimate.
 	 */
-	bufs_ahead = NBuffers - bufs_to_lap;
+	bufs_ahead = num_buffers - bufs_to_lap;
 	reusable_buffers_est = (float) bufs_ahead / smoothed_density;
 
 	/*
@@ -4004,10 +3997,10 @@ BgBufferSync(WritebackContext *wb_context)
 	 * a true average we want a fast-attack, slow-decline behavior: we
 	 * immediately follow any increase.
 	 */
-	if (smoothed_alloc <= (float) recent_alloc)
-		smoothed_alloc = recent_alloc;
+	if (smoothed_alloc <= (float) recent_alloc_partition)
+		smoothed_alloc = recent_alloc_partition;
 	else
-		smoothed_alloc += ((float) recent_alloc - smoothed_alloc) /
+		smoothed_alloc += ((float) recent_alloc_partition - smoothed_alloc) /
 			smoothing_samples;
 
 	/* Scale the estimate by a GUC to allow more aggressive tuning. */
@@ -4034,7 +4027,7 @@ BgBufferSync(WritebackContext *wb_context)
 	 * the BGW will be called during the scan_whole_pool time; slice the
 	 * buffer pool into that many sections.
 	 */
-	min_scan_buffers = (int) (NBuffers / (scan_whole_pool_milliseconds / BgWriterDelay));
+	min_scan_buffers = (int) (num_buffers / (scan_whole_pool_milliseconds / BgWriterDelay));
 
 	if (upcoming_alloc_est < (min_scan_buffers + reusable_buffers_est))
 	{
@@ -4059,20 +4052,20 @@ BgBufferSync(WritebackContext *wb_context)
 	/* Execute the LRU scan */
 	while (num_to_scan > 0 && reusable_buffers < upcoming_alloc_est)
 	{
-		int			sync_state = SyncOneBuffer(next_to_clean, true,
+		int			sync_state = SyncOneBuffer(saved->next_to_clean, true,
 											   wb_context);
 
-		if (++next_to_clean >= NBuffers)
+		if (++saved->next_to_clean >= (first_buffer + num_buffers))
 		{
-			next_to_clean = 0;
-			next_passes++;
+			saved->next_to_clean = first_buffer;
+			saved->next_passes++;
 		}
 		num_to_scan--;
 
 		if (sync_state & BUF_WRITTEN)
 		{
 			reusable_buffers++;
-			if (++num_written >= bgwriter_lru_maxpages)
+			if (++num_written >= (bgwriter_lru_maxpages / num_partitions))
 			{
 				PendingBgWriterStats.maxwritten_clean++;
 				break;
@@ -4086,7 +4079,7 @@ BgBufferSync(WritebackContext *wb_context)
 
 #ifdef BGW_DEBUG
 	elog(DEBUG1, "bgwriter: recent_alloc=%u smoothed=%.2f delta=%ld ahead=%d density=%.2f reusable_est=%d upcoming_est=%d scanned=%d wrote=%d reusable=%d",
-		 recent_alloc, smoothed_alloc, strategy_delta, bufs_ahead,
+		 recent_alloc_partition, smoothed_alloc, strategy_delta, bufs_ahead,
 		 smoothed_density, reusable_buffers_est, upcoming_alloc_est,
 		 bufs_to_lap - num_to_scan,
 		 num_written,
@@ -4116,8 +4109,83 @@ BgBufferSync(WritebackContext *wb_context)
 #endif
 	}
 
+	/* can this partition hibernate */
+	return (bufs_to_lap == 0 && recent_alloc_partition == 0);
+}
+
+/*
+ * BgBufferSync -- Write out some dirty buffers in the pool.
+ *
+ * This is called periodically by the background writer process.
+ *
+ * Returns true if it's appropriate for the bgwriter process to go into
+ * low-power hibernation mode.  (This happens if the strategy clock-sweep
+ * has been "lapped" and no buffer allocations have occurred recently,
+ * or if the bgwriter has been effectively disabled by setting
+ * bgwriter_lru_maxpages to 0.)
+ */
+bool
+BgBufferSync(WritebackContext *wb_context)
+{
+	/* info obtained from freelist.c */
+	uint32		recent_alloc;
+	uint32		recent_alloc_partition;
+	int			num_partitions;
+
+	/* assume we can hibernate, any partition can set to false */
+	bool		hibernate = true;
+
+	/* get the number of clocksweep partitions, and total alloc count */
+	StrategySyncPrepare(&num_partitions, &recent_alloc);
+
+	/* allocate space for per-partition information between calls */
+	if (saved_info == NULL)
+	{
+		/*
+		 * XXX Not great it's using malloc(), but how else to allocate a
+		 * variable-length array?
+		 */
+		saved_info = malloc(sizeof(BufferSyncPartition) * num_partitions);
+	}
+
+	/* Report buffer alloc counts to pgstat */
+	PendingBgWriterStats.buf_alloc += recent_alloc;
+
+	/* average alloc buffers per partition */
+	recent_alloc_partition = (recent_alloc / num_partitions);
+
+	/*
+	 * If we're not running the LRU scan, just stop after doing the stats
+	 * stuff.  We mark the saved state invalid so that we can recover sanely
+	 * if LRU scan is turned back on later.
+	 */
+	if (bgwriter_lru_maxpages <= 0)
+	{
+		saved_info_valid = false;
+		return true;
+	}
+
+	/*
+	 * now process the clocksweep partitions, one by one, using the same
+	 * cleanup that we used for all buffers
+	 *
+	 * XXX Maybe we should randomize the order of partitions a bit, so that we
+	 * don't start from partition 0 all the time? Perhaps not entirely, but at
+	 * least pick a random starting point?
+	 */
+	for (int partition = 0; partition < num_partitions; partition++)
+	{
+		/* hibernate if all partitions can hibernate */
+		hibernate &= BgBufferSyncPartition(wb_context, num_partitions,
+										   partition, recent_alloc_partition,
+										   &saved_info[partition]);
+	}
+
+	/* now that we've scanned all partitions, mark the cached info as valid */
+	saved_info_valid = true;
+
 	/* Return true if OK to hibernate */
-	return (bufs_to_lap == 0 && recent_alloc == 0);
+	return hibernate;
 }
 
 /*
diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index 53ef5239e8d..2d56579682e 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -36,17 +36,28 @@

 /*
- * The shared freelist control information.
+ * Information about one partition of the ClockSweep (on a subset of buffers).
+ *
+ * XXX Should be careful to align this to cachelines, etc.
  */
 typedef struct
 {
 	/* Spinlock: protects the values below */
-	slock_t		buffer_strategy_lock;
+	slock_t		clock_sweep_lock;
+
+	/* range for this clock sweep partition */
+	int32		node;
+	int32		firstBuffer;
+	int32		numBuffers;
 
 	/*
 	 * clock-sweep hand: index of next buffer to consider grabbing. Note that
 	 * this isn't a concrete buffer - we only ever increase the value. So, to
 	 * get an actual buffer, it needs to be used modulo NBuffers.
+	 *
+	 * XXX This is relative to firstBuffer, so needs to be offset properly.
+	 *
+	 * XXX firstBuffer + (nextVictimBuffer % numBuffers)
 	 */
 	pg_atomic_uint32 nextVictimBuffer;
 
@@ -57,11 +68,32 @@ typedef struct
 	uint32		completePasses; /* Complete cycles of the clock-sweep */
 	pg_atomic_uint32 numBufferAllocs;	/* Buffers allocated since last reset */
 
+	/* running total of allocs */
+	pg_atomic_uint64 numTotalAllocs;
+
+} ClockSweep;
+
+/*
+ * The shared freelist control information.
+ */
+typedef struct
+{
+	/* Spinlock: protects the values below */
+	slock_t		buffer_strategy_lock;
+
 	/*
 	 * Bgworker process to be notified upon activity or -1 if none. See
 	 * StrategyNotifyBgWriter.
 	 */
 	int			bgwprocno;
+
+	/* cached info about freelist partitioning */
+	int			num_nodes;
+	int			num_partitions;
+	int			num_partitions_per_node;
+
+	/* clocksweep partitions */
+	ClockSweep	sweeps[FLEXIBLE_ARRAY_MEMBER];
 } BufferStrategyControl;
 
 /* Pointers to shared state */
@@ -108,6 +140,7 @@ static BufferDesc *GetBufferFromRing(BufferAccessStrategy strategy,
 									 uint64 *buf_state);
 static void AddBufferToRing(BufferAccessStrategy strategy,
 							BufferDesc *buf);
+static ClockSweep *ChooseClockSweep(void);
 
 /*
  * ClockSweepTick - Helper routine for StrategyGetBuffer()
@@ -119,6 +152,7 @@ static inline uint32
 ClockSweepTick(void)
 {
 	uint32		victim;
+	ClockSweep *sweep = ChooseClockSweep();
 
 	/*
 	 * Atomically move hand ahead one buffer - if there's several processes
@@ -126,14 +160,14 @@ ClockSweepTick(void)
 	 * apparent order.
 	 */
 	victim =
-		pg_atomic_fetch_add_u32(&StrategyControl->nextVictimBuffer, 1);
+		pg_atomic_fetch_add_u32(&sweep->nextVictimBuffer, 1);
 
-	if (victim >= NBuffers)
+	if (victim >= sweep->numBuffers)
 	{
 		uint32		originalVictim = victim;
 
 		/* always wrap what we look up in BufferDescriptors */
-		victim = victim % NBuffers;
+		victim = victim % sweep->numBuffers;
 
 		/*
 		 * If we're the one that just caused a wraparound, force
@@ -159,19 +193,118 @@ ClockSweepTick(void)
 				 * could lead to an overflow of nextVictimBuffers, but that's
 				 * highly unlikely and wouldn't be particularly harmful.
 				 */
-				SpinLockAcquire(&StrategyControl->buffer_strategy_lock);
+				SpinLockAcquire(&sweep->clock_sweep_lock);
 
-				wrapped = expected % NBuffers;
+				wrapped = expected % sweep->numBuffers;
 
-				success = pg_atomic_compare_exchange_u32(&StrategyControl->nextVictimBuffer,
+				success = pg_atomic_compare_exchange_u32(&sweep->nextVictimBuffer,
 														 &expected, wrapped);
 				if (success)
-					StrategyControl->completePasses++;
-				SpinLockRelease(&StrategyControl->buffer_strategy_lock);
+					sweep->completePasses++;
+				SpinLockRelease(&sweep->clock_sweep_lock);
 			}
 		}
 	}
-	return victim;
+
+	/*
+	 * Make sure we've calculated a buffer in the range of the partition. Buffer
+	 * IDs are 1-based, we're calculating 0-based indexes.
+	 */
+	Assert((victim >= 0) && (victim < sweep->numBuffers));
+	Assert(BufferIsValid(1 + sweep->firstBuffer + victim));
+
+	return sweep->firstBuffer + victim;
+}
+
+/*
+ * ClockSweepPartitionIndex
+ *		pick the clock-sweep partition to use based on PID and NUMA node
+ *
+ * With libnuma, use the NUMA node and PID to pick the partition. Otherwise
+ * use just PID (as if there's a single NUMA node).
+ *
+ * XXX This should also check if buffers are NUMA-partitioned, not just if
+ * compiled with libnuma.
+ */
+static int
+ClockSweepPartitionIndex(void)
+{
+	int		node = 0,
+			index;
+	pid_t	pid = MyProcPid;;
+
+	Assert(StrategyControl->num_partitions ==
+		   (StrategyControl->num_nodes * StrategyControl->num_partitions_per_node));
+
+	/*
+	 * If buffers are NUMA-partitioned, determine the partition using the NUMA
+	 * node and PID. Without NUMA assume everything is a single NUMA node 0, and
+	 * we pick the partition based on PID.
+	 */
+#ifdef USE_LIBNUMA
+	if (shared_buffers_numa)
+	{
+		int		cpu;
+
+		/* XXX do we need to check sched_getcpu is available, somehow? */
+		if ((cpu = sched_getcpu()) < 0)
+			elog(ERROR, "sched_getcpu failed: %m");
+
+		node = numa_node_of_cpu(cpu);
+	}
+#endif
+
+	/*
+	 * We should't get unexpected NUMA nodes, not considered when setting up the
+	 * buffer partitions. It could happen if the allowed NUMA nodes get adjusted
+	 * at runtime, but at this point we just create partitions for all existing
+	 * nodes. We could plan for allowed partitions, but then what if those get
+	 * disabled, and the user allows some other partitions?
+	 */
+	if ((node < 0) || (node > StrategyControl->num_nodes))
+		elog(ERROR, "node out of range: %d > %u", node, StrategyControl->num_nodes);
+
+	/*
+	 * Calculate the partition index. Nodes have the same number of partitions,
+	 * and we use the PID to pick one of those (for a given node). If there's
+	 * only a single partition per node, we can ignore PID and use node directly.
+	 */
+	if (StrategyControl->num_partitions_per_node == 1)
+	{
+		/* fast-path */
+		index = node;
+	}
+	else
+	{
+		/* use PID to pick one of node's partitions */
+		index = (node * StrategyControl->num_partitions_per_node)
+			+ (pid % StrategyControl->num_partitions_per_node);
+	}
+
+	/* should have a valid partition index */
+	Assert((index >= 0) && (index < StrategyControl->num_partitions));
+
+	return index;
+}
+
+/*
+ * ChooseClockSweep
+ *		pick a clocksweep partition based on NUMA node and PID
+ *
+ * Pick a partition mapped to the NUMA node the backend is currently running
+ * on, and use PID if there are multiple partitions per node. Without NUMA
+ * supported/enabled, use just PID.
+ *
+ * XXX Maybe we should do both the total and "per group" counts a power of
+ * two? That'd allow using shifts instead of divisions in the calculation,
+ * and that's cheaper. But how would that deal with odd number of nodes?
+ */
+static ClockSweep *
+ChooseClockSweep(void)
+{
+	int			index = ClockSweepPartitionIndex();
+
+	return &StrategyControl->sweeps[index];
 }
 
 /*
@@ -242,10 +375,37 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint64 *buf_state, bool *from_r
 	 * We count buffer allocation requests so that the bgwriter can estimate
 	 * the rate of buffer consumption.  Note that buffers recycled by a
 	 * strategy object are intentionally not counted here.
+	 *
+	 * XXX It's not quite right we call ChooseClockSweep twice - now, and then
+	 * a couple lines later (through ClockSweepTick). If the process moves
+	 * between CPUs / NUMA nodes in between, these call may pick different
+	 * partitions, confusing the logic a bit.
 	 */
-	pg_atomic_fetch_add_u32(&StrategyControl->numBufferAllocs, 1);
+	pg_atomic_fetch_add_u32(&ChooseClockSweep()->numBufferAllocs, 1);
 
-	/* Use the "clock sweep" algorithm to find a free buffer */
+	/*
+	 * Use the "clock sweep" algorithm to find a free buffer
+	 *
+	 * XXX Note that ClockSweepTick() is NUMA-aware, i.e. it only looks at
+	 * buffers from a single partition, aligned with the NUMA node. That means
+	 * a process "sweeps" only a fraction of buffers, even if the other buffers
+	 * are better candidates for eviction. Maybe there should be some logic to
+	 * "steal" buffers from other partitions or other nodes?
+	 *
+	 * XXX This only searches a single partition, which can result in "no
+	 * unpinned buffers available" even if there are buffers in other
+	 * partitions. Needs to scan partitions if needed, as a fallback.
+	 *
+	 * XXX Would that also mean we should have multiple bgwriters, one for each
+	 * node, or would one bgwriter still handle all nodes?
+	 *
+	 * XXX Also, the trycounter should not be set to NBuffers, but to buffer
+	 * count for that one partition. In fact, this should not call ClockSweepTick
+	 * for every iteration. The call is likely quite expensive (does a lot
+	 * of stuff), and also may return a different partition on each call.
+	 * We should just do it once, and then do the for(;;) loop. And then
+	 * maybe advance to the next partition, until we scan through all of them.
+	 */
 	trycounter = NBuffers;
 	for (;;)
 	{
@@ -325,6 +485,48 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint64 *buf_state, bool *from_r
 	}
 }
 
+/*
+ * StrategySyncPrepare -- prepare for sync of all partitions
+ *
+ * Determine the number of clocksweep partitions, and calculate the recent
+ * buffers allocs (as a sum of all the partitions). This allows BgBufferSync
+ * to calculate average number of allocations per partition for the next
+ * sync cycle.
+ *
+ * In addition it returns the count of recent buffer allocs, which is a total
+ * summed from all partitions. The alloc counts are reset after being read,
+ * as the partitions are walked.
+ */
+void
+StrategySyncPrepare(int *num_parts, uint32 *num_buf_alloc)
+{
+	*num_buf_alloc = 0;
+	*num_parts = StrategyControl->num_partitions;
+
+	/*
+	 * We lock the partitions one by one, so not exacly in sync, but that
+	 * should be fine. We're only looking for heuristics anyway.
+	 */
+	for (int i = 0; i < StrategyControl->num_partitions; i++)
+	{
+		ClockSweep *sweep = &StrategyControl->sweeps[i];
+
+		/* XXX Do we need the lock, if we're only accessing atomics? Surely not. */
+		/* XXX Are we ever calling this without num_buf_alloc? */
+		SpinLockAcquire(&sweep->clock_sweep_lock);
+		if (num_buf_alloc)
+		{
+			uint32	allocs = pg_atomic_exchange_u32(&sweep->numBufferAllocs, 0);
+
+			/* include the count in the running total */
+			pg_atomic_fetch_add_u64(&sweep->numTotalAllocs, allocs);
+
+			*num_buf_alloc += allocs;
+		}
+		SpinLockRelease(&sweep->clock_sweep_lock);
+	}
+}
+
 /*
  * StrategySyncStart -- tell BgBufferSync where to start syncing
  *
@@ -332,37 +534,44 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint64 *buf_state, bool *from_r
  * BgBufferSync() will proceed circularly around the buffer array from there.
  *
  * In addition, we return the completed-pass count (which is effectively
- * the higher-order bits of nextVictimBuffer) and the count of recent buffer
- * allocs if non-NULL pointers are passed.  The alloc count is reset after
- * being read.
+ * the higher-order bits of nextVictimBuffer).
+ *
+ * This only considers a single clocksweep partition, as BgBufferSync looks
+ * at them one by one.
  */
 int
-StrategySyncStart(uint32 *complete_passes, uint32 *num_buf_alloc)
+StrategySyncStart(int partition, uint32 *complete_passes,
+				  int *first_buffer, int *num_buffers)
 {
 	uint32		nextVictimBuffer;
 	int			result;
+	ClockSweep *sweep = &StrategyControl->sweeps[partition];
 
-	SpinLockAcquire(&StrategyControl->buffer_strategy_lock);
-	nextVictimBuffer = pg_atomic_read_u32(&StrategyControl->nextVictimBuffer);
-	result = nextVictimBuffer % NBuffers;
+	Assert((partition >= 0) && (partition < StrategyControl->num_partitions));
+
+	SpinLockAcquire(&sweep->clock_sweep_lock);
+	nextVictimBuffer = pg_atomic_read_u32(&sweep->nextVictimBuffer);
+	result = nextVictimBuffer % sweep->numBuffers;
+
+	*first_buffer = sweep->firstBuffer;
+	*num_buffers = sweep->numBuffers;
 
 	if (complete_passes)
 	{
-		*complete_passes = StrategyControl->completePasses;
+		*complete_passes = sweep->completePasses;
 
 		/*
 		 * Additionally add the number of wraparounds that happened before
 		 * completePasses could be incremented. C.f. ClockSweepTick().
 		 */
-		*complete_passes += nextVictimBuffer / NBuffers;
+		*complete_passes += nextVictimBuffer / sweep->numBuffers;
 	}
+	SpinLockRelease(&sweep->clock_sweep_lock);
 
-	if (num_buf_alloc)
-	{
-		*num_buf_alloc = pg_atomic_exchange_u32(&StrategyControl->numBufferAllocs, 0);
-	}
-	SpinLockRelease(&StrategyControl->buffer_strategy_lock);
-	return result;
+	/* XXX buffer IDs start at 1, we're calculating 0-based indexes */
+	Assert(BufferIsValid(1 + sweep->firstBuffer + result));
+
+	return sweep->firstBuffer + result;
 }
 
 /*
@@ -394,8 +603,14 @@ StrategyNotifyBgWriter(int bgwprocno)
 static void
 StrategyCtlShmemRequest(void *arg)
 {
+	int			num_partitions;
+
+	/* get the number of buffer partitions */
+	BufferPartitionsCalculate(NULL, &num_partitions, NULL);
+
 	ShmemRequestStruct(.name = "Buffer Strategy Status",
-					   .size = sizeof(BufferStrategyControl),
+					   .size = offsetof(BufferStrategyControl, sweeps) +
+							mul_size(num_partitions, sizeof(ClockSweep)),
 					   .ptr = (void **) &StrategyControl
 		);
 }
@@ -408,12 +623,42 @@ StrategyCtlShmemInit(void *arg)
 {
 	SpinLockInit(&StrategyControl->buffer_strategy_lock);
 
-	/* Initialize the clock-sweep pointer */
-	pg_atomic_init_u32(&StrategyControl->nextVictimBuffer, 0);
+	/* Remember the number of partitions */
+	BufferPartitionsParams(&StrategyControl->num_nodes,
+						   &StrategyControl->num_partitions,
+						   &StrategyControl->num_partitions_per_node);
+
+	/* Initialize the clock sweep pointers (for all partitions) */
+	for (int i = 0; i < StrategyControl->num_partitions; i++)
+	{
+		int			node,
+					num_buffers,
+					first_buffer,
+					last_buffer;
+
+		SpinLockInit(&StrategyControl->sweeps[i].clock_sweep_lock);
 
-	/* Clear statistics */
-	StrategyControl->completePasses = 0;
-	pg_atomic_init_u32(&StrategyControl->numBufferAllocs, 0);
+		pg_atomic_init_u32(&StrategyControl->sweeps[i].nextVictimBuffer, 0);
+
+		/* get info about the buffer partition */
+		BufferPartitionGet(i, &node, &num_buffers,
+						   &first_buffer, &last_buffer);
+
+		/*
+		 * FIXME This may not quite right, because if NBuffers is not a
+		 * perfect multiple of numBuffers, the last partition will have
+		 * numBuffers set too high. buf_init handles this by tracking the
+		 * remaining number of buffers, and not overflowing.
+		 */
+		StrategyControl->sweeps[i].node = node;
+		StrategyControl->sweeps[i].numBuffers = num_buffers;
+		StrategyControl->sweeps[i].firstBuffer = first_buffer;
+
+		/* Clear statistics */
+		StrategyControl->sweeps[i].completePasses = 0;
+		pg_atomic_init_u32(&StrategyControl->sweeps[i].numBufferAllocs, 0);
+		pg_atomic_init_u64(&StrategyControl->sweeps[i].numTotalAllocs, 0);
+	}
 
 	/* No pending notification */
 	StrategyControl->bgwprocno = -1;
@@ -777,3 +1022,23 @@ StrategyRejectBuffer(BufferAccessStrategy strategy, BufferDesc *buf, bool from_r
 
 	return true;
 }
+
+void
+ClockSweepPartitionGetInfo(int idx,
+						   uint32 *complete_passes, uint32 *next_victim_buffer,
+						   uint64 *buffer_total_allocs, uint32 *buffer_allocs)
+{
+	ClockSweep *sweep = &StrategyControl->sweeps[idx];
+
+	Assert((idx >= 0) && (idx < StrategyControl->num_partitions));
+
+	/* get the clocksweep stats */
+	*complete_passes = sweep->completePasses;
+	*next_victim_buffer = pg_atomic_read_u32(&sweep->nextVictimBuffer);
+
+	*buffer_allocs = pg_atomic_read_u32(&sweep->numBufferAllocs);
+	*buffer_total_allocs = pg_atomic_read_u64(&sweep->numTotalAllocs);
+
+	/* calculate the actual buffer ID */
+	*next_victim_buffer = sweep->firstBuffer + (*next_victim_buffer % sweep->numBuffers);
+}
diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h
index e944cee2e91..5ab0cee4281 100644
--- a/src/include/storage/buf_internals.h
+++ b/src/include/storage/buf_internals.h
@@ -593,7 +593,9 @@ extern BufferDesc *StrategyGetBuffer(BufferAccessStrategy strategy,
 extern bool StrategyRejectBuffer(BufferAccessStrategy strategy,
 								 BufferDesc *buf, bool from_ring);
 
-extern int	StrategySyncStart(uint32 *complete_passes, uint32 *num_buf_alloc);
+extern void StrategySyncPrepare(int *num_parts, uint32 *num_buf_alloc);
+extern int	StrategySyncStart(int partition, uint32 *complete_passes,
+							  int *first_buffer, int *num_buffers);
 extern void StrategyNotifyBgWriter(int bgwprocno);
 
 /* buf_table.c */
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index 1cf09e8fb7c..e0bb4cc1df1 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -410,6 +410,11 @@ extern int	GetAccessStrategyBufferCount(BufferAccessStrategy strategy);
 extern int	GetAccessStrategyPinLimit(BufferAccessStrategy strategy);
 
 extern void FreeAccessStrategy(BufferAccessStrategy strategy);
+extern void ClockSweepPartitionGetInfo(int idx,
+									   uint32 *complete_passes,
+									   uint32 *next_victim_buffer,
+									   uint64 *buffer_total_allocs,
+									   uint32 *buffer_allocs);

 /* inline functions */
diff --git a/src/test/recovery/t/027_stream_regress.pl b/src/test/recovery/t/027_stream_regress.pl
index ae977297849..f68e08e5697 100644
--- a/src/test/recovery/t/027_stream_regress.pl
+++ b/src/test/recovery/t/027_stream_regress.pl
@@ -18,6 +18,11 @@ $node_primary->adjust_conf('postgresql.conf', 'max_connections', '25');
 $node_primary->append_conf('postgresql.conf',
 	'max_prepared_transactions = 10');
 
+# The default is 1MB, which is not enough with clock-sweep partitioning.
+# Increase to 32MB, so that we don't get "no unpinned buffers".
+$node_primary->append_conf('postgresql.conf',
+	'shared_buffers = 32MB');
+
 # Enable pg_stat_statements to force tests to do query jumbling.
 # pg_stat_statements.max should be large enough to hold all the entries
 # of the regression database.
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index ea756015249..183570b4d43 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -451,6 +451,7 @@ ClientCertName
 ClientConnectionInfo
 ClientData
 ClientSocket
+ClockSweep
 ClonePtrType
 ClosePortalStmt
 ClosePtrType
-- 
2.54.0

From 4672b0b53f68177ca76b9400e7f9ca4172a08ddb Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 2 Jun 2026 23:27:13 +0200
Subject: [PATCH v20260605 3/6] NUMA: shared buffers partitioning

Ensure shared buffers are allocated from all NUMA nodes, in a balanced
way, instead of just using the node where Postgres initially starts, or
where the kernel decides to migrate the page, etc.

In cases like pre-warming a database from a single worker (e.g. using
pg_prewarm), we may end up with severely unbalanced memory distribution
(with most memory located on a single NUMA node). Unbalanced allocation
may put a lot of pressure on the memory system on a small number of NUMA
nodes, limiting the bandwidth etc.

With zone_reclaim, the kernel would eventually move some of the memory
to other nodes, but that tends to take a long time and is unpredictable.

This change forces even distribution of shared buffers on all NUMA
nodes, improving predictability, reducing the time needed for warmup
during benchmarking, etc. It's also less dependent on what the CPU
scheduler decides to do (which cores get used for the warmup.)

The effect is similar to

     numactl --interleave=all

in that the buffers are distributed on the NUMA nodes evenly, but
there's also a number of important differences.

Firstly, it's applied only to shared buffers (and buffer descriptors),
not to the whole shared memory segment. It's possible to enable memory
interleaving using the shmem_interleave GUC, introduced in an earlier
patch in this series.

NUMA works at the granularity of a memory page, which is typically
either 4K or 2MB (hugepage), but other sizes are possible. For systems
where NUMA matters, we expect large amounts of memory (hundreds of
gigabytes) and hugepages enabled. But not necessarily.

The partitioning scheme is best-effort with respect to memory page size.
The shared buffers do not "align" with memory pages (i.e. a partition
may not end at the memory page boundary), in which case we simply locate
just the section of the partition with complete memory pages. This means
there may be ~one unmapped memory page between partitions. Considering
the expected amounts of memory, this is negligible, and the alternative
would be a significant amount of complexity to align the pages and
enforce "allowed" partition sizes.

Buffer descriptors are affected by this too, and the effect may be more
significant, simply because the descriptors are much smaller (~64B). So
the array is smaller, and a single 2MB memory page is worth ~32K buffer
descriptors. But with large systems it's still negligible.

The "buffer partitions" may not be 1:1 with NUMA nodes. We want to allow
clock-sweep partitioning even on non-NUMA systems, or when running only
on a small number of NUMA nodes. There's a minimal number of partitions
(default: 4), and a node may get multiple partitions. Nodes always get
the same number of partitions (e.g. with 3 NUMA nodes there will be 6
partitions in total, as each node gets 2 partitions).

The feature is enabled by dshared_buffers_numa GUC (default: false).
---
 .../pg_buffercache--1.7--1.8.sql              |   1 +
 contrib/pg_buffercache/pg_buffercache_pages.c |  24 +-
 src/backend/storage/buffer/buf_init.c         | 244 ++++++++++++++++--
 src/backend/storage/buffer/freelist.c         |   9 +
 src/backend/utils/misc/guc_parameters.dat     |   6 +
 src/backend/utils/misc/postgresql.conf.sample |   1 +
 src/include/port/pg_numa.h                    |   7 +
 src/include/storage/buf_internals.h           |  16 +-
 src/include/storage/bufmgr.h                  |   8 +
 src/port/pg_numa.c                            | 112 ++++++++
 10 files changed, 392 insertions(+), 36 deletions(-)

diff --git a/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql b/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
index d62b8339bfc..a6e49fd1652 100644
--- a/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
+++ b/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
@@ -11,6 +11,7 @@ LANGUAGE C PARALLEL SAFE;
 CREATE VIEW pg_buffercache_partitions AS
 	SELECT P.* FROM pg_buffercache_partitions() AS P
 	(partition integer,			-- partition index
+	 numa_node integer,			-- NUMA node of the partitioon
 	 num_buffers integer,		-- number of buffers in the partition
 	 first_buffer integer,		-- first buffer of partition
 	 last_buffer integer);		-- last buffer of partition
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index d678cb045ba..e3efeeda675 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -31,7 +31,7 @@
 #define NUM_BUFFERCACHE_MARK_DIRTY_ALL_ELEM 3
 
 #define NUM_BUFFERCACHE_OS_PAGES_ELEM	3
-#define NUM_BUFFERCACHE_PARTITIONS_ELEM	4
+#define NUM_BUFFERCACHE_PARTITIONS_ELEM	5
 
 PG_MODULE_MAGIC_EXT(
 					.name = "pg_buffercache",
@@ -904,11 +904,13 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
 		TupleDescInitEntry(tupledesc, (AttrNumber) 1, "partition",
 						   INT4OID, -1, 0);
-		TupleDescInitEntry(tupledesc, (AttrNumber) 2, "num_buffers",
+		TupleDescInitEntry(tupledesc, (AttrNumber) 2, "numa_node",
 						   INT4OID, -1, 0);
-		TupleDescInitEntry(tupledesc, (AttrNumber) 3, "first_buffer",
+		TupleDescInitEntry(tupledesc, (AttrNumber) 3, "num_buffers",
 						   INT4OID, -1, 0);
-		TupleDescInitEntry(tupledesc, (AttrNumber) 4, "last_buffer",
+		TupleDescInitEntry(tupledesc, (AttrNumber) 4, "first_buffer",
+						   INT4OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 5, "last_buffer",
 						   INT4OID, -1, 0);
 
 		funcctx->user_fctx = BlessTupleDesc(tupledesc);
@@ -926,28 +928,32 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 	{
 		uint32		i = funcctx->call_cntr;
 
-		int			num_buffers,
+		int			numa_node,
+					num_buffers,
 					first_buffer,
 					last_buffer;
 
 		Datum		values[NUM_BUFFERCACHE_PARTITIONS_ELEM];
 		bool		nulls[NUM_BUFFERCACHE_PARTITIONS_ELEM];
 
-		BufferPartitionGet(i, &num_buffers,
+		BufferPartitionGet(i, &numa_node, &num_buffers,
 						   &first_buffer, &last_buffer);
 
 		values[0] = Int32GetDatum(i);
 		nulls[0] = false;
 
-		values[1] = Int32GetDatum(num_buffers);
+		values[1] = Int32GetDatum(numa_node);
 		nulls[1] = false;
 
-		values[2] = Int32GetDatum(first_buffer);
+		values[2] = Int32GetDatum(num_buffers);
 		nulls[2] = false;
 
-		values[3] = Int32GetDatum(last_buffer);
+		values[3] = Int32GetDatum(first_buffer);
 		nulls[3] = false;
 
+		values[4] = Int32GetDatum(last_buffer);
+		nulls[4] = false;
+
 		/* Build and return the tuple. */
 		tuple = heap_form_tuple((TupleDesc) funcctx->user_fctx, values, nulls);
 		result = HeapTupleGetDatum(tuple);
diff --git a/src/backend/storage/buffer/buf_init.c b/src/backend/storage/buffer/buf_init.c
index e593b02e0ca..1f93a31d451 100644
--- a/src/backend/storage/buffer/buf_init.c
+++ b/src/backend/storage/buffer/buf_init.c
@@ -14,12 +14,20 @@
  */
 #include "postgres.h"
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
+#include "port/pg_numa.h"
 #include "storage/aio.h"
 #include "storage/buf_internals.h"
 #include "storage/bufmgr.h"
 #include "storage/proclist.h"
 #include "storage/shmem.h"
 #include "storage/subsystems.h"
+#include "utils/guc_hooks.h"
+#include "utils/varlena.h"
 
 BufferDescPadded *BufferDescriptors;
 char	   *BufferBlocks;
@@ -71,9 +79,12 @@ const ShmemCallbacks BufferManagerShmemCallbacks = {
  *		multiple times. Check the PrivateRefCount infrastructure in bufmgr.c.
  */
 
-/* number of buffer partitions */
-#define NUM_CLOCK_SWEEP_PARTITIONS	4
+/*
+ * Minimum number of buffer partitions, no matter the number of NUMA nodes.
+ */
+#define MIN_BUFFER_PARTITIONS	4
 
+bool	shared_buffers_numa = false;
 
 /*
  * Register shared memory area for the buffer pool.
@@ -81,6 +92,10 @@ const ShmemCallbacks BufferManagerShmemCallbacks = {
 static void
 BufferManagerShmemRequest(void *arg)
 {
+	int		nparts;
+
+	BufferPartitionsCalculate(NULL, &nparts, NULL);
+
 	ShmemRequestStruct(.name = "Buffer Descriptors",
 					   .size = NBuffers * sizeof(BufferDescPadded),
 	/* Align descriptors to a cacheline boundary. */
@@ -103,7 +118,7 @@ BufferManagerShmemRequest(void *arg)
 		);
 
 	ShmemRequestStruct(.name = "Buffer Partition Registry",
-					   .size = NUM_CLOCK_SWEEP_PARTITIONS * sizeof(BufferPartition),
+					   .size = nparts * sizeof(BufferPartition),
 	/* Align descriptors to a cacheline boundary. */
 					   .alignment = PG_CACHE_LINE_SIZE,
 					   .ptr = (void **) &BufferPartitionsRegistry,
@@ -134,6 +149,10 @@ BufferManagerShmemInit(void *arg)
 	/*
 	 * Initialize the buffer partition registry first, before other parts
 	 * have a chance to touch the memory.
+	 *
+	 * Also moves memory to different NUMA nodes (if enabled by a GUC).
+	 * Do this before the loop that initializes buffer headers etc. which
+	 * may fault some of the memory pages etc.
 	 */
 	BufferPartitionsInit();
 
@@ -231,35 +250,203 @@ BufferPartitionsInit(void)
 {
 	int			buffer = 0;
 
-	/* number of buffers per partition (make sure to not overflow) */
-	int			part_buffers = NBuffers / NUM_CLOCK_SWEEP_PARTITIONS;
-	int			remaining_buffers = NBuffers % NUM_CLOCK_SWEEP_PARTITIONS;
+	int			nnodes,
+				npartitions,
+				npartitions_per_node;
 
-	BufferPartitionsRegistry->npartitions = NUM_CLOCK_SWEEP_PARTITIONS;
+	int			buffers_per_partition,
+				buffers_remaining;
 
-	for (int n = 0; n < BufferPartitionsRegistry->npartitions; n++)
-	{
-		BufferPartition *part = &BufferPartitionsRegistry->partitions[n];
+	/* calculate partitioning parameters */
+	BufferPartitionsCalculate(&nnodes, &npartitions, &npartitions_per_node);
+
+	/* paranoia */
+	Assert(nnodes > 0);
+	Assert(npartitions >= MIN_BUFFER_PARTITIONS);
+	Assert((npartitions % nnodes) == 0);
+	Assert((npartitions_per_node * nnodes) == npartitions);
 
-		int			num_buffers = part_buffers;
-		if (n < remaining_buffers)
-			num_buffers += 1;
+	BufferPartitionsRegistry->nnodes = nnodes;
+	BufferPartitionsRegistry->npartitions = npartitions;
+	BufferPartitionsRegistry->npartitions_per_node = npartitions_per_node;
 
-		remaining_buffers -= num_buffers;
+	/* regular partition size, the first couple get an extra buffer */
+	buffers_per_partition = (NBuffers / npartitions);
+	buffers_remaining = (NBuffers % buffers_per_partition);
 
-		Assert((num_buffers > 0) && (num_buffers <= part_buffers));
-		Assert((buffer >= 0) && (buffer < NBuffers));
+	/* should have all the buffers */
+	Assert((buffers_per_partition * npartitions + buffers_remaining) == NBuffers);
 
-		part->num_buffers = num_buffers;
-		part->first_buffer = buffer;
-		part->last_buffer = buffer + (num_buffers - 1);
+	/*
+	 * Now walk the partitions, and set the buffer range. Optionally, place
+	 * the partitions on a given node (for all partitions at once).
+	 */
+	for (int n = 0; n < nnodes; n++)
+	{
+		for (int p = 0; p < npartitions_per_node; p++)
+		{
+			int			idx = (n * npartitions_per_node) + p;
+			BufferPartition *part = &BufferPartitionsRegistry->partitions[idx];
+
+			/*
+			 * Assign to the NUMA node, but only with shared_buffers_numa=on.
+			 *
+			 * XXX we should get an actual node ID from the mask, in case the
+			 * task is restricted to only some nodes.
+			 */
+			part->numa_node = (shared_buffers_numa) ? n : -1;
+
+			/* The first couple partitions may get an extra buffer. */
+			part->num_buffers = buffers_per_partition;
+			if (idx < buffers_remaining)
+				part->num_buffers += 1;
+
+			/* remember the buffer range */
+			part->first_buffer = buffer;
+			part->last_buffer = buffer + (part->num_buffers - 1);
+
+			/* remember start of the next partition */
+			buffer += part->num_buffers;
+		}
 
-		buffer += num_buffers;
+#ifdef USE_LIBNUMA
+		/*
+		 * Now try to locate buffers and buffer descriptors to the node (all
+		 * partitions for the node at once).
+		 */
+		if (shared_buffers_numa)
+		{
+			Size	numa_page_size = pg_numa_page_size();
+
+			int		part_first,
+					part_last,
+					buff_first,
+					buff_last;
+
+			char   *startptr,
+				   *endptr;
+
+			/* first/last partition for this node */
+			part_first = (n * npartitions_per_node);
+			part_last = part_first + (npartitions_per_node - 1);
+
+			/* buffers (blocks) */
+
+			/* first/last buffer */
+			buff_first = BufferPartitionsRegistry->partitions[part_first].first_buffer;
+			buff_last = BufferPartitionsRegistry->partitions[part_last].last_buffer;
+
+			/* beginning of the first block, end of last block */
+			startptr = BufferBlocks + ((Size) buff_first * BLCKSZ);
+			endptr = BufferBlocks + ((Size) (buff_last + 1) * BLCKSZ);
+
+			/* print some warnings when the partitions are not aligned */
+			if ((startptr != (char *) TYPEALIGN(numa_page_size, startptr)) ||
+				(endptr != (char *) TYPEALIGN_DOWN(numa_page_size, endptr)))
+			{
+				elog(WARNING, "buffers for node %d not well aligned [%p,%p] aligned [%p,%p]",
+					 n, startptr, endptr,
+					 (char *) TYPEALIGN(numa_page_size, startptr),
+					 (char *) TYPEALIGN_DOWN(numa_page_size, endptr));
+			}
+
+			/* best effort: align the pointers, so that the mbind() works */
+			startptr = (char *) TYPEALIGN(numa_page_size, startptr);
+			endptr = (char *) TYPEALIGN_DOWN(numa_page_size, endptr);
+
+			/* XXX or should we use pg_numa_move_to_node? */
+			pg_numa_bind_to_node(startptr, endptr, n);
+
+			/* buffer descriptors */
+
+			/* beginning of the first descriptor, end of last descriptor */
+			startptr = (char *) &BufferDescriptors[buff_first];
+			endptr = (char *) &BufferDescriptors[buff_last] + 1;
+
+			/* print some warnings when the partitions are not aligned */
+			if ((startptr != (char *) TYPEALIGN(numa_page_size, startptr)) ||
+				(endptr != (char *) TYPEALIGN_DOWN(numa_page_size, endptr)))
+			{
+				elog(WARNING, "buffers descriptors for node %d not well aligned [%p,%p] aligned [%p,%p]",
+					 n, startptr, endptr,
+					 (char *) TYPEALIGN(numa_page_size, startptr),
+					 (char *) TYPEALIGN_DOWN(numa_page_size, endptr));
+			}
+
+			/* best effort: align the pointers, so that the mbind() works */
+			startptr = (char *) TYPEALIGN(numa_page_size, startptr);
+			endptr = (char *) TYPEALIGN_DOWN(numa_page_size, endptr);
+
+			/* XXX or should we use pg_numa_move_to_node? */
+			pg_numa_bind_to_node(startptr, endptr, n);
+		}
+#endif
 	}
 
 	AssertCheckBufferPartitions();
 }
 
+/*
+ * BufferPartitionsCalculate
+ *		Pick number of buffer partitions for the number of nodes and
+ *		MIN_BUFFER_PARTITIONS.
+ *
+ * Picks the smallest number of partitions higher thah MIN_BUFFER_PARTITIONS,
+ * such that all nodes have the same number of partitions.
+ *
+ * This is best-effort with respect to size of the partitions. It's possible
+ * the partitions are not a perfect multiple of page size, in which case
+ * we set location only for the part where that is possible. The buffers on
+ * the "boundary" may get located up on arbitrary nodes.
+ *
+ * The extra complexity of figuring out the right "partition size" is not
+ * worth it, and it can lead to some partitions being much smaller. This way
+ * we end up with partitions of almost exactly the same size (one BLCKSZ is
+ * the largest difference).
+ *
+ * We expect shared buffers to be much larger than page size (at least on
+ * system where NUMA is a relevant feature), so the number of "not located"
+ * buffers should be a negligible fraction. This only affects pages between
+ * partitions for different nodes, so (nodes-1) pages. This is certainly
+ * fine with 2MB huge pages, but even with 1GB pages it should be OK (as
+ * such systems should have humongous amounts of memory).
+ *
+ * It also means we don't need to worry about memory page size before knowing
+ * if huge pages got used (which we only learn during allocation).
+ */
+void
+BufferPartitionsCalculate(int *num_nodes, int *num_partitions,
+						  int *num_partitions_per_node)
+{
+	int		nnodes,
+			nparts,
+			nparts_per_node;
+
+#if USE_LIBNUMA
+	nnodes = numa_num_configured_nodes();
+	nparts_per_node = 1;	/* at least one partition per node */
+
+	while ((nparts_per_node * nnodes) < MIN_BUFFER_PARTITIONS)
+		nparts_per_node++;
+
+	nparts = (nnodes * nparts_per_node);
+#else
+	/* without NUMA, assume there's just one node */
+	nnodes = 1;
+	nparts = MIN_BUFFER_PARTITIONS;
+	nparts_per_node = MIN_BUFFER_PARTITIONS;
+#endif
+
+	if (num_nodes)
+		*num_nodes = nnodes;
+
+	if (num_partitions)
+		*num_partitions = nparts;
+
+	if (num_partitions_per_node)
+		*num_partitions_per_node = nparts_per_node;
+}
+
 /*
  * BufferPartitionCount
  *		Returns the number of partitions created.
@@ -277,13 +464,14 @@ BufferPartitionCount(void)
  * The returned information is first/last buffer, number of buffers.
  */
 void
-BufferPartitionGet(int idx, int *num_buffers,
+BufferPartitionGet(int idx, int *node, int *num_buffers,
 				   int *first_buffer, int *last_buffer)
 {
 	if ((idx >= 0) && (idx < BufferPartitionsRegistry->npartitions))
 	{
 		BufferPartition *part = &BufferPartitionsRegistry->partitions[idx];
 
+		*node = part->numa_node;
 		*num_buffers = part->num_buffers;
 		*first_buffer = part->first_buffer;
 		*last_buffer = part->last_buffer;
@@ -293,3 +481,17 @@ BufferPartitionGet(int idx, int *num_buffers,
 
 	elog(ERROR, "invalid partition index");
 }
+
+void
+BufferPartitionsParams(int *num_nodes, int *num_partitions,
+					   int *num_partitions_per_node)
+{
+	if (num_nodes)
+		*num_nodes = BufferPartitionsRegistry->nnodes;
+
+	if (num_partitions)
+		*num_partitions = BufferPartitionsRegistry->npartitions;
+
+	if (num_partitions_per_node)
+		*num_partitions_per_node = BufferPartitionsRegistry->npartitions_per_node;
+}
diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index fdb5bad7910..53ef5239e8d 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -15,6 +15,15 @@
  */
 #include "postgres.h"
 
+#ifdef USE_LIBNUMA
+#include <sched.h>
+#endif
+
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
 #include "pgstat.h"
 #include "port/atomics.h"
 #include "storage/buf_internals.h"
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index f15e74198c5..2e71c04282c 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -2724,6 +2724,12 @@
   max => 'INT_MAX / 2',
 },
 
+{ name => 'shared_buffers_numa', type => 'bool', context => 'PGC_POSTMASTER', group => 'RESOURCES_MEM',
+  short_desc => 'Locate partitions of shared buffers (and descriptors) to NUMA nodes.',
+  variable => 'shared_buffers_numa',
+  boot_val => 'false',
+},
+
 { name => 'shared_memory_size', type => 'int', context => 'PGC_INTERNAL', group => 'PRESET_OPTIONS',
   short_desc => 'Shows the size of the server\'s main shared memory area (rounded up to the nearest MB).',
   flags => 'GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE | GUC_UNIT_MB | GUC_RUNTIME_COMPUTED',
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index ac38cddaaf9..c0f79c779cc 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -142,6 +142,7 @@
 #temp_buffers = 8MB                     # min 800kB
 #max_prepared_transactions = 0          # zero disables the feature
                                         # (change requires restart)
+#shared_buffers_numa = off              # NUMA-aware partitioning
 # Caution: it is not advisable to set max_prepared_transactions nonzero unless
 # you actively intend to use prepared transactions.
 #work_mem = 4MB                         # min 64kB
diff --git a/src/include/port/pg_numa.h b/src/include/port/pg_numa.h
index 1b668fe1d91..8fe4d4ab7e3 100644
--- a/src/include/port/pg_numa.h
+++ b/src/include/port/pg_numa.h
@@ -17,6 +17,13 @@
 extern PGDLLIMPORT int pg_numa_init(void);
 extern PGDLLIMPORT int pg_numa_query_pages(int pid, unsigned long count, void **pages, int *status);
 extern PGDLLIMPORT int pg_numa_get_max_node(void);
+extern PGDLLIMPORT Size pg_numa_page_size(void);
+extern PGDLLIMPORT void pg_numa_move_to_node(char *startptr, char *endptr, int node);
+extern PGDLLIMPORT int pg_numa_bind_to_node(char *startptr, char *endptr, int node);
+
+extern PGDLLIMPORT int numa_flags;
+
+#define		NUMA_BUFFERS		0x01
 
 #ifdef USE_LIBNUMA
 
diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h
index e5a887b9969..e944cee2e91 100644
--- a/src/include/storage/buf_internals.h
+++ b/src/include/storage/buf_internals.h
@@ -365,10 +365,10 @@ typedef struct BufferDesc
  * line sized.
  *
  * XXX: As this is primarily matters in highly concurrent workloads which
- * probably all are 64bit these days, and the space wastage would be a bit
- * more noticeable on 32bit systems, we don't force the stride to be cache
- * line sized on those. If somebody does actual performance testing, we can
- * reevaluate.
+ * probably all are 64bit these days. We force the stride to be cache line
+ * sized even on 32bit systems, where the space wastage is be a bit more
+ * noticeable, to allow partitioning of shared buffers (which requires the
+ * memory page be a multiple of buffer descriptor).
  *
  * Note that local buffer descriptors aren't forced to be aligned - as there's
  * no concurrent access to those it's unlikely to be beneficial.
@@ -378,7 +378,7 @@ typedef struct BufferDesc
  * platform with either 32 or 128 byte line sizes, it's good to align to
  * boundaries and avoid false sharing.
  */
-#define BUFFERDESC_PAD_TO_SIZE	(SIZEOF_VOID_P == 8 ? 64 : 1)
+#define BUFFERDESC_PAD_TO_SIZE	64
 
 typedef union BufferDescPadded
 {
@@ -416,8 +416,12 @@ extern PGDLLIMPORT ConditionVariableMinimallyPadded *BufferIOCVArray;
 extern PGDLLIMPORT WritebackContext BackendWritebackContext;
 
 extern int	BufferPartitionCount(void);
-extern void BufferPartitionGet(int idx, int *num_buffers,
+extern void BufferPartitionGet(int idx, int *node, int *num_buffers,
 							   int *first_buffer, int *last_buffer);
+extern void BufferPartitionsCalculate(int *num_nodes, int *num_partitions,
+									  int *num_partitions_per_node);
+extern void BufferPartitionsParams(int *num_nodes, int *num_partitions,
+								   int *num_partitions_per_node);
 
 /* in localbuf.c */
 extern PGDLLIMPORT BufferDesc *LocalBufferDescriptors;
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index 79a3f44747a..1cf09e8fb7c 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -158,10 +158,12 @@ typedef struct ReadBuffersOperation ReadBuffersOperation;
 /*
  * information about one partition of shared buffers
  *
+ * numa_nod specifies node for this partition (-1 means allocated on any node)
  * first/last buffer - the values are inclusive
  */
 typedef struct BufferPartition
 {
+	int			numa_node;		/* NUMA node (-1 no node) */
 	int			num_buffers;	/* number of buffers */
 	int			first_buffer;	/* first buffer of partition */
 	int			last_buffer;	/* last buffer of partition */
@@ -170,7 +172,9 @@ typedef struct BufferPartition
 /* an array of information about all partitions */
 typedef struct BufferPartitions
 {
+	int			nnodes;			/* number of NUMA nodes */
 	int			npartitions;	/* number of partitions */
+	int			npartitions_per_node;	/* for convenience */
 	BufferPartition partitions[FLEXIBLE_ARRAY_MEMBER];
 } BufferPartitions;
 
@@ -206,6 +210,7 @@ extern PGDLLIMPORT const PgAioHandleCallbacks aio_local_buffer_readv_cb;
 
 /* in buf_init.c */
 extern PGDLLIMPORT char *BufferBlocks;
+extern PGDLLIMPORT bool shared_buffers_numa;
 
 /* in localbuf.c */
 extern PGDLLIMPORT int NLocBuffer;
@@ -390,6 +395,9 @@ extern void MarkDirtyAllUnpinnedBuffers(int32 *buffers_dirtied,
 										int32 *buffers_already_dirty,
 										int32 *buffers_skipped);
 
+/* in buf_init.c */
+extern int	BufferGetNode(Buffer buffer);
+
 /* in localbuf.c */
 extern void AtProcExit_LocalBuffers(void);
 
diff --git a/src/port/pg_numa.c b/src/port/pg_numa.c
index 8954669273a..11c8a4503da 100644
--- a/src/port/pg_numa.c
+++ b/src/port/pg_numa.c
@@ -18,6 +18,9 @@
 
 #include "miscadmin.h"
 #include "port/pg_numa.h"
+#include "storage/pg_shmem.h"
+
+int	numa_flags;
 
 /*
  * At this point we provide support only for Linux thanks to libnuma, but in
@@ -118,6 +121,94 @@ pg_numa_get_max_node(void)
 	return numa_max_node();
 }
 
+/*
+ * pg_numa_move_to_node
+ *		move memory to different NUMA nodes in larger chunks
+ *
+ * startptr - start of the region (should be aligned to page size)
+ * endptr - end of the region (doesn't need to be aligned)
+ * node - node to move the memory to
+ *
+ * The "startptr" is expected to be a multiple of system memory page size, as
+ * determined by pg_numa_page_size.
+ *
+ * XXX We only expect to do this during startup, when the shared memory is
+ * still being setup.
+ */
+void
+pg_numa_move_to_node(char *startptr, char *endptr, int node)
+{
+	Size		sz = (endptr - startptr);
+
+	Assert((int64) startptr % pg_numa_page_size() == 0);
+
+	/*
+	 * numa_tonode_memory does not actually cause a page fault, and thus does
+	 * not locate the memory on the node. So it's fast, at least compared to
+	 * pg_numa_query_pages, and does not make startup longer. But it also
+	 * means the expensive part happen later, on the first access.
+	 */
+	numa_tonode_memory(startptr, sz, node);
+}
+
+int
+pg_numa_bind_to_node(char *startptr, char *endptr, int node)
+{
+	int				ret;
+	struct bitmask *nodemask;
+
+	if (node < 0)
+	{
+		errno = EINVAL;
+		return -1;
+	}
+
+	nodemask = numa_allocate_nodemask();
+	if (nodemask == NULL)
+	{
+		errno = ENOMEM;
+		return -1;
+	}
+
+	numa_bitmask_setbit(nodemask, node);
+
+	/*
+	 * MPOL_BIND places the pages strictly on the node, and MPOL_MF_MOVE migrates
+	 * pages already faulted in to that node. If mbind() fails, leave the default
+	 * placement in effect, and report the failure.
+	 */
+	ret = mbind(startptr, (endptr - startptr),
+				MPOL_BIND, nodemask->maskp, node, MPOL_MF_MOVE);
+
+	numa_free_nodemask(nodemask);
+
+	return ret;
+}
+
+Size
+pg_numa_page_size(void)
+{
+	Size		os_page_size;
+	Size		huge_page_size;
+
+#ifdef WIN32
+	SYSTEM_INFO sysinfo;
+
+	GetSystemInfo(&sysinfo);
+	os_page_size = sysinfo.dwPageSize;
+#else
+	os_page_size = sysconf(_SC_PAGESIZE);
+#endif
+
+	/* assume huge pages get used, unless HUGE_PAGES_OFF */
+	if (huge_pages_status != HUGE_PAGES_OFF)
+		GetHugePageSize(&huge_page_size, NULL);
+	else
+		huge_page_size = 0;
+
+	return Max(os_page_size, huge_page_size);
+}
+
 #else
 
 /* Empty wrappers */
@@ -140,4 +231,25 @@ pg_numa_get_max_node(void)
 	return 0;
 }
 
+void
+pg_numa_move_to_node(char *startptr, char *endptr, int node)
+{
+	/* we don't expect to ever get here in builds without libnuma */
+	Assert(false);
+}
+
+int
+pg_numa_bind_to_node(char *startptr, char *endptr, int node)
+{
+	/* we don't expect to ever get here in builds without libnuma */
+	Assert(false);
+}
+
+Size
+pg_numa_page_size(void)
+{
+	/* we don't expect to ever get here in builds without libnuma */
+	Assert(false);
+}
+
 #endif
-- 
2.54.0

From 052ac2256afbba3f25f20f683449a0bbb0af4241 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 2 Jun 2026 22:09:33 +0200
Subject: [PATCH v20260605 2/6] Infrastructure for partitioning of shared
 buffers

The patch introduces a simple "registry" of buffer partitions, keeping
track of the first/last buffer, etc. This serves as a source of truth
for later patches (e.g. to partition clock-sweep or to make the
partitioning NUMA-aware).

The registry is a small array of BufferPartition entries in shared
memory, with partitions sized to be a fair share of shared buffers.

Notes:

* Maybe the number of partitions should be configurable? Right now it's
  hard-coded as 4, but testing shows increasing to e.g. 16) can be
  beneficial.

* This partitioning is independent of the partitions defined in
  lwlock.h, which defines 128 partitions to reduce lock conflict on the
  buffer mapping hashtable. The number of partitions introduced by this
  patch is expected to be much lower (a dozen or so).

* The buffers are divided as evenly as possible, with the first couple
  partitions possibly getting an extra buffer.
---
 contrib/pg_buffercache/Makefile               |   3 +-
 .../pg_buffercache--1.7--1.8.sql              |  23 +++
 contrib/pg_buffercache/pg_buffercache.control |   2 +-
 contrib/pg_buffercache/pg_buffercache_pages.c |  86 +++++++++++
 src/backend/storage/buffer/buf_init.c         | 142 ++++++++++++++++++
 src/include/storage/buf_internals.h           |   5 +
 src/include/storage/bufmgr.h                  |  19 +++
 src/tools/pgindent/typedefs.list              |   2 +
 8 files changed, 280 insertions(+), 2 deletions(-)
 create mode 100644 contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql

diff --git a/contrib/pg_buffercache/Makefile b/contrib/pg_buffercache/Makefile
index 0e618f66aec..7fd5cdfc43d 100644
--- a/contrib/pg_buffercache/Makefile
+++ b/contrib/pg_buffercache/Makefile
@@ -9,7 +9,8 @@ EXTENSION = pg_buffercache
 DATA = pg_buffercache--1.2.sql pg_buffercache--1.2--1.3.sql \
 	pg_buffercache--1.1--1.2.sql pg_buffercache--1.0--1.1.sql \
 	pg_buffercache--1.3--1.4.sql pg_buffercache--1.4--1.5.sql \
-	pg_buffercache--1.5--1.6.sql pg_buffercache--1.6--1.7.sql
+	pg_buffercache--1.5--1.6.sql pg_buffercache--1.6--1.7.sql \
+	pg_buffercache--1.7--1.8.sql
 PGFILEDESC = "pg_buffercache - monitoring of shared buffer cache in real-time"
 
 REGRESS = pg_buffercache pg_buffercache_numa
diff --git a/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql b/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
new file mode 100644
index 00000000000..d62b8339bfc
--- /dev/null
+++ b/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
@@ -0,0 +1,23 @@
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "ALTER EXTENSION pg_buffercache UPDATE TO '1.8'" to load this file. \quit
+
+-- Register the new functions.
+CREATE OR REPLACE FUNCTION pg_buffercache_partitions()
+RETURNS SETOF RECORD
+AS 'MODULE_PATHNAME', 'pg_buffercache_partitions'
+LANGUAGE C PARALLEL SAFE;
+
+-- Create a view for convenient access.
+CREATE VIEW pg_buffercache_partitions AS
+	SELECT P.* FROM pg_buffercache_partitions() AS P
+	(partition integer,			-- partition index
+	 num_buffers integer,		-- number of buffers in the partition
+	 first_buffer integer,		-- first buffer of partition
+	 last_buffer integer);		-- last buffer of partition
+
+-- Don't want these to be available to public.
+REVOKE ALL ON FUNCTION pg_buffercache_partitions() FROM PUBLIC;
+REVOKE ALL ON pg_buffercache_partitions FROM PUBLIC;
+
+GRANT EXECUTE ON FUNCTION pg_buffercache_partitions() TO pg_monitor;
+GRANT SELECT ON pg_buffercache_partitions TO pg_monitor;
diff --git a/contrib/pg_buffercache/pg_buffercache.control b/contrib/pg_buffercache/pg_buffercache.control
index 11499550945..d2fa8ba53ba 100644
--- a/contrib/pg_buffercache/pg_buffercache.control
+++ b/contrib/pg_buffercache/pg_buffercache.control
@@ -1,5 +1,5 @@
 # pg_buffercache extension
 comment = 'examine the shared buffer cache'
-default_version = '1.7'
+default_version = '1.8'
 module_pathname = '$libdir/pg_buffercache'
 relocatable = true
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index bf2e6c97220..d678cb045ba 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -31,6 +31,7 @@
 #define NUM_BUFFERCACHE_MARK_DIRTY_ALL_ELEM 3
 
 #define NUM_BUFFERCACHE_OS_PAGES_ELEM	3
+#define NUM_BUFFERCACHE_PARTITIONS_ELEM	4
 
 PG_MODULE_MAGIC_EXT(
 					.name = "pg_buffercache",
@@ -75,6 +76,7 @@ PG_FUNCTION_INFO_V1(pg_buffercache_evict_all);
 PG_FUNCTION_INFO_V1(pg_buffercache_mark_dirty);
 PG_FUNCTION_INFO_V1(pg_buffercache_mark_dirty_relation);
 PG_FUNCTION_INFO_V1(pg_buffercache_mark_dirty_all);
+PG_FUNCTION_INFO_V1(pg_buffercache_partitions);

 /* Only need to touch memory once per backend process lifetime */
@@ -871,3 +873,87 @@ pg_buffercache_mark_dirty_all(PG_FUNCTION_ARGS)
 
 	PG_RETURN_DATUM(result);
 }
+
+/*
+ * Inquire about partitioning of shared buffers.
+ */
+Datum
+pg_buffercache_partitions(PG_FUNCTION_ARGS)
+{
+	FuncCallContext *funcctx;
+	MemoryContext oldcontext;
+	TupleDesc	tupledesc;
+	TupleDesc	expected_tupledesc;
+	HeapTuple	tuple;
+	Datum		result;
+
+	if (SRF_IS_FIRSTCALL())
+	{
+		funcctx = SRF_FIRSTCALL_INIT();
+
+		/* Switch context when allocating stuff to be used in later calls */
+		oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
+
+		if (get_call_result_type(fcinfo, NULL, &expected_tupledesc) != TYPEFUNC_COMPOSITE)
+			elog(ERROR, "return type must be a row type");
+
+		if (expected_tupledesc->natts != NUM_BUFFERCACHE_PARTITIONS_ELEM)
+			elog(ERROR, "incorrect number of output arguments");
+
+		/* Construct a tuple descriptor for the result rows. */
+		tupledesc = CreateTemplateTupleDesc(expected_tupledesc->natts);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 1, "partition",
+						   INT4OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 2, "num_buffers",
+						   INT4OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 3, "first_buffer",
+						   INT4OID, -1, 0);
+		TupleDescInitEntry(tupledesc, (AttrNumber) 4, "last_buffer",
+						   INT4OID, -1, 0);
+
+		funcctx->user_fctx = BlessTupleDesc(tupledesc);
+
+		/* Return to original context when allocating transient memory */
+		MemoryContextSwitchTo(oldcontext);
+
+		/* Set max calls and remember the user function context. */
+		funcctx->max_calls = BufferPartitionCount();
+	}
+
+	funcctx = SRF_PERCALL_SETUP();
+
+	if (funcctx->call_cntr < funcctx->max_calls)
+	{
+		uint32		i = funcctx->call_cntr;
+
+		int			num_buffers,
+					first_buffer,
+					last_buffer;
+
+		Datum		values[NUM_BUFFERCACHE_PARTITIONS_ELEM];
+		bool		nulls[NUM_BUFFERCACHE_PARTITIONS_ELEM];
+
+		BufferPartitionGet(i, &num_buffers,
+						   &first_buffer, &last_buffer);
+
+		values[0] = Int32GetDatum(i);
+		nulls[0] = false;
+
+		values[1] = Int32GetDatum(num_buffers);
+		nulls[1] = false;
+
+		values[2] = Int32GetDatum(first_buffer);
+		nulls[2] = false;
+
+		values[3] = Int32GetDatum(last_buffer);
+		nulls[3] = false;
+
+		/* Build and return the tuple. */
+		tuple = heap_form_tuple((TupleDesc) funcctx->user_fctx, values, nulls);
+		result = HeapTupleGetDatum(tuple);
+
+		SRF_RETURN_NEXT(funcctx, result);
+	}
+	else
+		SRF_RETURN_DONE(funcctx);
+}
diff --git a/src/backend/storage/buffer/buf_init.c b/src/backend/storage/buffer/buf_init.c
index 1407c930c56..e593b02e0ca 100644
--- a/src/backend/storage/buffer/buf_init.c
+++ b/src/backend/storage/buffer/buf_init.c
@@ -26,10 +26,12 @@ char	   *BufferBlocks;
 ConditionVariableMinimallyPadded *BufferIOCVArray;
 WritebackContext BackendWritebackContext;
 CkptSortItem *CkptBufferIds;
+BufferPartitions *BufferPartitionsRegistry;
 
 static void BufferManagerShmemRequest(void *arg);
 static void BufferManagerShmemInit(void *arg);
 static void BufferManagerShmemAttach(void *arg);
+static void BufferPartitionsInit(void);
 
 const ShmemCallbacks BufferManagerShmemCallbacks = {
 	.request_fn = BufferManagerShmemRequest,
@@ -69,6 +71,9 @@ const ShmemCallbacks BufferManagerShmemCallbacks = {
  *		multiple times. Check the PrivateRefCount infrastructure in bufmgr.c.
  */
 
+/* number of buffer partitions */
+#define NUM_CLOCK_SWEEP_PARTITIONS	4
+
 
 /*
  * Register shared memory area for the buffer pool.
@@ -97,6 +102,13 @@ BufferManagerShmemRequest(void *arg)
 					   .ptr = (void **) &BufferIOCVArray,
 		);
 
+	ShmemRequestStruct(.name = "Buffer Partition Registry",
+					   .size = NUM_CLOCK_SWEEP_PARTITIONS * sizeof(BufferPartition),
+	/* Align descriptors to a cacheline boundary. */
+					   .alignment = PG_CACHE_LINE_SIZE,
+					   .ptr = (void **) &BufferPartitionsRegistry,
+		);
+
 	/*
 	 * The array used to sort to-be-checkpointed buffer ids is located in
 	 * shared memory, to avoid having to allocate significant amounts of
@@ -119,6 +131,12 @@ BufferManagerShmemRequest(void *arg)
 static void
 BufferManagerShmemInit(void *arg)
 {
+	/*
+	 * Initialize the buffer partition registry first, before other parts
+	 * have a chance to touch the memory.
+	 */
+	BufferPartitionsInit();
+
 	/*
 	 * Initialize all the buffer headers.
 	 */
@@ -151,3 +169,127 @@ BufferManagerShmemAttach(void *arg)
 	WritebackContextInit(&BackendWritebackContext,
 						 &backend_flush_after);
 }
+
+/*
+ * Sanity checks of buffers partitions - there must be no gaps, it must cover
+ * the whole range of buffers, etc.
+ */
+static void
+AssertCheckBufferPartitions(void)
+{
+#ifdef USE_ASSERT_CHECKING
+	int			num_buffers = 0;
+
+	Assert(BufferPartitionsRegistry->npartitions > 0);
+
+	for (int i = 0; i < BufferPartitionsRegistry->npartitions; i++)
+	{
+		BufferPartition *part = &BufferPartitionsRegistry->partitions[i];
+
+		/*
+		 * We can get a single-buffer partition, if the sizing forces the last
+		 * partition to be just one buffer. But it's unlikely (and
+		 * undesirable).
+		 */
+		Assert(part->first_buffer <= part->last_buffer);
+		Assert((part->last_buffer - part->first_buffer + 1) == part->num_buffers);
+
+		num_buffers += part->num_buffers;
+
+		/*
+		 * The first partition needs to start on buffer 0. Later partitions
+		 * need to be contiguous, without skipping any buffers.
+		 */
+		if (i == 0)
+		{
+			Assert(part->first_buffer == 0);
+		}
+		else
+		{
+			BufferPartition *prev = &BufferPartitionsRegistry->partitions[i - 1];
+
+			Assert((part->first_buffer - 1) == prev->last_buffer);
+		}
+
+		/* the last partition needs to end on buffer (NBuffers - 1) */
+		if (i == (BufferPartitionsRegistry->npartitions - 1))
+		{
+			Assert(part->last_buffer == (NBuffers - 1));
+		}
+	}
+
+	Assert(num_buffers == NBuffers);
+#endif
+}
+
+/*
+ * BufferPartitionsInit
+ *		Initialize registry of buffer partitions.
+ */
+static void
+BufferPartitionsInit(void)
+{
+	int			buffer = 0;
+
+	/* number of buffers per partition (make sure to not overflow) */
+	int			part_buffers = NBuffers / NUM_CLOCK_SWEEP_PARTITIONS;
+	int			remaining_buffers = NBuffers % NUM_CLOCK_SWEEP_PARTITIONS;
+
+	BufferPartitionsRegistry->npartitions = NUM_CLOCK_SWEEP_PARTITIONS;
+
+	for (int n = 0; n < BufferPartitionsRegistry->npartitions; n++)
+	{
+		BufferPartition *part = &BufferPartitionsRegistry->partitions[n];
+
+		int			num_buffers = part_buffers;
+		if (n < remaining_buffers)
+			num_buffers += 1;
+
+		remaining_buffers -= num_buffers;
+
+		Assert((num_buffers > 0) && (num_buffers <= part_buffers));
+		Assert((buffer >= 0) && (buffer < NBuffers));
+
+		part->num_buffers = num_buffers;
+		part->first_buffer = buffer;
+		part->last_buffer = buffer + (num_buffers - 1);
+
+		buffer += num_buffers;
+	}
+
+	AssertCheckBufferPartitions();
+}
+
+/*
+ * BufferPartitionCount
+ *		Returns the number of partitions created.
+ */
+int
+BufferPartitionCount(void)
+{
+	return BufferPartitionsRegistry->npartitions;
+}
+
+/*
+ * BufferPartitionGet
+ *		Returns information about a partition at the provided index.
+ *
+ * The returned information is first/last buffer, number of buffers.
+ */
+void
+BufferPartitionGet(int idx, int *num_buffers,
+				   int *first_buffer, int *last_buffer)
+{
+	if ((idx >= 0) && (idx < BufferPartitionsRegistry->npartitions))
+	{
+		BufferPartition *part = &BufferPartitionsRegistry->partitions[idx];
+
+		*num_buffers = part->num_buffers;
+		*first_buffer = part->first_buffer;
+		*last_buffer = part->last_buffer;
+
+		return;
+	}
+
+	elog(ERROR, "invalid partition index");
+}
diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h
index 89615a254a3..e5a887b9969 100644
--- a/src/include/storage/buf_internals.h
+++ b/src/include/storage/buf_internals.h
@@ -411,9 +411,14 @@ typedef struct WritebackContext
 
 /* in buf_init.c */
 extern PGDLLIMPORT BufferDescPadded *BufferDescriptors;
+extern PGDLLIMPORT BufferPartitions *BufferPartitionsRegistry;
 extern PGDLLIMPORT ConditionVariableMinimallyPadded *BufferIOCVArray;
 extern PGDLLIMPORT WritebackContext BackendWritebackContext;
 
+extern int	BufferPartitionCount(void);
+extern void BufferPartitionGet(int idx, int *num_buffers,
+							   int *first_buffer, int *last_buffer);
+
 /* in localbuf.c */
 extern PGDLLIMPORT BufferDesc *LocalBufferDescriptors;
 
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index 6837b35fc6d..79a3f44747a 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -155,6 +155,25 @@ struct ReadBuffersOperation
 
 typedef struct ReadBuffersOperation ReadBuffersOperation;
 
+/*
+ * information about one partition of shared buffers
+ *
+ * first/last buffer - the values are inclusive
+ */
+typedef struct BufferPartition
+{
+	int			num_buffers;	/* number of buffers */
+	int			first_buffer;	/* first buffer of partition */
+	int			last_buffer;	/* last buffer of partition */
+} BufferPartition;
+
+/* an array of information about all partitions */
+typedef struct BufferPartitions
+{
+	int			npartitions;	/* number of partitions */
+	BufferPartition partitions[FLEXIBLE_ARRAY_MEMBER];
+} BufferPartitions;
+
 /* to avoid having to expose buf_internals.h here */
 typedef struct WritebackContext WritebackContext;
 
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 8cf40c87043..ea756015249 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -365,6 +365,8 @@ BufferHeapTupleTableSlot
 BufferLockMode
 BufferLookupEnt
 BufferManagerRelation
+BufferPartition
+BufferPartitions
 BufferStrategyControl
 BufferTag
 BufferUsage
-- 
2.54.0

From 238935f06c793dcd0271423af83e039d2ee00120 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 3 Jun 2026 16:30:28 +0200
Subject: [PATCH v20260605 1/6] Add shmem_populate and shmem_interleave GUCs

- shmem_populate - Forces mmap() with MAP_POPULATE, which faults all
  memory pages backing the shared memory segment.

- shmem_interleave - Applies NUMA interleaving on the whole shared
  memory segment, to balance allocations between nodes.
---
 src/backend/port/sysv_shmem.c             | 47 +++++++++++++++++++++++
 src/backend/utils/misc/guc_parameters.dat | 14 +++++++
 src/include/miscadmin.h                   |  4 ++
 3 files changed, 65 insertions(+)

diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 2e3886cf9fe..9eaff838a04 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -27,6 +27,10 @@
 #include <sys/shm.h>
 #include <sys/stat.h>
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#endif
+
 #include "miscadmin.h"
 #include "port/pg_bitutils.h"
 #include "portability/mem.h"
@@ -98,6 +102,10 @@ void	   *UsedShmemSegAddr = NULL;
 static Size AnonymousShmemSize;
 static void *AnonymousShmem = NULL;
 
+/* GUCs */
+bool shmem_populate = false;	/* MAP_POPULATE */
+bool shmem_interleave = false;	/* NUMA interleaving */
+
 static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
 static void IpcMemoryDetach(int status, Datum shmaddr);
 static void IpcMemoryDelete(int status, Datum shmId);
@@ -604,6 +612,21 @@ CreateAnonymousSegment(Size *size)
 	int			mmap_errno = 0;
 	int			mmap_flags = MAP_SHARED | MAP_ANONYMOUS | MAP_HASSEMAPHORE;
 
+	/* If requested, populate the shared memory by MAP_POPULATE. */
+	if (shmem_populate)
+		mmap_flags |= MAP_POPULATE;
+
+#ifdef USE_LIBNUMA
+	/*
+	 * If requested, interleave the shared memory by setting a memory policy
+	 * before the mmap() call. This really matters only with MAP_POPULATE,
+	 * because without page faults the memory does not actually get placed
+	 * to the nodes. But without MAP_POPULATE it's virtually free.
+	 */
+	if (shmem_interleave)
+		numa_set_interleave_mask(numa_all_nodes_ptr);
+#endif
+
 #ifndef MAP_HUGETLB
 	/* PGSharedMemoryCreate should have dealt with this case */
 	Assert(huge_pages != HUGE_PAGES_ON);
@@ -665,6 +688,30 @@ CreateAnonymousSegment(Size *size)
 						 allocsize) : 0));
 	}
 
+#ifdef USE_LIBNUMA
+	/*
+	 * If set the policy to interleaving by numa_set_membind(), undo it now by
+	 * setting the policy to localalloc. With MAP_POPULATE, all the pages were
+	 * faulted and are now interleaved on the available nodes.
+	 *
+	 * To handle the case without MAP_POPULATE, apply the interleaving policy to
+	 * the shared memory segment allocated by mmap() before touching it in any
+	 * way, so that it gets placed on the correct node on first access.
+	 *
+	 * This matters especially with huge pages, where it's possible to run out
+	 * of huge pages on some nodes and then crash. By explicitly interleaving
+	 * the whole segment, that's much less likely.
+	 */
+	if (shmem_interleave)
+	{
+		/* undo the policy set by numa_set_membind() earlier */
+		numa_set_localalloc();
+
+		/* set interleaving policy for not yet faulted memory */
+		numa_interleave_memory(ptr, allocsize, numa_all_nodes_ptr);
+	}
+#endif
+
 	*size = allocsize;
 	return ptr;
 }
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index afaa058b046..f15e74198c5 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -740,6 +740,20 @@
   ifdef => 'DEBUG_NODE_TESTS_ENABLED',
 },
 
+{ name => 'debug_shmem_interleave', type => 'bool', context => 'PGC_SUSET', group => 'DEVELOPER_OPTIONS',
+  short_desc => 'Forces interleaving for the whole shared memory segment.',
+  flags => 'GUC_NOT_IN_SAMPLE',
+  variable => 'shmem_interleave',
+  boot_val => 'false'
+},
+
+{ name => 'debug_shmem_populate', type => 'bool', context => 'PGC_SUSET', group => 'DEVELOPER_OPTIONS',
+  short_desc => 'Populates (faults) the whole shared memory segment using MAP_POPULATE.',
+  flags => 'GUC_NOT_IN_SAMPLE',
+  variable => 'shmem_populate',
+  boot_val => 'false'
+},
+
 { name => 'debug_write_read_parse_plan_trees', type => 'bool', context => 'PGC_SUSET', group => 'DEVELOPER_OPTIONS',
   short_desc => 'Set this to force all parse and plan trees to be passed through outfuncs.c/readfuncs.c, to facilitate catching errors and omissions in those modules.',
   flags => 'GUC_NOT_IN_SAMPLE',
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 7de0a115402..13bb29f8702 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -213,6 +213,10 @@ extern PGDLLIMPORT Oid MyDatabaseTableSpace;
 
 extern PGDLLIMPORT bool MyDatabaseHasLoginEventTriggers;
 
+extern PGDLLIMPORT bool shmem_populate;
+extern PGDLLIMPORT bool shmem_interleave;
+
+
 /*
  * Date/Time Configuration
  *
-- 
2.54.0

---

From: Jakub Wartak <[email-masked]>
Date: Tue, 16 Jun 2026 10:16:00 +0200

> Hi,

Hi Tomas, thanks for working on this.

[..]

1. 005 says:

but there isn't direct any check for checking if pg_numa_get_max_node() ->
numa_max_node() is not getting higher than allowed here. In theory this could
happen I think if ClockSweepPartitionIndex() would return
    numa = numa_node_of_cpu()
on some hypothethical very high-end setup (with plenty of sub-NUMA nodes)
and that would cause accesing .balance[] without bounds.

2. If we have in 004 struct ClockSweep with nextVictimBuffer, shouldn't
this be padded/aligned somehow later in BufferStrategyControl which does
    ClockSweep      sweeps[FLEXIBLE_ARRAY_MEMBER];
to avoid contention/false sharing? (comments says it should be but it
doesn't seem so?), maybe the comment should be TODO for now? I have not
quantified any potential benefit

With pahole after some hassle I've got:
 struct ClockSweep {
        slock_t                    clock_sweep_lock;     /*     0     1 */

        /* XXX 3 bytes hole, try to pack */

        int32                      node;                 /*     4     4 */
        int32                      firstBuffer;          /*     8     4 */
        int32                      numBuffers;           /*    12     4 */
        pg_atomic_uint32           nextVictimBuffer;     /*    16     4 */
        uint32                     completePasses;       /*    20     4 */
        pg_atomic_uint32           numBufferAllocs;      /*    24     4 */
        pg_atomic_uint32           numRequestedAllocs;   /*    28     4 */
        pg_atomic_uint64           numTotalAllocs;       /*    32     8 */
        pg_atomic_uint64           numTotalRequestedAllocs; /*    40     8 */
        uint8                      balance[32];          /*    48    32 */

        /* size: 80, cachelines: 2, members: 11 */
        /* sum members: 77, holes: 1, sum holes: 3 */
        /* last cacheline: 16 bytes */
 };
maybe with smaller MAX_BUFFER_PARTITIONS we could pack this into size=64 ?

3. In 004 sched_getcpu() is used and mentioned how to check if it is available

But my $0.02 (maybe not that important), but I've at least saw once where
(on EC2?) some clock_gettime() was very slow and that was because it was not
available in VDSO. It's usually some mix of kernel <-> arch <-> libc (not
always glibc?) compatibility matrix issue. My worry is that StrategyGetBuffer()
-> ChooseClockSweep() -> ClockSweepPartitionIndex() -> sched_getcpu() would be
available, but slow and it would mean real syscall price (and that's not once
there per buffer). I'm also somehow thinking other platforms (FreeBSD comes to
mind, but I haven't checked further). The point is: wouldn't it be cheaper
that to be refreshed from time to time instead otherwise we risk some slow
code on non-x86_64, but I doubt how proliferated is e.g. ARM64 with NUMA..
Or alternative is to have pg_test_numa proggie and this would be measuring
certain things about NUMA including timing of sched_getcpu (just like
pg_test_timing does for time), at least that could explain why somebody's
system/platform is slow.

4. Patch has problem (without fix for #8) that when number of available huge
pages in the OS is greatly higher than shared_memory_size_in_huge_pages it
will use only first NUMA node. This might be a problem when starting mulitple
DBs (they will occupy first available NUMA):

### with s_b=8GB and nr_hugepages=1500 it's OK

# find /sys/devices/system/node/ -name nr_hugepages -exec grep -H . {}
\; | grep 2048 | sort
/sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages:1250
/sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages:1250
/sys/devices/system/node/node2/hugepages/hugepages-2048kB/nr_hugepages:1250
/sys/devices/system/node/node3/hugepages/hugepages-2048kB/nr_hugepages:1250

## note the correct split below for N0/N1..
# grep huge /proc/`pgrep -f /usr/pgsql19/bin/postgres`/numa_maps
7fb1b4400000 default file=/anon_hugepage\040(deleted) huge dirty=4269
mapmax=6 N0=1250 N1=1250 N2=519 N3=1250 kernelpagesize_kB=2048

### still s_b=8GB but nr_hugepages = 19000 (~37GB), it ends all on N0=4269
# find /sys/devices/system/node/ -name nr_hugepages -exec grep -H . {}
\; | grep 2048 | sort
/sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages:4750
/sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages:4750
/sys/devices/system/node/node2/hugepages/hugepages-2048kB/nr_hugepages:4750
/sys/devices/system/node/node3/hugepages/hugepages-2048kB/nr_hugepages:4750
## all on N0...
# grep huge /proc/`pgrep -f /usr/pgsql19/bin/postgres`/numa_maps
7ff3a7a00000 default file=/anon_hugepage\040(deleted) huge dirty=4269
mapmax=6 N0=4269 kernelpagesize_kB=2048

I was even thinking go to lengths and add code for inspecting that /sys on
some later date that the kernel NUMA hugepages are really distributed
on the nodes as they should be (it's easy to end up on just 1 node out of
many; allocating via sysctl -w <higher> and then <lower> allocation is easy
way to force hugepages just to 1 node instead of many :o). I've hit the
problem multiple times, so we should bail out if we want NUMA and the
Buffer Blocks were just put on 1 node (instead of many).

5. In 005 we could mention more clealry what's the difference between
those 3: numRequestedAllocs, numTotalAllocs, numTotalRequestedAllocs
in the defintion to make it easier to read, maybe copy-cat those earlier
descriptions there too as we already have:

6. While at it, it would be helpful if we could reset the
pg_buffercache_partitions stats in some way (pg_buffercache_partitions
is very usefull).. or is there way to plug into main pg_reset functions?

7. If I add basic error checking for mbind() then it complains a lot, like
below with annotated strace -ffe mbind to show the point:

[pid  2856] mbind(0x7fd8d0e00000, 2145386496, MPOL_BIND, [], 0,
MPOL_MF_MOVE) = -1 EINVAL (Invalid argument)
WARNING:  mbind(): Invalid argument
WARNING:  buffers descriptors for node 0 not well aligned
[0x7fd8cccf5000,0x7fd8cdcf4fc1] aligned
[0x7fd8cce00000,0x7fd8cdc00000]

[pid  2856] mbind(0x7fd8cce00000, 14680064, MPOL_BIND, [], 0,
MPOL_MF_MOVE) = -1 EINVAL (Invalid argument)
WARNING:  mbind(): Invalid argument
WARNING:  buffers for node 1 not well aligned
[0x7fd950cf5000,0x7fd9d0cf5000] aligned
[0x7fd950e00000,0x7fd9d0c00000]

[pid  2856] mbind(0x7fd950e00000, 2145386496, MPOL_BIND,
0x5589057ded00, 1, MPOL_MF_MOVE) = -1 EINVAL (Invalid argument)
WARNING:  mbind(): Invalid argument
WARNING:  buffers descriptors for node 1 not well aligned
[0x7fd8cdcf5000,0x7fd8cecf4fc1] aligned
[0x7fd8cde00000,0x7fd8cec00000]
[..]

but with pg_numa.c fixed like below (node should be size):
        ret = mbind(startptr, (endptr - startptr),
-                               MPOL_BIND, nodemask->maskp, node, MPOL_MF_MOVE);
+                               MPOL_BIND, nodemask->maskp,
nodemask->size, MPOL_MF_MOVE);

it doesn't report errors anymore and suprisngly hugepages in numa_maps are
altered from:
7fb1b4400000 default file=/anon_hugepage\040(deleted) huge dirty=4269
mapmax=6 N0=1250 N1=1250 N2=519 N3=1250 kernelpagesize_kB=2048

to explicit "binds":
7f8540000000 default file=/anon_hugepage\040(deleted) huge dirty=25
mapmax=6 N0=25 kernelpagesize_kB=2048
7f8543200000 bind:0 file=/anon_hugepage\040(deleted) huge dirty=7
mapmax=3 N0=7 kernelpagesize_kB=2048
7f8544000000 default file=/anon_hugepage\040(deleted) huge dirty=1
mapmax=2 N0=1 kernelpagesize_kB=2048
7f8544200000 bind:1 file=/anon_hugepage\040(deleted) huge dirty=7
mapmax=2 N1=7 kernelpagesize_kB=2048
7f8545000000 default file=/anon_hugepage\040(deleted) huge dirty=1
mapmax=2 N0=1 kernelpagesize_kB=2048
7f8545200000 bind:2 file=/anon_hugepage\040(deleted) huge dirty=7
mapmax=2 N2=7 kernelpagesize_kB=2048
7f8546000000 default file=/anon_hugepage\040(deleted) huge dirty=1
mapmax=2 N0=1 kernelpagesize_kB=2048
7f8546200000 bind:3 file=/anon_hugepage\040(deleted) huge dirty=7
mapmax=2 N3=7 kernelpagesize_kB=2048
7f8547000000 default file=/anon_hugepage\040(deleted) huge dirty=1
mapmax=3 N0=1 kernelpagesize_kB=2048
7f8547200000 bind:0 file=/anon_hugepage\040(deleted) huge dirty=1023
mapmax=2 N0=1023 kernelpagesize_kB=2048
7f85c7000000 default file=/anon_hugepage\040(deleted) huge dirty=1
N0=1 kernelpagesize_kB=2048
7f85c7200000 bind:1 file=/anon_hugepage\040(deleted) huge dirty=1023
N1=1023 kernelpagesize_kB=2048
7f8647000000 default file=/anon_hugepage\040(deleted) huge dirty=1
N1=1 kernelpagesize_kB=2048
7f8647200000 bind:2 file=/anon_hugepage\040(deleted) huge dirty=1023
N2=1023 kernelpagesize_kB=2048
7f86c7000000 default file=/anon_hugepage\040(deleted) huge dirty=1
N3=1 kernelpagesize_kB=2048
7f86c7200000 bind:3 file=/anon_hugepage\040(deleted) huge dirty=1023
N3=1023 kernelpagesize_kB=2048
7f8747000000 default file=/anon_hugepage\040(deleted) huge dirty=117
mapmax=6 N2=117 kernelpagesize_kB=2048

so lots of VMAs were created (it could affect performance in some way, I think
for sure it would affect for worse fork() rates by postmaster for new conns).

To me it looks like there's plenty of "N[0..3]=1" with "default" indicating
single HP page being somehow left/missed in address calculations, but I
haven't pressed this harder.

NOTE: the patch works even without this fix, but I believe if got non-0 we
cannot reliably trust the optimizer memory layout has been deployed (I suspect
it's some kind luck it sharded the shm based on number of hugepages available)

> balancing algorithm of some sort).

8. The crux of this email and stuff I wanted to further discuss, when server
is started with on this 4-NUMA box with
* numactl --cpunodebind=0 pg_ctl start # so that all backends fork()ing will be
  on node#0
* the shm split onto 4 nodes properly
* s_b still just 8GB (with ideal split),
* DB size ~15GB with 8 pgbench partitions (and fully in VFS cache)
* pgbench -c 8 -j 8 postgres -T 20 -P 1 -f seqconcurrscans.pgb with:
  \set num (:client_id % 8) + 1
  select sum(octet_length(filler)) from pgbench_accounts_:num;
* mpstat repors correctly just node#0 used

a. with the patch for GUCs with numa on and defaults two clocksweep settings
   on, I'm getting:

   latency average = 3252.254 ms
   latency stddev = 72.011 ms

b. with debug_clocksweep_balance=off, I'm realiably getting

   latency average = 2688.742 ms
   latency stddev = 61.738 ms

so IMHO clocksweep partitioning is cool, but if we are discussing the current
balancing logic leaves some juice on the table from the most optimized variant
(~1.2x) with ~90ns:270ns (local vs remote latency). In the picture above it
was 8 backends accessing 8x 1.6GB tables (lower than NBuffers / 4).

Dunno if it should be optimized further, certainly we'll get reports from
quick benchmarks run by people that PG 20 could be *slower* because.. well,
they got (sub)optimal layout during startup (all HP on 1 node and some
query hitting just that one query with local affinity and this is visible
to naked eye). I was re-reading thread and Andres also wrote "We should use
the partitioned clock sweep to default to using local memory as long as
possible."

So two further ideas:

I. BufferAccessStrategy: we could derrive affinity from the BAS strategy
itself, couldn't we? If we are using capped ring buffer, we could indicate
that we want it just from local node as priority disregarding weights (?).
Same goes for BAS_VACUUM (why would one it on remote NUMA?). With BAS_BULKREAD
there would be some potential issue with sync-scan-table code though.
With BAS_BULKWRITE e.g. CTAS/CREATE INDEX it makes lot of sense too.
prewarm could be hacked to use some new special BAS_DISTRIBUTE or something
for ideal distribution amongst all NUMA nodes.

II. what if we could track if the relation is just all-local-access?

Another idea is that if we would know that's it's just us working on some
relation (created by us; or it's not being touched remotley) then we could
ask for local-only memory affinity. So something like this:

a. in case of locally-only access rels =>
   ask for local memory first
   if that fails failback to weighted RR (so to to weights)
b. in case other rels => weighted RR (so to to weights) directly

The tracking of the fact that Buffer was accessed just locally or remotley
itself is not hard to imagine (by using some free "bits" in BufferDesc.
"state" where refcount/usagecount itself are stored, well at least 4 bits
for my 4 nodes, but there's plenty of left there), I have some PoC for
that but that's just per-Buffer tracking of "was this Buffer accessed by
remote nodes", but I'm completley lost how to make transition to the
is-the-relation-being-accessed-accross-NUMA-nodes info to drive such
optimization (we would need some shared infra just for tracking such info;
assuming up to 2^31 or 2^32 relations [OID?] and just using at least >=
2..4 bits, that's already huge number: we are talking GBs of shm mem).

BTW: I've been experimenting with this patchset and added couple of things
(see attached), and with I'm able to get optimal just by forcing affinity
too using that earlier bench:

  latency average = 2512.929 ms
  latency stddev = 97.525 ms

and that was with pure 100% affinity to my local node:

  select pg_buffercache_set_partition(0, '{100,0,0,0}');
  debug_clocksweep_balance_recalc=off
  debug_clocksweep_balance=on
  debug_clocksweep_scan_all_partitions=on

  (so it's another proof that code is fine, it's just algorithm that would
  have to adjusted)

For benchmarks with pgbench -S for 100% local affinity vs 100% remote
(I can do that with that pg_buffercache_set_partition() of mine), I'm
getting just +/- 1-2k TPS (42-43k TPS vs 41-42k TPS), so not much, __but__
I've spotted some another bug in from where we are fetching memory from
unoptimal places if we are not on node#0, I'll need to dig into that more
though. Another thing is that pgbench -S runs are much less demanding in
terms of memory bandwidth used (under <1GB/s here vs 6-8GB/s for
seqconcurrscans.sql using the same amout of cores).

> node and crashed with a segfault (or was it sigbus?).

It was SIGBUS and with this patchset I think we are fine: I have never
witnessed this one crashing with SIGBUS.

> usual duration), let me know.

See above, but I think we would have to fix at at least: mbind() failure,
and those VMAs disconnected regions.

-J.

---

From: Tomas Vondra <[email-masked]>
Date: Tue, 16 Jun 2026 14:39:45 +0200

> and that would cause accesing .balance[] without bounds.

Yes, this should be capped to the MAX_BUFFER_PARTITIONS.

> maybe with smaller MAX_BUFFER_PARTITIONS we could pack this into size=64 ?

Possibly. Im not entirely happy with making the ClockSweep struct so
much larger, but I haven't found a better way to store the counters
needed for balancing. The only thing I can think of is storing it
outside the struct, and maybe that's the right thing to do.

But that assumes the current balancing approach is the right one.

> pg_test_timing does for time), at least that could explain why somebody's

Yes, I think we may need some sort of caching for this / check only
sometimes. I haven't seen it to matter, but that may be luck and on
other systems / platforms it may be worse.

> Buffer Blocks were just put on 1 node (instead of many).

How come the pg_numa_bind_to_node() calls don't move the parts to the
correct node?

If something is already using huge pages on the other nodes, then sure,
it will fail. But I think that's OK - it's a best-effort thing. Maybe we
should exit instead in this case?

> + * - numRequestedAllocs - number of allocatios requested in a partition

I agree the explanation for this is not entirely clear.

> is very usefull).. or is there way to plug into main pg_reset functions?

Hmm, I was afraid it'd interfere with the balancing. I'm not sure it
makes sense to reset just some of the fields - it'd make it much harder
to interpret the counters. I'll think abou this.

> nodemask->size, MPOL_MF_MOVE);

I think this is a silly bug on my side, clearly the nodemask should have
size > 0 even for node 0.

> for sure it would affect for worse fork() rates by postmaster for new conns).

Isn't this 4 VMAs for buffers ad 4 VMAs for buffer descriptors? Or maybe
it's the parts that could not be mapped due to insufficient alignment?

> it's some kind luck it sharded the shm based on number of hugepages available)

I'm not sure I understand what you mean. What non-0?

> was 8 backends accessing 8x 1.6GB tables (lower than NBuffers / 4).

How does this compare to master, i.e. without these NUMA patches?

> for ideal distribution amongst all NUMA nodes.

Yes, I think it might make sense to disable balancing in these cases.

> seqconcurrscans.sql using the same amout of cores).

No opinion. I need to look at this closer.

> witnessed this one crashing with SIGBUS.

Good. But I wonder if allocating just the precise number of huge pages
(per shared_memory_size_in_huge_pages) can prevent moving the partitions
to the correct node.

> and those VMAs disconnected regions.

Yes, the mbind() failure is a bug.

regards

-- 
Tomas Vondra

---

From: Jakub Wartak <[email-masked]>
Date: Wed, 17 Jun 2026 14:13:03 +0200

[..]
[..]
> But that assumes the current balancing approach is the right one.

Yeah, I'm just not sure if there is some wasted performance due to
false-sharing in very heavy benchmark scenarios (in theory the
nextVictimBuffer could bounce rather heavily).

> other systems / platforms it may be worse.

Okay, so maybe an action point for us much later would be to try this on 2s+
ARM or some other much more rare setup just to see if we need to do it at all
(perhaps annotate it with TODO, I have short memory ;))

> How come the pg_numa_bind_to_node() calls don't move the parts to the

Well, if we were not checking error code mbind() it could behave in
non-deterministic way I think (you ask for MPOL_BIND, maybe it does something,
maybe it doesnt work but we continued anyway).

> should exit instead in this case?

Yes, I think we should exit with FATAL.

> to interpret the counters. I'll think abou this.

Well, small hint: I find the the view very, very usefull in explaining
where/why memory is being allocated from. Maybe if we want to include it
in final version, then it might be worth (later on) to do it, if that
won't be included I'm fine just doing \watch 1 in psql to see what's
happening. Another hint is that maybe it doesnt belong to pg_buffercache
and would make it easier to integrate into pg_stat_reset*() -- dunno, how/
if extension can plug into it.

> it's the parts that could not be mapped due to insufficient alignment?

Yes, like with patchset I'm getting WARNINGs:

WARNING:  buffers for node 0 not well aligned
[0x7efc100f5000,0x7efc900f5000] aligned
[0x7efc10200000,0x7efc90000000]
WARNING:  buffers descriptors for node 0 not well aligned
[0x7efc0c0f4f80,0x7efc0d0f4f41] aligned
[0x7efc0c200000,0x7efc0d000000]
WARNING:  buffers for node 1 not well aligned
[0x7efc900f5000,0x7efd100f5000] aligned
[0x7efc90200000,0x7efd10000000]
WARNING:  buffers descriptors for node 1 not well aligned
[0x7efc0d0f4f80,0x7efc0e0f4f41] aligned
[0x7efc0d200000,0x7efc0e000000]
WARNING:  buffers for node 2 not well aligned
[0x7efd100f5000,0x7efd900f5000] aligned
[0x7efd10200000,0x7efd90000000]
WARNING:  buffers descriptors for node 2 not well aligned
[0x7efc0e0f4f80,0x7efc0f0f4f41] aligned
[0x7efc0e200000,0x7efc0f000000]
WARNING:  buffers for node 3 not well aligned
[0x7efd900f5000,0x7efe100f5000] aligned
[0x7efd90200000,0x7efe10000000]
WARNING:  buffers descriptors for node 3 not well aligned
[0x7efc0f0f4f80,0x7efc100f4f41] aligned
[0x7efc0f200000,0x7efc10000000]
LOG:  starting PostgreSQL 19beta1 on x86_64-linux, compiled by
gcc-12.2.0, 64-bit

relevant numa_maps for postmaster:

7efc0c200000 bind:0 file=/anon_hugepage\040(deleted) huge dirty=7
mapmax=2 N0=7 kernelpagesize_kB=2048
7efc0d000000 default file=/anon_hugepage\040(deleted) huge dirty=1
mapmax=2 N3=1 kernelpagesize_kB=2048
7efc0d200000 bind:1 file=/anon_hugepage\040(deleted) huge dirty=7
mapmax=2 N1=7 kernelpagesize_kB=2048
7efc0e000000 default file=/anon_hugepage\040(deleted) huge dirty=1
mapmax=2 N3=1 kernelpagesize_kB=2048
7efc0e200000 bind:2 file=/anon_hugepage\040(deleted) huge dirty=7
mapmax=2 N2=7 kernelpagesize_kB=2048
7efc0f000000 default file=/anon_hugepage\040(deleted) huge dirty=1
mapmax=2 N3=1 kernelpagesize_kB=2048
7efc0f200000 bind:3 file=/anon_hugepage\040(deleted) huge dirty=7
mapmax=2 N3=7 kernelpagesize_kB=2048
7efc10000000 default file=/anon_hugepage\040(deleted) huge dirty=1
mapmax=3 N3=1 kernelpagesize_kB=2048
7efc10200000 bind:0 file=/anon_hugepage\040(deleted) huge dirty=1023
N0=1023 kernelpagesize_kB=2048
7efc90000000 default file=/anon_hugepage\040(deleted) huge dirty=1
N3=1 kernelpagesize_kB=2048
7efc90200000 bind:1 file=/anon_hugepage\040(deleted) huge dirty=1023
N1=1023 kernelpagesize_kB=2048
7efd10000000 default file=/anon_hugepage\040(deleted) huge dirty=1
N0=1 kernelpagesize_kB=2048
7efd10200000 bind:2 file=/anon_hugepage\040(deleted) huge dirty=1023
N2=1023 kernelpagesize_kB=2048
7efd90000000 default file=/anon_hugepage\040(deleted) huge dirty=1
N2=1 kernelpagesize_kB=2048
7efd90200000 bind:3 file=/anon_hugepage\040(deleted) huge dirty=1023
N3=1023 kernelpagesize_kB=2048
7efe10000000 default file=/anon_hugepage\040(deleted) huge dirty=116
mapmax=6 N1=116 kernelpagesize_kB=2048

so if You take just 7efc10200000..7efc90000000 (from buffers@node0, 1st line)
and zoom in/grep You'll get:

7efc10200000 bind:0 file=/anon_hugepage\040(deleted) huge dirty=1023
N0=1023 kernelpagesize_kB=2048
7efc90000000 default file=/anon_hugepage\040(deleted) huge dirty=1
N3=1 kernelpagesize_kB=2048

so @ 7efc90000000 it's Node3=1 hugepage (so it's wrong by 1x 2048kb offset)

so it contains the tail of node 0 buffers and the head of node - and is
excluded from both mbind() calls. At first I've spotted this in 003/
BufferPartitionsInit() with:
  cstartptr = (char *) &BufferDescriptors[buff_first];
  endptr   = (char *) &BufferDescriptors[buff_last] + 1; // <-- BUG?
with
  endptr   = (char *) &BufferDescriptors[buff_last + 1];

but got the same issue, so I've forced the mbind() to use 2x TYPEALIGN_DOWN
to cover with policy everything and got:

7fc134e00000 default file=/anon_hugepage\040(deleted) huge dirty=24
mapmax=6 N3=24 kernelpagesize_kB=2048
7fc137e00000 bind:0 file=/anon_hugepage\040(deleted) huge dirty=8
mapmax=4 N0=8 kernelpagesize_kB=2048
7fc138e00000 bind:1 file=/anon_hugepage\040(deleted) huge dirty=8
mapmax=2 N1=8 kernelpagesize_kB=2048
7fc139e00000 bind:2 file=/anon_hugepage\040(deleted) huge dirty=8
mapmax=3 N2=8 kernelpagesize_kB=2048
7fc13ae00000 bind:3 file=/anon_hugepage\040(deleted) huge dirty=8
mapmax=2 N3=8 kernelpagesize_kB=2048
7fc13be00000 bind:0 file=/anon_hugepage\040(deleted) huge dirty=1024
N0=1024 kernelpagesize_kB=2048
7fc1bbe00000 bind:1 file=/anon_hugepage\040(deleted) huge dirty=1024
N1=1024 kernelpagesize_kB=2048
7fc23be00000 bind:2 file=/anon_hugepage\040(deleted) huge dirty=1024
mapmax=2 N2=1024 kernelpagesize_kB=2048
7fc2bbe00000 bind:3 file=/anon_hugepage\040(deleted) huge dirty=1024
N3=1024 kernelpagesize_kB=2048
7fc33be00000 default file=/anon_hugepage\040(deleted) huge dirty=116
mapmax=6 N1=116 kernelpagesize_kB=2048

that was with:
@@ -351,7 +351,7 @@ BufferPartitionsInit(void)
                        }

                        /* best effort: align the pointers, so that
the mbind() works */
-                       startptr = (char *) TYPEALIGN(numa_page_size, startptr);
+                       startptr = (char *)
TYPEALIGN_DOWN(numa_page_size, startptr);
                        endptr = (char *)
TYPEALIGN_DOWN(numa_page_size, endptr);

@@ -374,7 +374,7 @@ BufferPartitionsInit(void)
                        }

                        /* best effort: align the pointers, so that
the mbind() works */
-                       startptr = (char *) TYPEALIGN(numa_page_size, startptr);
+                       startptr = (char *)
TYPEALIGN_DOWN(numa_page_size, startptr);
                        endptr = (char *)
TYPEALIGN_DOWN(numa_page_size, endptr);

Looks way better, but it was wild guess (TBH I think I've prefered the
previous patchset version where the input shm addresses were already aligned,
but if You and others say it's easier route then sure)

> I'm not sure I understand what you mean. What non-0?

Non-0 return code from mbind() as the mbind() failed with -1, but it did somehow
alter numa pages probably(?), but without creating specific "isolated" VMA
explicit "bind" policy, probably maybe it did not even work at all..

> How does this compare to master, i.e. without these NUMA patches?

I had some problems comparing, but with "perfect setup" that includes the
following:
- pgbench/clients on node#0
- backends running on node#0
- hugepages memory on node#0..3, but with with this patchset and those goodies:
  debug_clocksweep_balance=on
  debug_clocksweep_balance_recalc=off
  debug_clocksweep_scan_all_partitions=on
  (so technically node0 backends accessing just Buffers / Buffers Desc from
  node0, technically node0 weights: "{100,0,0,0}")
- with today's new discovery for me that Linux's kernel VFS cache is also
  having also first-touch (!) NUMA policy and really important here (so
  VFS cached data also alters results of testing wildly!), so I had to
  force unloading VFS cache and force-loading it into node#0, I've was
  getting for seqconcurrscans:
  latency average = 2701.705 ms
  latency stddev = 111.608 ms

vs master, huh, but which scenario? the default one without any affinity?
- assuming you get the split shm split like "N0=1059 N1=1299 N2=879 N3=1031"
  but that's appeneded-only (so not interleaved (?)) and you even risk
  having shm placed on just __one__ node (if it is big enough and free
  enough)
- if you go straight to benchmarking it (CPU hits random nodes)
  latency average = 3439.200 ms
  latency stddev = 580.501 ms
- with backends forked() to node#0 (numactl --cpunodebind=0 pg_ctl start)
  latency average = 4937.543 ms
  latency stddev = 573.841 ms
  because of random VFS cache placement (I imagine it as flow of on node0 CPUs
    a. nextVictimBuffer contention
    b. getBuffer() - fetch random shm memory from random NUMA node
    c. pread() - fetch from VFS cache but from *remote* NUMA node
  )
- with backends forked() to node#0 and pinned VFS cached fully on node#0 too
  latency average = 4518.651 ms
  latency stddev = 797.369 ms
  (but this is still Buffers from other nodes)
- same as above above and numactl interleaved shm:
  latency average = 3792.016 ms
  latency stddev = 825.186 ms
- same as above and interleaved shm, but without pining to CPUs on specifc
  node and ensure random VFS cache vs nodes:
  latency average = 2913.813 ms
  latency stddev = 352.552 ms
- but the moment you read anything reads base/ into VFS cache to particular
  node (imagine pg_prewarm or even just tar) assuming it was not there it
  also pins that to that node memory and you'll get:
  latency average = 3594.180 ms
  latency stddev = 851.949 ms

> Yes, I think it might make sense to disable balancing in these cases.

OK, I did not code anything of that as

> No opinion. I need to look at this closer.

Great !

> (per shared_memory_size_in_huge_pages) can prevent moving the partitions

Not sure I understand (?) how's that related to SIGBUS?

-J.

---

From: Tomas Vondra <[email-masked]>
Date: Wed, 24 Jun 2026 22:26:29 +0200

Hi,

Here's an updated patch series, with only minor changes to fix the mbind
issues:

1) It uses the correct nodemask size, so that the mbind actually binds
the partition to the right node.

2) It aligns the start/end pointers so that there no pages are left with
the default memory policy. So now there should be only the bind:N
entries, not the single-page "default" ones. This means the last page of
a partition can be mapped to a different node, but that seems fine (in
the end it could have happened with the old approach too).

I've also included Jakub's "goodies" patch with the additional GUCs.
Those seem potentially useful to development.

I have some results from a new round of benchmarks, and it's a bit
disappointing. Or rather, there seem to be some issues that I can't
figure out, causing regressions.

Consider a very simple test, doing a lot of sequential scans to put a
fair amount of pressure on the clocksweep / buffer replacement. There's
a .tgz with the benchmark script attached, but it does about this:

* Initialize a pgbench database with scale 2000 (so ~30GB, about twice
the shared buffers).

* Uses --partitions=100, so that the partitions are small enough not to
trigger the 1/4 threshold (i.e. not use circular buffers).

* Does runs with custom script, forcing sequential scans of the table,
with two queries:

select count(1) from pgbench_accounts;

Those are called "count" and "offset" in the results. The script forces
serial sequential scans (no index scans, no parallelism), and does runs
with 1, 8 and 32 clients (this is an old-ish xeon with 44 physical cores
on two sockets, 2 NUMA nodes).

I did runs with "master" and the all the 7 patches, with the NUMA stuff
enabled/disabled since 0003 (which adds it). See the two PDFs with more
complete results, but here's the "count" query for a subset of the
patches (the omitted ones behave similarly to what's shown here).

This chart is for median latency (in milliseconds):

  clients       master     0003      0004    0003/on    0004/on
  -------------------------------------------------------------
        1        12767    12582     14509      12807      15307
        8        14383    14355     14149      14069      16165
       32        14756    15198     14836      14984      17128
       --------------------------------------------------------
        1                  103%      114%       100%       120%
        8                  101%       98%        98%       112%
       32                  102%      101%       102%       116%

The percentages are compared to "master", the columns with "/on" are
with shared_buffers_numa=on.

Clearly, there's no chance with 0003 (which binds shared buffer
partitions to NUMA nodes, even if that's enabled). The differences are
within noise, pretty much, for all client counts.

Then 0004 gets applied, which partitions the clock sweep. And well, that
doesn't go particularly well. There is a bit of a regression even with
numa=off, but it kinda recovers with the following patches. But with
numa=on, there's a consistent ~10% regression (give or take).

I've spent a fair bit of time investigating what's causing this, but so
far I have nothing. I assume it's something silly in the patches
partitioning the clocksweep, or maybe the approach is flawed in some
way. Not sure :-(

regards

-- 
Tomas Vondra
From e6d192ae573d63b5cb5c71773047b97c97eb7961 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Wed, 3 Jun 2026 16:30:28 +0200
Subject: [PATCH v20260624 1/7] Add shmem_populate and shmem_interleave GUCs

 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#endif
+
 #include "miscadmin.h"
 
+/* GUCs */

+#ifdef USE_LIBNUMA
+	/*
 
index 7170a4bff98..bf38aa6baa2 100644

From 1f3fe6ec274f459ca4de440cf9da4eb43818e48c Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 2 Jun 2026 22:09:33 +0200
Subject: [PATCH v20260624 2/7] Infrastructure for partitioning of shared
 buffers

Notes:

 
index 510455998aa..6c9838b4efc 100644

 PG_MODULE_MAGIC_EXT(
@@ -77,6 +78,7 @@ PG_FUNCTION_INFO_V1(pg_buffercache_evict_all);

 /* Only need to touch memory once per backend process lifetime */
@@ -922,3 +924,87 @@ pg_buffercache_mark_dirty_all(PG_FUNCTION_ARGS)
 
 	PG_RETURN_DATUM(result);
 }
+
+/*

 
 
 /*

 /* in buf_init.c */

 
 
+/*
 
index c5db6ca6705..45fc91fcb97 100644

From 5a8f757fea9a2755d1c4fadad067352f4ab307be Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 2 Jun 2026 23:27:13 +0200
Subject: [PATCH v20260624 3/7] NUMA: shared buffers partitioning

The effect is similar to

     numactl --interleave=all

 src/backend/storage/buffer/buf_init.c         | 251 ++++++++++++++++--
 10 files changed, 399 insertions(+), 36 deletions(-)

index 6c9838b4efc..46b6b85a2e3 100644

 PG_MODULE_MAGIC_EXT(
@@ -955,11 +955,13 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 
 		funcctx->user_fctx = BlessTupleDesc(tupledesc);
@@ -977,28 +979,32 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 	{
 		uint32		i = funcctx->call_cntr;
 
-		int			num_buffers,
+		int			numa_node,
+					num_buffers,
 					first_buffer,
 					last_buffer;

 

 
 
index e593b02e0ca..4ed79db7b49 100644
 
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+

 
 
 /*

@@ -231,35 +250,210 @@ BufferPartitionsInit(void)
 {
 	int			buffer = 0;

 

 

-		buffer += num_buffers;
+#ifdef USE_LIBNUMA
+		/*
+		 * Now try to locate buffers and buffer descriptors to the node (all
+		 * partitions for the node at once).
+		 */
+		if (shared_buffers_numa)
+		{
+			Size	numa_page_size = pg_numa_page_size();
+
+			int		part_first,
+					part_last,
+					buff_first,
+					buff_last;
+
+			char   *startptr,
+				   *endptr;
+
+			/* first/last partition for this node */
+			part_first = (n * npartitions_per_node);
+			part_last = part_first + (npartitions_per_node - 1);
+
+			/* buffers (blocks) */
+
+			/* first/last buffer */
+			buff_first = BufferPartitionsRegistry->partitions[part_first].first_buffer;
+			buff_last = BufferPartitionsRegistry->partitions[part_last].last_buffer;
+
+			/* beginning of the first block, end of last block */
+			startptr = BufferBlocks + ((Size) buff_first * BLCKSZ);
+			endptr = BufferBlocks + ((Size) (buff_last + 1) * BLCKSZ);
+
+			/* print some warnings when the partitions are not aligned */
+			if ((startptr != (char *) TYPEALIGN(numa_page_size, startptr)) ||
+				(endptr != (char *) TYPEALIGN(numa_page_size, endptr)))
+			{
+				elog(WARNING, "buffers for node %d not well aligned [%p,%p]",
+					 n, startptr, endptr);
+			}
+
+			/* best effort: align the pointers, so that the mbind() works */
+			startptr = (char *) TYPEALIGN_DOWN(numa_page_size, startptr);
+
+			/* the last partition aligns to the end of the buffer */
+			if (n == (nnodes - 1))
+				endptr = (char *) TYPEALIGN(numa_page_size, endptr);
+			else
+				endptr = (char *) TYPEALIGN_DOWN(numa_page_size, endptr);
+
+			/* XXX or should we use pg_numa_move_to_node? */
+			if (pg_numa_bind_to_node(startptr, endptr, n) != 0)
+				elog(WARNING, "failed to bind shared buffers partition to node %d", n);
+
+			/* buffer descriptors */
+
+			/* beginning of the first descriptor, end of last descriptor */
+			startptr = (char *) &BufferDescriptors[buff_first];
+			endptr = (char *) &BufferDescriptors[buff_last] + 1;
+
+			/* print some warnings when the partitions are not aligned */
+			if ((startptr != (char *) TYPEALIGN(numa_page_size, startptr)) ||
+				(endptr != (char *) TYPEALIGN(numa_page_size, endptr)))
+			{
+				elog(WARNING, "buffers descriptors for node %d not well aligned [%p,%p]",
+					 n, startptr, endptr);
+			}
+
+			/* best effort: align the pointers, so that the mbind() works */
+			startptr = (char *) TYPEALIGN_DOWN(numa_page_size, startptr);
+
+			if (n == (nnodes - 1))
+				endptr = (char *) TYPEALIGN(numa_page_size, endptr);
+			else
+				endptr = (char *) TYPEALIGN_DOWN(numa_page_size, endptr);
+
+			/* XXX or should we use pg_numa_move_to_node? */
+			if (pg_numa_bind_to_node(startptr, endptr, n) != 0)
+				elog(WARNING, "failed to bind shared buffer descriptors partition to node %d", n);
+		}
+#endif
 	}

+/*
@@ -277,13 +471,14 @@ BufferPartitionCount(void)
 
@@ -293,3 +488,17 @@ BufferPartitionGet(int idx, int *num_buffers,

+#ifdef USE_LIBNUMA
+#include <sched.h>
+#endif
+
+#ifdef USE_LIBNUMA
+#include <numa.h>
+#include <numaif.h>
+#endif
+
 #include "pgstat.h"

 #ifdef USE_LIBNUMA

 
 
 /* in localbuf.c */

 /* in buf_init.c */
 
 /* in localbuf.c */
 
+/* in buf_init.c */
 
diff --git a/src/port/pg_numa.c b/src/port/pg_numa.c
index 8954669273a..66985f32db3 100644
--- a/src/port/pg_numa.c
+++ b/src/port/pg_numa.c
@@ -18,6 +18,9 @@
 
 #include "miscadmin.h"
 
 /*
 
+/*
+ * pg_numa_move_to_node
+				MPOL_BIND, nodemask->maskp, nodemask->size, MPOL_MF_MOVE);
 
 /* Empty wrappers */
 
+void

From 8265b556de61802983a3fefc31e0e2e071b456ed Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 2 Jun 2026 22:16:55 +0200
Subject: [PATCH v20260624 4/7] clock-sweep: basic partitioning

 
index 46b6b85a2e3..b07fafda0d9 100644

 PG_MODULE_MAGIC_EXT(
@@ -963,6 +963,14 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)

@@ -984,12 +992,22 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 					first_buffer,
 					last_buffer;

 
 
@@ -1005,6 +1023,18 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		values[4] = Int32GetDatum(last_buffer);
 		nulls[4] = false;
 
index d6c0cc1f6d4..7bedaf8987a 100644
 
 /*

 
 	/*

 

 	/*
 
 	/*

 

 #ifdef BGW_DEBUG

 /*

 /*
 
 	/*

 
 
 /*
 
 	/*

 
 		/*

 
 /*

+/*

 	if (complete_passes)
 	{
 
 		/*
 
-	if (num_buf_alloc)
-	{
 
 /*

-	/* Clear statistics */

 	return true;
 }
+
+void

 /* buf_table.c */

 /* inline functions */
 
index 45fc91fcb97..a4617481f7c 100644

From 4ed4cf8669a6f379ebbf39a2283704931d4b3279 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 2 Jun 2026 22:28:10 +0200
Subject: [PATCH v20260624 5/7] clock-sweep: balancing of allocations

 
index b07fafda0d9..f26b2332c1d 100644
 
@@ -31,7 +33,7 @@

 PG_MODULE_MAGIC_EXT(
 					.name = "pg_buffercache",
@@ -940,6 +942,8 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 
 	if (SRF_IS_FIRSTCALL())
 	{
 
@@ -971,6 +975,12 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)

@@ -992,11 +1002,17 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 					first_buffer,
 					last_buffer;

 		uint32		complete_passes,
 					next_victim_buffer,
-					buffer_allocs;
+					buffer_allocs,
+					buffer_req_allocs;
+
+		int		   *weights;
+		Datum	   *dweights;
+		ArrayType  *array;
 
 		Datum		values[NUM_BUFFERCACHE_PARTITIONS_ELEM];
 		bool		nulls[NUM_BUFFERCACHE_PARTITIONS_ELEM];
@@ -1005,8 +1021,16 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 						   &first_buffer, &last_buffer);

 		values[0] = Int32GetDatum(i);
 		nulls[0] = false;
@@ -1035,6 +1059,15 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 		values[8] = Int64GetDatum(buffer_allocs);
 		nulls[8] = false;
 
index 7bedaf8987a..61be05a68d4 100644

+/*
 
+	/*
 
+	/*
 
 /*
 
 /*
 
 	/*

 /*
 
 	/*
 
+/*
+ * StrategySyncBalance

 void

 

 
 
 /* inline functions */
 
-- 
2.54.0

From fafa9ba54c0201cec1084f558cdad285b5fc903b Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email-masked]>
Date: Tue, 2 Jun 2026 22:39:38 +0200
Subject: [PATCH v20260624 6/7] clock-sweep: scan all partitions

 
 /*
 
 	/*
 
 	*from_ring = false;

 
 		/*

From f5b7b6733419186e56b06ed1d8b6da7e34984c2f Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email-masked]>
Date: Thu, 11 Jun 2026 12:38:43 +0200
Subject: [PATCH v20260624 7/7] Add parttioned clocksweep and NUMA goodies.

1. Add three clocksweep GUCs to allow manipulation of partitoned clocksweep
   in runtime.
2. Add pg_buffercache_set_weights(int, int[]) to alter partition allocs (with
   clocksweep_balance_recalc=off).
3. Add debug_numa_node GUC to pin to NUMA node.
---
 .../pg_buffercache--1.7--1.8.sql              |  7 ++
 contrib/pg_buffercache/pg_buffercache_pages.c | 37 ++++++++
 src/backend/storage/buffer/freelist.c         | 86 ++++++++++++++++++-
 src/backend/tcop/postgres.c                   | 57 ++++++++++++
 src/backend/utils/misc/guc_parameters.dat     | 31 +++++++
 src/include/miscadmin.h                       |  1 +
 src/include/storage/bufmgr.h                  |  6 ++
 7 files changed, 224 insertions(+), 1 deletion(-)

diff --git a/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql b/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
index 43d2e84f9d2..9d8f4969555 100644
--- a/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
+++ b/contrib/pg_buffercache/pg_buffercache--1.7--1.8.sql
@@ -25,9 +25,16 @@ CREATE VIEW pg_buffercache_partitions AS
 	 num_req_allocs bigint,		-- handled allocs (current cycle)
 	 weights int[]);			-- balancing weights
 
+-- Register the function to set clock-sweep balance weights.
+CREATE FUNCTION pg_buffercache_set_partition(IN partition int, IN weights int[])
+RETURNS void
+AS 'MODULE_PATHNAME', 'pg_buffercache_set_partition'
+LANGUAGE C VOLATILE PARALLEL UNSAFE;
+
 -- Don't want these to be available to public.
 REVOKE ALL ON FUNCTION pg_buffercache_partitions() FROM PUBLIC;
 REVOKE ALL ON pg_buffercache_partitions FROM PUBLIC;
+REVOKE ALL ON FUNCTION pg_buffercache_set_partition(int, int[]) FROM PUBLIC;
 
 GRANT EXECUTE ON FUNCTION pg_buffercache_partitions() TO pg_monitor;
 GRANT SELECT ON pg_buffercache_partitions TO pg_monitor;
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index f26b2332c1d..2a1b7cb7549 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -81,6 +81,7 @@ PG_FUNCTION_INFO_V1(pg_buffercache_mark_dirty);
 PG_FUNCTION_INFO_V1(pg_buffercache_mark_dirty_relation);
 PG_FUNCTION_INFO_V1(pg_buffercache_mark_dirty_all);
 PG_FUNCTION_INFO_V1(pg_buffercache_partitions);
+PG_FUNCTION_INFO_V1(pg_buffercache_set_partition);

 /* Only need to touch memory once per backend process lifetime */
@@ -1077,3 +1078,39 @@ pg_buffercache_partitions(PG_FUNCTION_ARGS)
 	else
 		SRF_RETURN_DONE(funcctx);
 }
+
+/*
+ * Set the clock-sweep balance weights for a single partition.
+ */
+Datum
+pg_buffercache_set_partition(PG_FUNCTION_ARGS)
+{
+	int			partition = PG_GETARG_INT32(0);
+	ArrayType  *array = PG_GETARG_ARRAYTYPE_P(1);
+	Datum	   *elems;
+	bool	   *nulls;
+	int			nelems;
+	int		   *weights;
+
+	if (ARR_NDIM(array) > 1)
+		ereport(ERROR,
+				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+				 errmsg("weights must be a one-dimensional array")));
+
+	deconstruct_array_builtin(array, INT4OID, &elems, &nulls, &nelems);
+
+	weights = palloc_array(int, nelems);
+	for (int i = 0; i < nelems; i++)
+	{
+		if (nulls[i])
+			ereport(ERROR,
+					(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+					 errmsg("weights must not contain NULL values")));
+
+		weights[i] = DatumGetInt32(elems[i]);
+	}
+
+	ClockSweepSetWeights(partition, weights, nelems);
+
+	PG_RETURN_VOID();
+}
diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c
index 1ac1e3e3490..e677c71e0b3 100644
--- a/src/backend/storage/buffer/freelist.c
+++ b/src/backend/storage/buffer/freelist.c
@@ -55,6 +55,26 @@
  */
 #define CLOCKSWEEP_HISTORY_COEFF	0.5
 
+/*
+ * GUCs controlling the NUMA-aware clock-sweep behavior.
+ *
+ * clocksweep_balance - when enabled, allocations may get redirected between
+ * clock-sweep partitions to keep them balanced (see StrategySyncBalance).
+ *
+ * clocksweep_balance_recalc - when enabled, the balance weights are
+ * periodically recalculated (see StrategySyncBalance). Disabling this keeps
+ * the current weights, e.g. ones configured manually using
+ * pg_buffercache_set_partition(), while still using them to balance
+ * allocations (if clocksweep_balance is enabled).
+ *
+ * clocksweep_scan_all_partitions - when enabled, looking for a free buffer
+ * scans all clock-sweep partitions (in a round-robin way), not just the
+ * backend's "home" partition.
+ */
+bool		clocksweep_balance = true;
+bool		clocksweep_balance_recalc = true;
+bool		clocksweep_scan_all_partitions = true;
+
 /*
  * Information about one partition of the ClockSweep (on a subset of buffers).
  *
@@ -436,8 +456,11 @@ ChooseClockSweep(bool balance)
 	 * When balancing allocations, redirect the allocations to other partitions
 	 * according to the budgets. We move through partitions in a round-robin way,
 	 * after allocating the "budget" of allocations from the current one.
+	 *
+	 * Balancing can be disabled at runtime using the clocksweep_balance GUC, in
+	 * which case allocations always stay in the backend's "home" partition.
 	 */
-	if (balance)
+	if (balance && clocksweep_balance)
 	{
 		/*
 		 * Ran out of budget from the current partition? Move to the next one
@@ -551,6 +574,10 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint64 *buf_state, bool *from_r
 	 * Start with the "preferred" partition, and then proceed in a round-robin
 	 * manner. If we cycle back to the starting partition, it means none of the
 	 * partitions has unpinned buffers.
+	 *
+	 * Scanning of the other partitions can be disabled at runtime using the
+	 * clocksweep_scan_all_partitions GUC. In that case we only scan the
+	 * backend's "home" partition, and fail if it has no unpinned buffers.
 	 */
 	sweep = ChooseClockSweep(true);
 	sweep_start = sweep;
@@ -562,6 +589,10 @@ StrategyGetBuffer(BufferAccessStrategy strategy, uint64 *buf_state, bool *from_r
 		if (buf != NULL)
 			return buf;
 
+		/* don't look at other partitions unless allowed to */
+		if (!clocksweep_scan_all_partitions)
+			break;
+
 		/*
 		 * Try advancing to the next partition, round-robin (if last partition,
 		 * wrap around to the beginning).
@@ -739,6 +770,9 @@ StrategySyncBalance(void)
 			avg_allocs,			/* average allocations (per partition) */
 			delta_allocs = 0;	/* sum of allocs above average */
 
+	if (!clocksweep_balance || !clocksweep_balance_recalc)
+		return;
+
 	/*
 	 * Collect the number of allocations requested in the past interval.
 	 * While at it, reset the counter to start the new interval.
@@ -1481,3 +1515,53 @@ ClockSweepPartitionGetInfo(int idx,
 		(*weights)[i] = (int) sweep->balance[i];
 	}
 }
+
+/*
+ * ClockSweepSetWeights override the clock-sweep balance weights of a single
+ * partition.
+ */
+void
+ClockSweepSetWeights(int partition, int *weights, int nweights)
+{
+	ClockSweep *sweep;
+
+	/*
+	 * Disallow manual weights while the automatic recalculation is enabled, as
+	 * StrategySyncBalance would just recompute (and overwrite) them.
+	 */
+	if (clocksweep_balance_recalc)
+		ereport(ERROR,
+				(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+				 errmsg("cannot set clock-sweep weights while debug_clocksweep_balance_recalc is enabled"),
+				 errhint("Set debug_clocksweep_balance_recalc to off before setting the weights manually.")));
+
+	if ((partition < 0) || (partition >= StrategyControl->num_partitions))
+		ereport(ERROR,
+				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+				 errmsg("clock-sweep partition %d out of range", partition),
+				 errhint("There are %d clock-sweep partitions, numbered from 0.",
+						 StrategyControl->num_partitions)));
+
+	if (nweights != StrategyControl->num_partitions)
+		ereport(ERROR,
+				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+				 errmsg("number of weights (%d) does not match number of clock-sweep partitions (%d)",
+						nweights, StrategyControl->num_partitions)));
+
+	for (int i = 0; i < nweights; i++)
+	{
+		if ((weights[i] < 0) || (weights[i] > 100))
+			ereport(ERROR,
+					(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+					 errmsg("clock-sweep weight %d out of range",
+							weights[i]),
+					 errhint("Each weight must be between 0 and 100.")));
+	}
+
+	sweep = &StrategyControl->sweeps[partition];
+
+	SpinLockAcquire(&sweep->clock_sweep_lock);
+	for (int j = 0; j < nweights; j++)
+		sweep->balance[j] = (uint8) weights[j];
+	SpinLockRelease(&sweep->clock_sweep_lock);
+}
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index dbef734a93f..02787062c83 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -86,6 +86,7 @@
 #include "utils/timeout.h"
 #include "utils/timestamp.h"
 #include "utils/varlena.h"
+#include <numa.h>
 
 /* ----------------
  *		global variables
@@ -110,6 +111,9 @@ int			client_connection_check_interval = 0;
 /* flags for non-system relation kinds to restrict use */
 int			restrict_nonsystem_relation_kind;
 
+/* NUMA node to pin the backend to at query start; -1 disables pinning */
+int			debug_numa_node = -1;
+
 /*
  * Include signal sender PID/UID in the server log when available
  * (SA_SIGINFO). The caller must supply the already-captured pid and uid
@@ -1020,6 +1024,53 @@ pg_plan_queries(List *querytrees, const char *query_string, int cursorOptions,
 }

+/*
+ * process_debug_numa_node
+ *
+ * If the debug_numa_node GUC is set (>= 0), pin this backend to run on the
+ * CPUs of the requested NUMA node.  -1 disables it (default)
+ */
+static void
+process_debug_numa_node(void)
+{
+#ifdef USE_LIBNUMA
+	static int	applied_numa_node = -1;
+
+	if (debug_numa_node == applied_numa_node)
+		return;
+
+	/* Nothing we can do if the kernel/library has no NUMA support. */
+	if (numa_available() < 0)
+	{
+		applied_numa_node = debug_numa_node;
+		return;
+	}
+
+	if (debug_numa_node < 0)
+	{
+		/* Pinning disabled: allow running on all nodes again. */
+		numa_run_on_node_mask(numa_all_nodes_ptr);
+	}
+	else if (debug_numa_node > numa_max_node())
+	{
+		ereport(WARNING,
+				(errmsg("debug_numa_node %d exceeds the highest available NUMA node %d, ignoring",
+						debug_numa_node, numa_max_node())));
+	}
+	else if (numa_run_on_node(debug_numa_node) != 0)
+	{
+		ereport(WARNING,
+				(errmsg("could not pin backend to NUMA node %d: %m",
+						debug_numa_node)));
+	}
+	else
+		elog(DEBUG1, "pinned backend to NUMA node %d", debug_numa_node);
+
+	applied_numa_node = debug_numa_node;
+#endif
+}
+
+
 /*
  * exec_simple_query
  *
@@ -1044,6 +1095,9 @@ exec_simple_query(const char *query_string)
 
 	pgstat_report_activity(STATE_RUNNING, query_string);
 
+	/* Pin the backend to the configured NUMA node, if requested. */
+	process_debug_numa_node();
+
 	TRACE_POSTGRESQL_QUERY_START(query_string);
 
 	/*
@@ -2185,6 +2239,9 @@ exec_execute_message(const char *portal_name, long max_rows)
 
 	pgstat_report_activity(STATE_RUNNING, sourceText);
 
+	/* Pin the backend to the configured NUMA node, if requested. */
+	process_debug_numa_node();
+
 	foreach(lc, portal->stmts)
 	{
 		PlannedStmt *stmt = lfirst_node(PlannedStmt, lc);
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 2e71c04282c..b72fce4b92b 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -641,6 +641,27 @@
   boot_val => 'DEFAULT_ASSERT_ENABLED',
 },
 
+{ name => 'debug_clocksweep_balance', type => 'bool', context => 'PGC_USERSET', group => 'DEVELOPER_OPTIONS',
+  short_desc => 'Enables balancing of buffer allocations between clock-sweep partitions.',
+  flags => 'GUC_NOT_IN_SAMPLE',
+  variable => 'clocksweep_balance',
+  boot_val => 'true'
+},
+
+{ name => 'debug_clocksweep_balance_recalc', type => 'bool', context => 'PGC_USERSET', group => 'DEVELOPER_OPTIONS',
+  short_desc => 'Enables periodic recalculation of clock-sweep partition balance weights.',
+  flags => 'GUC_NOT_IN_SAMPLE',
+  variable => 'clocksweep_balance_recalc',
+  boot_val => 'true'
+},
+
+{ name => 'debug_clocksweep_scan_all_partitions', type => 'bool', context => 'PGC_USERSET', group => 'DEVELOPER_OPTIONS',
+  short_desc => 'Enables scanning all clock-sweep partitions when looking for a free buffer.',
+  flags => 'GUC_NOT_IN_SAMPLE',
+  variable => 'clocksweep_scan_all_partitions',
+  boot_val => 'true'
+},
+
 { name => 'debug_copy_parse_plan_trees', type => 'bool', context => 'PGC_SUSET', group => 'DEVELOPER_OPTIONS',
   short_desc => 'Set this to force all parse and plan trees to be passed through copyObject(), to facilitate catching errors and omissions in copyObject().',
   flags => 'GUC_NOT_IN_SAMPLE',
@@ -693,6 +714,16 @@
   options => 'debug_logical_replication_streaming_options',
 },
 
+{ name => 'debug_numa_node', type => 'int', context => 'PGC_USERSET', group => 'DEVELOPER_OPTIONS',
+  short_desc => 'Pins the backend to the given NUMA node at query start.',
+  long_desc => '-1 (the default) disables pinning.',
+  flags => 'GUC_NOT_IN_SAMPLE',
+  variable => 'debug_numa_node',
+  boot_val => '-1',
+  min => '-1',
+  max => 'INT_MAX',
+},
+
 { name => 'debug_parallel_query', type => 'enum', context => 'PGC_USERSET', group => 'DEVELOPER_OPTIONS',
   short_desc => 'Forces the planner\'s use parallel query nodes.',
   long_desc => 'This can be useful for testing the parallel query infrastructure by forcing the planner to generate plans that contain nodes that perform tuple communication between workers and the main process.',
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index bf38aa6baa2..9590f7ac467 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -215,6 +215,7 @@ extern PGDLLIMPORT bool MyDatabaseHasLoginEventTriggers;
 
 extern PGDLLIMPORT bool shmem_populate;
 extern PGDLLIMPORT bool shmem_interleave;
+extern PGDLLIMPORT int debug_numa_node;

 /*
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index 02833b19b0c..b5c3873acae 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -190,6 +190,11 @@ extern PGDLLIMPORT int bgwriter_lru_maxpages;
 extern PGDLLIMPORT double bgwriter_lru_multiplier;
 extern PGDLLIMPORT bool track_io_timing;
 
+/* in freelist.c */
+extern PGDLLIMPORT bool clocksweep_balance;
+extern PGDLLIMPORT bool clocksweep_balance_recalc;
+extern PGDLLIMPORT bool clocksweep_scan_all_partitions;
+
 #define DEFAULT_EFFECTIVE_IO_CONCURRENCY 16
 #define DEFAULT_MAINTENANCE_IO_CONCURRENCY 16
 extern PGDLLIMPORT int effective_io_concurrency;
@@ -418,6 +423,7 @@ extern void ClockSweepPartitionGetInfo(int idx,
 									 uint64 *buffer_total_req_allocs,
 									 uint32 *buffer_req_allocs,
 									 int **weights);
+extern void ClockSweepSetWeights(int partition, int *weights, int nweights);
 
 /* inline functions */
 
-- 
2.54.0

---

From: Jakub Wartak <[email-masked]>
Date: Thu, 25 Jun 2026 14:19:36 +0200

On Wed, Jun 24, 2026 at 10:29 PM Tomas Vondra <[email-masked]> wrote:

Hi,

[..]
> Those seem potentially useful to development.

Cool!

[..]
>   clients       master     0003      0004    0003/on    0004/on

I haven't tried it yet, however I can spot some things:

No crystal clear idea why, but in the script I can see that you have
io_method = io_uring and are not dropping_caches, so IMHO it is too complex
interaction at this stage.

One hint: such setup is going to be problematic for proving numbers.
On the meeting I've tried to describe that I've been using io_method = sync
instead of 'worker' to get more predicitable results (together with echo 3
> drop_caches), because then it is that backend's CPU/$NODE doing that
pread()/pwrite() -- or any other operating performing the load --
it is going to put that file onto that_specific_$NODE --
so even if you have sequence like:
    pgbench -i
    pg_ctl restart
    pgbench -c XX

then pgbench -i even with shared_buffers_numa=on will spread into many
nodes the Buffers, yet after the restart the VFS cache portion of the data
will still reside on single specific $NODE that wrote it to the filesystem
(due to local-first-tocuh-affinity even for VFS cache), so any further reading:
VFS cache --pread()--> s_b will take the hit of remote interconnect with
some probablity depending on where the new backends are running. Also
with worker it is even worse as we have those memory queue in between. I
think we even can have this:

file in VFS cache @ node0   --because of first touch policy (pgbench -i/prewarm)
io worker @ node1           --hits latency from node0 and node2
shm io worker queue @ node2 --well
client backend @ node 3     --puts into shm io worker on node2

Therefore I'm sticking to 'sync' to ease the pain... but with uring, I suspect
the situation is kind of similiar as we call io_uring_submit(), and we
may endup using io-wq kernel threads, and we have those submission/receive
(memory) queues that are located somewhere (that is on some node) too.

I think, we simply lack affinity for IO/NUMA for all io modes except sync, but
it's too early I suspect and way outside of scope for this $thread. I've
started thinking about it just last week, so... (but hopefully I'll be able
to ship helper fscachenuma.c to show layout of file across VFS caches on nodes
next week I hope)

Maybe some other suggestions:

Q1) Maybe some crosschecks first?
       # balance should be equal between nodes even for baseline
       # linux kernel has tendency to fit shm into one if it fits
       find /sys/devices/system/node*/ -name 'free_hugepages' -exec
grep -H . {} \;

       # check N0 and N1 even for default policy, might also reveal imbalance
       # lots of RAM and too big huge_pages allows fitting whole shm
into just N0
       # see point 4 from [1]
       grep /anon_h /proc/$SOMEREALBACKENDPID/numa_maps

       # then during pgbench -c run maybe those:
       mpstat -N ALL 1
       perf stat -a -e uncore_imc/cas_count_read/,uncore_imc/cas_count_write/ \
          --per-socket -I 1000  # or -M
memory_bandwidth_read,memory_bandwidth_write

    (it might reveal that problem I've described above about io_method:
    even with pgbench -c 1 you might be reading from all sockets/wrong sockets
    instead of the correct one with affinity)

    I like to pin CPUs to just one node for pgbench -c
<NUMBER_OF_CPUS/NUMBER_OF_NODES>
    [to saturate one node only] and start server also with CPU pining
    [or use this debug_numa_node to force] to that one node and cross-check
    what's being read (using perf) and usually I have to disarm clock balancing
    and override weights using pg_buffercache_set_partition() to also force
    weight to stay local only - only then I'm able to outrun master. That's
    how this idea was born that if we are only working on node $N with
some relations
    then let's use only node $N's Buffers. But I have 90us:~280us
local vs remote
    latency, so it's probably way easier for me to see results even without
    disabling CPU-idle-states/turboboost/etc.

Q2) Dunno, but 0007 is not changing anything in runtime and you get huge
    discrepeancy results when going 0006 -> 0007 for clients=1 (see
128% -> 112%).
    Literally, as the same code but different rebuild (ELF image)
would be having
    vastly different layout enough to cause perf issues?

Hopefully next week I'll try to repro those numbers to see if I can
help more.

-J.

[1] - https://www.postgresql.org/message-id/CAKZiRmzo9xnJSgO4b26DTZqPuObcQ-6ncay%2BmOEKs9rzCkegUA%40mail.gmail.com

---

From: Tomas Vondra <[email-masked]>
Date: Thu, 25 Jun 2026 15:49:00 +0200

> interaction at this stage.

By caches I assume you mean page cache? The test is meant so simulate a
cached system, copying data between shared buffers and page cache. My
expectation is that once we start hitting I/O, it'll completely hide
most differences due to NUMA.

> so even if you have sequence like:

Hmm, I missed that point during the meeting. I wonder if "worker" is
interacting with the NUMA somehow (I mean, does it load it into the
right node?). But I'm using io_uring, and it's not clear to me why sync
would be better for benchmarking?

Ultimately, we need to make sure it works well with io_uring anyway,
right? Even if "sync" happens to be better for benchmarking (or even for
NUMA stuff), we have to make it work with worker/io_uring. Because
that's what practical systems use.

> to ship helper fscachenuma.c to show layout of file across VFS caches on nodes

Ah, you're suggesting the page cache stuff will be placed on a single
NUMA node? That may be true, it's a good point. And maybe it could skew
the results in a bad way. Still, that would be the case even without the
NUMA partitioning, no?

>     instead of the correct one with affinity)

I'll try, but if you could try running some experiments on your own,
that might be helpful.

>     discrepeancy results when going 0006 -> 0007 for clients=1 (see

Thank you! That'd be great.

regards

-- 
Tomas Vondra

---

From: Jakub Wartak <[email-masked]>
Date: Mon, 29 Jun 2026 09:42:49 +0200

> most differences due to NUMA.

No, it wont completley hide it, those differences at least here still matter
(AFAIR right now like +/- 10% here)

> that's what practical systems use.

Yes, we need to make work with more advanced, but I don't think we are there
yet (we'll need some more patches in orde rto demonstrate it reliably).

> > (due to local-first-tocuh-affinity even for VFS cache),
> > [.. blabla , use io_method=sync ]
> >
>
> the results in a bad way.

I've just published [0], see for yourself:

This happens especiall after pgbench -i, so:
     pgbench -i # pagecache placement on one NUMA node
     pg_ctl restart
     pgbench -c XX

is day and night different than let's say:
     pgbench -i
     echo 3 > drop_caches
     pg_ctl restart
     pgbench -c XX # pagecache placement happens by many backends
                   # potentially many NUMA nodes

> Still, that would be the case even without the NUMA partitioning, no?

Right, in my experience we should not benchmark against master started
with the default pg_ctl (that's is without numactl --interleave=all) because
it is confusing to reason about it due how the s_b could laid out without
that interleaving. I mean later we can switch to that default, but IMHO not
yet.

[..]
> Thank you! That'd be great.

Yeah, I'll try my best, we'll see how it goes. Right now I've just dropped
that fscachenuma proggie to aid us in troubleshooting.

-J.

[0] - https://github.com/jakubwartakEDB/fscachenuma

---

From: Jakub Wartak <[email-masked]>
Date: Tue, 30 Jun 2026 14:51:18 +0200

> > >>   clients       master     0003      0004    0003/on    0004/on

[..lots of variables..]

> [0] - https://github.com/jakubwartakEDB/fscachenuma

Hi Tomas,

OK, so I've run couple of tests and modified run.sh and also tried to fix
some inefficiencies spotted while testing this. Note the attached
performance matrix is in TPS (so more is better). Raw results/CSV and
scripts are attached too.

* run2 = 2 workloads, partitioned pgbench_accounts
* run3 = just pgbenchS w/o partitioning + warmup
* run4 = semi-like pgbenchS w/o partitioning but 100k rows + warmup

One important modification in those run shell scripts is that they
clean page-cache (drop_cached) as mentioned earlier to avoid false results
where everything would on node#N after pgbench -i ran. Probably I did
not get any regressions you've got, because of this. Or better diff -u
run*.sh scripts.

The "inst-optimized" is just the same patchset (so "inst-patchset") + crude
attempt in 0008 to make further smooth out things and avoid regressions while
I've been working on this. 0008 does couple of things:

a. implements CPU/node caching instead quering it every single buffer. Even
   if on x86_64 that is optimized by vdso/kernel to avoid the real syscall,
   the semi-syscall tax seems to be visible when fetching lots of buffers.
   128 is arbitrary and still kind of low (128*8kB=1MB, and we are doing
   hundreths of MB/s; while rescheduling happened only every couple of
   seconds).

b1. minimize the attempt to use other partittions till some threshold (
   and then it relies on the scan-all-partitions)

b2. avoids selecting idle partitions (defined as avg_allocs/2) - if there
   are low allocations there it is debatable if cache utilization is better
   or sticking to lower latency is better (e.g. in some workloads buffer
   reuse is close to 0, so lower latency is clearly better)

Results are attached, some observations:

0.There were vast differences in how pg_ctl is started (interleaved or not),
  so I've decided in the end to show relative to both situations.

1.In run2/seqconcurrscans I've saturated my interconnect and that's why
  it's giving 129-155% there. I don't have access physiscal hw, but I suspect
  that modern 2socket EPYC5 has like ~614GB/s per socket RAM bandwidth,
  but the max oneway bandwith of the interconnect is around ~220GB/s (
  no way to provie it), so *IF* with hundreths of cores we would be able
  fetch at this rate we could saturte modern hardware too that way (and
  we birefly touched related topic: batched executor, accelerating it
  so fast those effects could be more easily achieveable)

2.run3 has no partiitioning because according to perf and my eyes, it
  spent time not on the buffers itself (thus it was way heavier on CPU
  [partitioning] than on memory...), so that's how run3 was born without
  partitions :D

3.The warmup is critical for run3/pgbenchS, as I've noticed that depending
  on ${luck} if you start the "master" (baseline w/o interleaving) and pgbench
  it right away everything might land on node0 (s_b, pagecache), so "master"
  was basically cheating in benchmarks vs especially Your's patchset where
  it was spreading way too soon. Having drop_caches, additional warump and
  only then proper pgbench kind of reduces that luck-factor. In general I
  think all runs with c=1 seem to have kind of low singal-to-noise ratio. I
  was thinking about pinning to always stick to the same NUMA node from start
  to win against master just for this c=1 scenarios, but "meh".

3b. in short for pgbench -S we can gain like 2-5%

4.run4 was made just to prove that workload fetching more buffers, than
  the standard pgbench -S (1 row?), seems to be the key to prove
  optimizations in 0008 (other than showing good benefits for seqconcurrscans
  of course). So run4 just shows benefit compared to 0001-0007 alone.

Stil on the table:

1. maybe even better balancing is possible (?), but this one is seems enough?
   I'm out of other ideas, well other than the
   "shared-relation-use-by-foreign-node" idea described much earlier (but
   I won't be able to pull that off), so I'm not entering this rabbit hole
   any deeper.

2. Digging into io_method=worker optimizations (answering question: are they
   necessary?) Maybe I'll throw in run5 quite soon, this is going to be
   crucial to answer.

3. Potentially mentioned earlier BAS strategies (forcing just use of local
   partitions for known-to-be-only-local-users: CTAS/VACCUM/etc), but I'm
   afarid that's not for me as I would certainly break/violate some
   invisible to me boundary.

Maybe You could run those run*.sh with master vs inst-patchset/optimized?
(I'm not sure, maybe there's even different factor at play too...)

-J.
From 8513d188ed5ed999e72fc3a58046bbc1ff9f5688 Mon Sep 17 00:00:00 2001
From: Jakub Wartak <[email-masked]>
Date: Tue, 30 Jun 2026 14:22:02 +0200
Subject: [PATCH v20260630-0008] clock-sweep: cached CPU/NUMA node and more
 locality-aware balancing

Enhancements on top of 0001-0007, to have sligthly better NUMA locality
and perfromance.

1. Cache numa_node_of_cpu()/sched_getcpu() per backend in
   ClockSweepPartitionIndex(), refreshing every CLOCKSWEEP_CPU_NODE_REFRESH
   allocations rather than on every call (visible hot buffer path in perf)

2. CLOCKSWEEP_BALANCE_THRESHOLD - make it less likely to redirect on any
   surplus of allocations (so scatter buffers LESS onto remote nodes).
   With this, it redirects its allocations to other (remote?) partitions
   when the allocation exceeds the per-partition average allocation rate
   by this percentage factor .

3. Avoid redirects to "idle" partitions: a redirect partition target
   must have some traffic which is at least 2x our demand. This elimnates
   cold partitions, but we can still reach them using scan-all-partitions
   fallback.
---
 src/backend/storage/buffer/freelist.c | 85 +++++++++++++++++++++++----
 1 file changed, 74 insertions(+), 11 deletions(-)

index e677c71e0b3..d64c2c67eb6 100644
@@ -55,6 +55,9 @@
 
+/* How often backend should re-fetch the CPU/node on which it is running on? */
+#define CLOCKSWEEP_CPU_NODE_REFRESH	128
+
 /*
  * GUCs controlling the NUMA-aware clock-sweep behavior.
  *
@@ -70,6 +73,7 @@
  * clocksweep_scan_all_partitions - when enabled, looking for a free buffer
  * scans all clock-sweep partitions (in a round-robin way), not just the
  * backend's "home" partition.
+ *
  */
 bool		clocksweep_balance = true;
 bool		clocksweep_balance_recalc = true;
@@ -368,13 +372,29 @@ ClockSweepPartitionIndex(void)
 #ifdef USE_LIBNUMA
 	if (shared_buffers_numa)
 	{
-		int		cpu;
+		/*
+		 * Cache the CPU/NUMA node, refreshing only every CLOCKSWEEP_CPU_NODE_REFRESH
+		 * allocations. It appears that sched_getcpu()/numa_node_of_cpu() are not free.
+		 * On some platforms it take price of full system call, or the rest (x86_64?)
+		 * is can be use VDSO optimization. The backend rarely migrates between NUMA
+		 * nodes, and the balance logic only needs to notice migration after some time,
+		 * so an occasional refresh is good enough.
+		 */
+		static int		cached_node = -1;
+		static uint32	refresh_counter = 0;
+
+		if (cached_node < 0 || (refresh_counter++ % CLOCKSWEEP_CPU_NODE_REFRESH) == 0)
+		{
+		  int cpu;
 
-		/* XXX do we need to check sched_getcpu is available, somehow? */
-		if ((cpu = sched_getcpu()) < 0)
+		  /* XXX do we need to check sched_getcpu is available, somehow? */
+		  if ((cpu = sched_getcpu()) < 0)
 			elog(ERROR, "sched_getcpu failed: %m");
 
-		node = numa_node_of_cpu(cpu);
+		  /* XXX/JW: use libnuma wrapper for this */
+		  cached_node = numa_node_of_cpu(cpu);
+		}
+		node = cached_node;
 	}
 #endif
 
@@ -768,7 +788,8 @@ StrategySyncBalance(void)
 
 	uint32	total_allocs = 0,	/* total number of allocations */
 			avg_allocs,			/* average allocations (per partition) */
-			delta_allocs = 0;	/* sum of allocs above average */
+			delta_allocs = 0,	/* sum of allocs above average */
+			redirect_cutoff;	/* redirect only above this many allocs */
 
 	if (!clocksweep_balance || !clocksweep_balance_recalc)
 		return;
@@ -852,6 +873,20 @@ StrategySyncBalance(void)
 		return;
 	}
 
+	/*
+	 * A partition only redirects allocations to other partitions when it
+	 * exceeds the average by more than some threshold percent.
+	 * Below this cutoff we keep allocations local, to preserve NUMA locality.
+	 *
+	 * TODO: maybe better value is possible. On 4s with 25 I've got good results,
+	 *       but with value of 50 I've got slight degradation. Maybe it should 
+	 *       be equal to 100/numa_nodes ?
+	 *
+	 */
+#define CLOCKSWEEP_CUTOFF_THRESHOLD 25
+	redirect_cutoff = avg_allocs +
+		(uint32) ((uint64) avg_allocs * CLOCKSWEEP_CUTOFF_THRESHOLD / 100);
+
 	/*
 	 * The actual rebalancing
 	 *
@@ -884,10 +919,15 @@ StrategySyncBalance(void)
 		/* reset the weights to start from scratch */
 		memset(balance, 0, sizeof(uint8) * MAX_BUFFER_PARTITIONS);
 
-		/* does this partition has fewer or more than avg_allocs? */
-		if (allocs[i] < avg_allocs)
+		/*
+		 * Does this partition exceed its fair share by more than the
+		 * threshold? If not, keep all allocations local - redirecting them
+		 * would push memory onto remote NUMA nodes for no real benefit when
+		 * the load is already close to balanced.
+		 */
+		if (allocs[i] <= redirect_cutoff)
 		{
-			/* fewer - don't redirect any allocations elsewhere */
+			/* near fair share (or below) - keep allocations local */
 			balance[i] = 100;
 		}
 		else
@@ -902,22 +942,45 @@ StrategySyncBalance(void)
 			/* fraction of the "total" delta */
 			double	delta_frac = (allocs[i] - avg_allocs) * 1.0 / delta_allocs;
 
-			/* keep just enough allocations to meet the target */
-			balance[i] = (100.0 * avg_allocs / allocs[i]);
+			/* how much we keep local; we hand out the rest below */
+			int		kept = 100;
 
 			/* redirect the extra allocations */
 			for (int j = 0; j < StrategyControl->num_partitions; j++)
 			{
 				/* How many allocations to receive from i-th partition? */
 				uint32	receive_allocs = delta_frac * (avg_allocs - allocs[j]);
+				int		w;
+
+				/* do not redirect to ourselves */
+				if (j == i)
+					continue;
 
 				/* ignore partitions that don't need additional allocations */
 				if (allocs[j] > avg_allocs)
 					continue;
 
+				/*
+				 * Only use other partitions that actually have demand of
+				 * their own (avoid idle). If we fail, there's always the
+				 * scan-all-partitions fallback.
+				 *
+				 * TODO:: just guessing,heuristics
+				 */
+				if (allocs[j] < (avg_allocs / 2))
+					continue;
+
 				/* fraction to redirect */
-				balance[j] = (100.0 * receive_allocs / allocs[i]) + 0.5;
+				w = (int) ((100.0 * receive_allocs / allocs[i]) + 0.5);
+				balance[j] = w;
+				kept -= w;
 			}
+
+			/* avoid negative balances */
+			if (kept > 0)
+				balance[i] = kept;
+			else
+				balance[i] = 1;
 		}
 
 		/* combine the old and new weights (hysteresis) */
-- 
2.43.0

---

From: Jakub Wartak <[email-masked]>
Date: Thu, 2 Jul 2026 11:24:21 +0200

[..]
>    I won't be able to pull that off), so I'm not entering this rabbit hole

See below, seems like not needed (?)

>    necessary?) Maybe I'll throw in run5 quite soon, this is going to be

OK, I'm attaching are results from mine runs 5 and 6:
- only seqconcurrscans was tested, well because for other workloads io_worker
  method was not getting load for those workers (only seq scans were offloaded)
- checksums were disabled, because IMHO that would be unfair comparision
  (AFAIR there are offloaded)
- those optimizations for 0008 "optimized (numa=on, bal=on)" easily beat
  "patched (numa=on, bal=on)" and seem to be crucial. We get like 1.2x-1.4x
  across every io_method, but only with 0008.
- even when then doing just those logical fully cached reads from fully VFS
  cached case, io_urings shines (I've added raw TPS number to show this,
  compare across tables e.g. io_uring vs sync  13.378/8.993=1.487x for
  io_uring with NUMA, but for master's for io_uring:sync it was just 8.79/7.389
  = 1.189x without NUMA); seems like io_uring is more lightweight to show
  more benefits of remote memory latencies
- there's some more juice to get out of the balancer for 0-reuse workloads
  (but IMHO it's pointless to squeeze more, it's hard already)
- I was probably wrong when expecting that io_worker's worker processes/queues
  should get NUMA affinity. They don't need to be apparently for me to see
  benefits (maybe they could  be and it would even better, but meh).
  So with ruling io_method impact (I speculated earlier that his could be it),
  this means that you were either hitting lack of opimizations needed from
  0008 or were impacted by lack of drop_caches before the runs

> (I'm not sure, maybe there's even different factor at play too...)

This is seems to be crucial now, to double confirm the results / loaded-tested
on your hw with 0008. (but that hardware really needs to have effective latency
difference between at least 2 NUMA nodes -- Intel's mlc is good for this);
maybe also tweak those 125% inside 0008 to some other values, I've got 4 nodes,
so 100/4=25%)

>    invisible to me boundary.

And this one is still potentially on the table as nice thing to have.

-J.