Message-ID: <1caea3b1-3e0b-4bdb-a630-75671f1f837a@data-bene.io>
Date: Sat, 5 Jul 2025 07:09:00 +0000
MIME-Version: 1.0
Subject: Re: Adding basic NUMA awareness - Preliminary feedback and outline
 for an extensible approach
To: Tomas Vondra <tomas@vondra.me>,
 PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
References: <099b9433-2855-4f1b-b421-d078a5d82017@vondra.me>
From: =?UTF-8?Q?C=C3=A9dric_Villemain?= <cedric.villemain@data-bene.io>
Autocrypt: addr=cedric.villemain@data-bene.io; keydata=
 xsBNBEvW1WwBCAC6CG3zZnRpQwmSiFoTjaHlgt7RQ2RmYDqSB7DNtOItV9bBDUe4DKxV9iNF
 SoD2yV2LrzsYghWXwI0snHGVGCLkJFYBkqyMnJeli5PoHcn9jI12mOz0hdpsTmTJbH+JCLXF
 admIjh+sz+m7X4net3UQhj8tCaExkdNsw7FWfNr6oHOVkLWlTWh1df1dfTZ3CBZ8vc1L6dIn
 mWe3i2sRGQ/fCy7XfLqjFe5U+D3jXmqF92p7n8v0vwDZarn4MVeBegTtBYUxpM8Bj78Lga9a
 SW+ojYo21Zk/TYn+fdOX4L9nNiRFgnAYVjPlWwjV30WpVlQ56o8U+zV2hCNRQyl5QxkNABEB
 AAHNMUPDqWRyaWMgVmlsbGVtYWluIDxjZWRyaWMudmlsbGVtYWluQGRhdGEtYmVuZS5pbz7C
 wI4EEwEKADgWIQQ66LJpfJwdoY4rdHTbIB9Ms1QXmwUCY3ARzwIbAwULCQgHAgYVCgkICwIE
 FgIDAQIeAQIXgAAKCRDbIB9Ms1QXm9oTB/41lFaXCcwtzLjjN7Qr7lImBjiTOntvhzFOBUMP
 xZWmrXzVl1jVzQm8ubsZNJFB98kw5MrhZjY0sY4x6Txlh6y/OTtpT0RWfZQOGSmfpMq8QDuV
 gymCfXyr0t4XHFou31nlaqEz1CyUi8ZaYdutJAV0L8j1e0LUHvTcCmlHOsb1uS+Xf5yc7ub+
 eRiGtjXwM+OJBYYkFZ2Ebyw7mHWRF4+B3QXUjZhPCQZWBQE1GeX1KxrSCfSKfqujVWCwgiF9
 q5TQisgzVRwPBwsi5Fam2X7UHD8I6ZjaAiKHJXezP/Ied/bup457DlXfOg5ORePJgIldMo0p
 AtHIY1061TckCPs9zsBNBEvW1WwBCAC1iEAHk3lEWEDp9O/DebejwyNr9Zmun6NXPN5axuSj
 5VKWP/2CCmsPkcIQlH70rEtZl4vQjXrdRviRcc56TC/SSXti8uOVr/55uStitTCjKRkpxPdf
 ReOFuOB7OUkHrE3gF6N0UNgOMgmBsT5+rgFm+xqY6TOrvnDVGUbAGkJIdzgTjzeu96BcBEuF
 59r+grsmff8PUGtertKeXBr8NlliYZEFCilqus3PheOetWewysHROQJrqnSkpRP7zJWyFYY5
 W6PWKQlUgcItrWsgPlHl5gmoH68zesTlW1rX1Z56V8vLPF7RTAPG6g1J7l29wLxUYTwF5Y64
 gy+jXAEUJHKZABEBAAHCwF8EGAECAAkFAkvW1WwCGwwACgkQ2yAfTLNUF5ubKAgAtZdh0QJU
 Wc7WfZhoGOcAEADZJ9n6GSuhL+oxuhPfym1zsU70241rKYZH1m53CWWo+aNyThCXJx+HSSRV
 /hLNVW34DErJeNJIYwdxPo5/AjOhIUr4e8gMYuviFESBaMpgl8mSM1B1cHVBMZcsQb43Rc3V
 VfIMNRc4NB16uc7Sm++fbraQq13rB8HC6uYu7PbEAdX3C0IokR3JGb1NghQKga/OoU22KPEx
 Y+bFk4h3vq9MjlKLir0UJZHKSQ5xec6RcehoMBy0FHBEuK/72xN36dR/3bVwyeR76S7xf9UQ
 gXdpciBG95/tLRcpelwx8GaRcivTGc0FCBUzqG/YcCqvcA==
In-Reply-To: <099b9433-2855-4f1b-b421-d078a5d82017@vondra.me>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: quoted-printable
Archived-At: 
 <https://www.postgresql.org/message-id/1caea3b1-3e0b-4bdb-a630-75671f1f837a%40data-bene.io>
Precedence: bulk

Hi Tomas,


I haven't yet had time to fully read all the work and proposals around=20
NUMA and related features, but I hope to catch up over the summer.

However, I think it's important to share some thoughts before it's too=20
late, as you might find them relevant to the NUMA management code.


> 6) v1-0006-NUMA-pin-backends-to-NUMA-nodes.patch
>=20
> This is an experimental patch, that simply pins the new process to the
> NUMA node obtained from the freelist.
>=20
> Driven by GUC "numa_procs_pin" (default: off).


In my work on more careful PostgreSQL resource management, I've come to=20
the conclusion that we should avoid pushing policy too deeply into the=20
PostgreSQL core itself. Therefore, I'm quite skeptical about integrating=20
NUMA-specific management directly into core PostgreSQL in such a way.


We are working on a PROFILE and PROFILE MANAGER specification to provide=20
PostgreSQL with only the APIs and hooks needed so that extensions can=20
manage whatever they want externally.

The basic syntax (not meant to be discussed here, and even the names=20
might change) is roughly as follows, just to illustrate the intent:


CREATE PROFILE MANAGER manager_name [IF NOT EXISTS]
   [ HANDLER handler_function | NO HANDLER ]
   [ VALIDATOR validator_function | NO VALIDATOR ]
   [ OPTIONS ( option 'value' [, ... ] ) ]

CREATE PROFILE profile_name
   [IF NOT EXISTS]
   USING profile_manager
     SET key =3D value [, key =3D value]...
   [USING profile_manager
     SET key =3D value [, key =3D value]...]
   [...];

CREATE PROFILE MAPPING
   [IF NOT EXISTS]
   FOR PROFILE profile_name
   [MATCH [ ALL | ANY ] (
     [ROLE role_name],
     [BACKEND TYPE backend_type],
     [DATABASE database_name],
     [APPLICATION appname]
   )];

## PROFILE RESOLUTION ORDER

1. ALTER ROLE IN DATABASE
2. ALTER ROLE
3. ALTER DATABASE
4. First matching PROFILE MAPPING (global or specific)
5. No profile (fallback)

As currently designed, this approach allows quite a lot of flexibility:

* pg_psi is used to ensure the spec is suitable for a cgroup profile=20
manager (moving PIDs as needed; NUMA and cgroups could work well=20
together, see e.g. this Linux kernel summary:=20
https://blogs.oracle.com/linux/post/numa-balancing )

* Someone else could implement support for Windows or BSD specifics.

* Others might use it to integrate PostgreSQL's own resources (e.g.,=20
"areas" of shared buffers) into policies.

Hope this perspective is helpful.

Best regards,
--=20
C=C3=A9dric Villemain +33 6 20 30 22 52
https://www.Data-Bene.io
PostgreSQL Support, Expertise, Training, R&D