public inbox for [email protected]  

help / color / mirror / Atom feed

From: Cédric Villemain <[email protected]>
To: Tomas Vondra <[email protected]>
To: PostgreSQL Hackers <[email protected]>
Subject: Re: Adding basic NUMA awareness - Preliminary feedback and outline for an extensible approach
Date: Sat, 5 Jul 2025 07:09:00 +0000
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>

Hi Tomas,


I haven't yet had time to fully read all the work and proposals around 
NUMA and related features, but I hope to catch up over the summer.

However, I think it's important to share some thoughts before it's too 
late, as you might find them relevant to the NUMA management code.


> 6) v1-0006-NUMA-pin-backends-to-NUMA-nodes.patch
> 
> This is an experimental patch, that simply pins the new process to the
> NUMA node obtained from the freelist.
> 
> Driven by GUC "numa_procs_pin" (default: off).


In my work on more careful PostgreSQL resource management, I've come to 
the conclusion that we should avoid pushing policy too deeply into the 
PostgreSQL core itself. Therefore, I'm quite skeptical about integrating 
NUMA-specific management directly into core PostgreSQL in such a way.


We are working on a PROFILE and PROFILE MANAGER specification to provide 
PostgreSQL with only the APIs and hooks needed so that extensions can 
manage whatever they want externally.

The basic syntax (not meant to be discussed here, and even the names 
might change) is roughly as follows, just to illustrate the intent:


CREATE PROFILE MANAGER manager_name [IF NOT EXISTS]
   [ HANDLER handler_function | NO HANDLER ]
   [ VALIDATOR validator_function | NO VALIDATOR ]
   [ OPTIONS ( option 'value' [, ... ] ) ]

CREATE PROFILE profile_name
   [IF NOT EXISTS]
   USING profile_manager
     SET key = value [, key = value]...
   [USING profile_manager
     SET key = value [, key = value]...]
   [...];

CREATE PROFILE MAPPING
   [IF NOT EXISTS]
   FOR PROFILE profile_name
   [MATCH [ ALL | ANY ] (
     [ROLE role_name],
     [BACKEND TYPE backend_type],
     [DATABASE database_name],
     [APPLICATION appname]
   )];

## PROFILE RESOLUTION ORDER

1. ALTER ROLE IN DATABASE
2. ALTER ROLE
3. ALTER DATABASE
4. First matching PROFILE MAPPING (global or specific)
5. No profile (fallback)

As currently designed, this approach allows quite a lot of flexibility:

* pg_psi is used to ensure the spec is suitable for a cgroup profile 
manager (moving PIDs as needed; NUMA and cgroups could work well 
together, see e.g. this Linux kernel summary: 
https://blogs.oracle.com/linux/post/numa-balancing )

* Someone else could implement support for Windows or BSD specifics.

* Others might use it to integrate PostgreSQL's own resources (e.g., 
"areas" of shared buffers) into policies.

Hope this perspective is helpful.

Best regards,
-- 
Cédric Villemain +33 6 20 30 22 52
https://www.Data-Bene.io
PostgreSQL Support, Expertise, Training, R&D

view thread (79+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected]
  Subject: Re: Adding basic NUMA awareness - Preliminary feedback and outline for an extensible approach
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox