Message-ID: <2616b89d-3cd0-4a38-915d-2741da81e0a1@vondra.me>
Date: Tue, 8 Jul 2025 00:35:43 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Adding basic NUMA awareness - Preliminary feedback and outline
 for an extensible approach
To: =?UTF-8?Q?C=C3=A9dric_Villemain?= <cedric.villemain@data-bene.io>,
 PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
References: <099b9433-2855-4f1b-b421-d078a5d82017@vondra.me>
 <1caea3b1-3e0b-4bdb-a630-75671f1f837a@data-bene.io>
 <8b3341ee-1f8e-4dbb-8046-410f71d4a0cb@vondra.me>
 <f1f03ad5-91b4-4028-8304-4e60c3b0a469@data-bene.io>
Content-Language: en-US
From: Tomas Vondra <tomas@vondra.me>
In-Reply-To: <f1f03ad5-91b4-4028-8304-4e60c3b0a469@data-bene.io>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Archived-At: 
 <https://www.postgresql.org/message-id/2616b89d-3cd0-4a38-915d-2741da81e0a1%40vondra.me>
Precedence: bulk

On 7/7/25 16:51, Cédric Villemain wrote:
>>> * Others might use it to integrate PostgreSQL's own resources (e.g.,
>>> "areas" of shared buffers) into policies.
>>>
>>> Hope this perspective is helpful.
>>
>> Can you explain how you want to manage this by an extension defined at
>> the SQL level, when most of this stuff has to be done when setting up
>> shared memory, which is waaaay before we have any access to catalogs?
> 
> I should have said module instead, I didn't follow carefully but at some
> point there were discussion about shared buffers resized "on-line".
> Anyway, it was just to give some few examples, maybe this one is to be
> considered later (I'm focused on cgroup/psi, and precisely reassigning
> PIDs as needed).
> 

I don't know. I have a hard time imagining what exactly would the
policies / profiles do exactly to respond to changes in the system
utilization. And why should that interfere with this patch ...

The main thing patch series aims to implement is partitioning different
pieces of shared memory (buffers, freelists, ...) to better work for
NUMA. I don't think there's that many ways to do this, and I doubt it
makes sense to make this easily customizable from external modules of
any kind. I can imagine providing some API allowing to isolate the
instance on selected NUMA nodes, but that's about it.

Yes, there's some relation to the online resizing of shared buffers, in
which case we need to "refresh" some of the information. But AFAICS it's
not very extensive (on top of what already needs to happen after the
resize), and it'd happen within the boundaries of the partitioning
scheme. There's not that much flexibility.

The last bit (pinning backends to a NUMA node) is experimental, and
mostly intended for easier evaluation of the earlier parts (e.g. to
limit the noise when processes get moved to a CPU from a different NUMA
node, and so on).


regards

-- 
Tomas Vondra