public inbox for [email protected]  

help / color / mirror / Atom feed

From: Burd, Greg <[email protected]>
To: Andres Freund <[email protected]>
Cc: Tomas Vondra <[email protected]>
Cc: Ashutosh Bapat <[email protected]>
Cc: PostgreSQL Hackers <[email protected]>
Cc: Dmitry Dolgov <[email protected]>
Subject: Re: Adding basic NUMA awareness
Date: Fri, 11 Jul 2025 13:34:26 -0400
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <uaqt63a2kz2g3qk2ujr55gdvmymnak3zsplcgrlh63ekbrytbc@a2wy22p3slge>
	<[email protected]>
	<ndvygkpdx44pmi4xbkf52gfrl77cohpefr42tipvd5dgiaeuyd@fe2og2kxyjnc>
	<[email protected]>



> On Jul 10, 2025, at 8:13 AM, Burd, Greg <[email protected]> wrote:
> 
> 
>> On Jul 9, 2025, at 1:23 PM, Andres Freund <[email protected]> wrote:
>> 
>> Hi,
>> 
>> On 2025-07-09 12:55:51 -0400, Greg Burd wrote:
>>> On Jul 9 2025, at 12:35 pm, Andres Freund <[email protected]> wrote:
>>> 
>>>> FWIW, I've started to wonder if we shouldn't just get rid of the freelist
>>>> entirely. While clocksweep is perhaps minutely slower in a single
>>>> thread than
>>>> the freelist, clock sweep scales *considerably* better [1]. As it's rather
>>>> rare to be bottlenecked on clock sweep speed for a single thread
>>>> (rather then
>>>> IO or memory copy overhead), I think it's worth favoring clock sweep.
>>> 
>>> Hey Andres, thanks for spending time on this.  I've worked before on
>>> freelist implementations (last one in LMDB) and I think you're onto
>>> something.  I think it's an innovative idea and that the speed
>>> difference will either be lost in the noise or potentially entirely
>>> mitigated by avoiding duplicate work.
>> 
>> Agreed. FWIW, just using clock sweep actually makes things like DROP TABLE
>> perform better because it doesn't need to maintain the freelist anymore...
>> 
>> 
>>>> Also needing to switch between getting buffers from the freelist and
>>>> the sweep
>>>> makes the code more expensive.  I think just having the buffer in the sweep,
>>>> with a refcount / usagecount of zero would suffice.
>>> 
>>> If you're not already coding this, I'll jump in. :)
>> 
>> My experimental patch is literally a four character addition ;), namely adding
>> "0 &&" to the relevant code in StrategyGetBuffer().
>> 
>> Obviously a real patch would need to do some more work than that.  Feel free
>> to take on that project, I am not planning on tackling that in near term.
>> 
> 
> I started on this last night, making good progress.  Thanks for the inspiration.  I'll create a new thread to track the work and cross-reference when I have something reasonable to show (hopefully later today).
> 
>> There's other things around this that could use some attention. It's not hard
>> to see clock sweep be a bottleneck in concurrent workloads - partially due to
>> the shared maintenance of the clock hand. A NUMAed clock sweep would address
>> that.
> 
> Working on it.

For archival sake, and to tie up loose ends I'll link from here to a new thread I just started that proposes the removal of the freelist and the buffer_strategy_lock [1].

That patch set doesn't address any NUMA-related tasks directly, but it should remove some pain when working in that direction by removing code that requires partitioning and locking and...

best.

-greg

[1] https://postgr.es/m/[email protected]

view thread (89+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
  Subject: Re: Adding basic NUMA awareness
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox