public inbox for [email protected]
help / color / mirror / Atom feedFrom: Andres Freund <[email protected]>
To: Melanie Plageman <[email protected]>
Cc: [email protected]
Cc: Thomas Munro <[email protected]>
Cc: Peter Geoghegan <[email protected]>
Cc: Tomas Vondra <[email protected]>
Cc: Nazir Bilal Yavuz <[email protected]>
Subject: Re: AIO / read stream heuristics adjustments for index prefetching
Date: Thu, 2 Apr 2026 17:30:05 -0400
Message-ID: <hxnsd6madv7em6gataql6fmjekfc5zthb3kp5w3e3mxjdonplf@spudt4jcmeep> (raw)
In-Reply-To: <CAAKRu_bbZFF9dMOOMopOMy6BgvCxEGeWmcrCvwffc6ibGE+ZdQ@mail.gmail.com>
References: <f3xxfrkafjxpyqxywcxricxgyizjirfceychyxsgn7bwjp5eda@kwbduhy7tfmu>
<CAAKRu_bfwBzg7=Zy88st6gBJf97Wkd3k=+m1ecApn=59SwmKSw@mail.gmail.com>
<pj4kgtdrevvkfbmlri6p27belctxru7ytyprcb6v74c7zbh3l6@m4dcu2rljedv>
<CAAKRu_bbZFF9dMOOMopOMy6BgvCxEGeWmcrCvwffc6ibGE+ZdQ@mail.gmail.com>
Hi,
On 2026-04-02 17:13:34 -0400, Melanie Plageman wrote:
> On Thu, Apr 2, 2026 at 11:47 AM Andres Freund <[email protected]> wrote:
> >
> > > On some level, relying on worker mode overhead feels fragile. If
> > > worker overhead decreases—say, by moving to IO worker threads—we won't
> > > be able to rely on this to keep the distance to an advantageous level.
> >
> > I don't see why lower overhead would prevent this from working?
>
> needed_wait has to be true to increase the readahead distance and for
> io_uring, when data was in the kernel buffer cache, needed_wait is
> false, meaning the distance doesn't increase. Worker mode didn't have
> this problem because of overhead. So needed_wait is true for workers.
> But, now that we will have combine_distance, I guess we don't need to
> rely on workers having overhead.
I think we still do, but that it will continue to work, even if the overhead
is much smaller than today. The workers will complete the IOs only after the
memory copy is finished (duh). Therefore, if the distance is too small to
allow workers to complete the copy, the distance will be increased, due to
needed_wait.
> So we are saying that readahead_distance is completely irrelevant for
> copying from the kernel buffer cache and only combine_distance matters for
> that now, right?
I don't think so! The combine_distance thing is crucial to allow for IO
combining, and, indirectly, for triggering the size based "async" heuristics
with io_uring. Once the io_uring async heuristic is triggered, the needed_wait
mechanism works to further increase the distance.
That does mean that for random BLCKSZ sized IOs that are in the page cache the
async mechanism won't typically be triggered - but from what I can tell that's
ok, because lots of 8kB IOs is also where the dispatch overhead to the kernel
threads doing the copying is the highest.
Greetings,
Andres Freund
view thread (23+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: AIO / read stream heuristics adjustments for index prefetching
In-Reply-To: <hxnsd6madv7em6gataql6fmjekfc5zthb3kp5w3e3mxjdonplf@spudt4jcmeep>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox