public inbox for [email protected]  

help / color / mirror / Atom feed

From: Andres Freund <[email protected]>
To: Melanie Plageman <[email protected]>
Cc: [email protected]
Cc: Thomas Munro <[email protected]>
Cc: Peter Geoghegan <[email protected]>
Cc: Tomas Vondra <[email protected]>
Cc: Nazir Bilal Yavuz <[email protected]>
Subject: Re: AIO / read stream heuristics adjustments for index prefetching
Date: Wed, 1 Apr 2026 11:49:30 -0400
Message-ID: <kwzdd2tiow5ai25ehbrsoo6wmiokw5vckjfxle643k6dzskdv6@c2ti7opcwsiv> (raw)
In-Reply-To: <CAAKRu_aG2zvObX6gQ3Kw=MPydbphZ-qKeMGwZDEFe55FFiXYWw@mail.gmail.com>
References: <f3xxfrkafjxpyqxywcxricxgyizjirfceychyxsgn7bwjp5eda@kwbduhy7tfmu>
	<CAAKRu_aG2zvObX6gQ3Kw=MPydbphZ-qKeMGwZDEFe55FFiXYWw@mail.gmail.com>

Hi,

On 2026-04-01 10:52:03 -0400, Melanie Plageman wrote:
> On Tue, Mar 31, 2026 at 12:02 PM Andres Freund <[email protected]> wrote:
> >
> > 0008: WIP: read stream: Split decision about look ahead for AIO and combining
> >
> >     Until now read stream has used a single look-ahead distance to control
> >     lookahead for both IO combining and read-ahead. That's sub-optimal, as we
> >     want to do IO combining even when we don't need to do any readahead, as
> >     avoiding the syscall overhead is important to reduce CPU overhead when
> >     data is in the kernel page cache.
> >
> >     This is a prototype for what it could look like to split those
> >     decisions. Thereby fixing the regression mentioned in 0006.
>
> I wonder if we need to keep the combine_limit member in the read
> stream. Could we just use io_combine_limit without ramping up and
> down? This is mainly for code complexity reasons.

I thought so at first too, but it unfortunately leads to substantial
regressions with index prefetching, due to reading ahead unnecessarily far in
cases where we really just needed one block.


> Perhaps to allow fast path reentry, we could use distance_decay_holdoff == 0
> and ios_in_progress == 0 instead of combine_distance == 0.

Somewhat orthogonal: I really dislike the code to re-enter fastpath. I've now
broken it a few times without noticing. Especially when using a lower
distance, it's easy for the gating conditions to be fulfilled if
read_stream_look_ahead() decided to not *yet* do look ahead, because there's
still a pinned buffer and the distance is low.

ISTM that it really should only be checked after we did a lookahead and found
it to be a buffer hit.

Greetings,

Andres Freund

view thread (23+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
  Subject: Re: AIO / read stream heuristics adjustments for index prefetching
  In-Reply-To: <kwzdd2tiow5ai25ehbrsoo6wmiokw5vckjfxle643k6dzskdv6@c2ti7opcwsiv>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox