public inbox for [email protected]  

help / color / mirror / Atom feed

From: Andres Freund <[email protected]>
To: [email protected]
Subject: pgsql: read_stream: Introduce and use optional batchmode support
Date: Sun, 30 Mar 2025 22:59:25 +0000
Message-ID: <[email protected]> (raw)

read_stream: Introduce and use optional batchmode support

Submitting IO in larger batches can be more efficient than doing so
one-by-one, particularly for many small reads. It does, however, require
the ReadStreamBlockNumberCB callback to abide by the restrictions of AIO
batching (c.f. pgaio_enter_batchmode()). Basically, the callback may not:
a) block without first calling pgaio_submit_staged(), unless a
   to-be-waited-on lock cannot be part of a deadlock, e.g. because it is
   never held while waiting for IO.

b) directly or indirectly start another batch pgaio_enter_batchmode()

As this requires care and is nontrivial in some cases, batching is only
used with explicit opt-in.

This patch adds an explicit flag (READ_STREAM_USE_BATCHING) to read_stream and
uses it where appropriate.

There are two cases where batching would likely be beneficial, but where we
aren't using it yet:

1) bitmap heap scans, because the callback reads the VM

   This should soon be solved, because we are planning to remove the use of
   the VM, due to that not being sound.

2) The first phase of heap vacuum

   This could be made to support batchmode, but would require some care.

Reviewed-by: Noah Misch <[email protected]>
Reviewed-by: Thomas Munro <[email protected]>
Discussion: https://postgr.es/m/uvrtrknj4kdytuboidbhwclo4gxhswwcpgadptsjvjqcluzmah%40brqs62irg4dt

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/ae3df4b34155d4a3811762e78d5076e64df66fba

Modified Files
--------------
contrib/amcheck/verify_heapam.c       | 13 ++++++++++++-
contrib/pg_prewarm/pg_prewarm.c       |  7 ++++++-
contrib/pg_visibility/pg_visibility.c |  8 +++++++-
src/backend/access/gist/gistvacuum.c  |  8 +++++++-
src/backend/access/heap/heapam.c      | 16 +++++++++++++++-
src/backend/access/heap/vacuumlazy.c  | 21 ++++++++++++++++++---
src/backend/access/nbtree/nbtree.c    |  8 +++++++-
src/backend/access/spgist/spgvacuum.c |  8 +++++++-
src/backend/commands/analyze.c        |  7 ++++++-
src/backend/storage/aio/read_stream.c | 16 ++++++++++++++++
src/backend/storage/buffer/bufmgr.c   |  8 +++++++-
src/include/storage/read_stream.h     | 21 +++++++++++++++++++++
12 files changed, 129 insertions(+), 12 deletions(-)

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected]
  Subject: Re: pgsql: read_stream: Introduce and use optional batchmode support
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox