Subject: Use WALReadFromBuffers in more places

From: Bharath Rupireddy <[email-masked]>
Date: Wed, 24 Apr 2024 21:38:57 +0530

Hi,

Commit 91f2cae7a4e that introduced WALReadFromBuffers only used it for
physical walsenders. It can also be used in more places benefitting
logical walsenders, backends running pg_walinspect and logical
decoding functions if the WAL is available in WAL buffers. I'm
attaching a 0001 patch for this.

While at it, I've also added a test module in 0002 patch to
demonstrate 2 things: 1) how the caller can ensure the requested WAL
is fully copied to WAL buffers using WaitXLogInsertionsToFinish before
reading from WAL buffers. 2) how one can implement an xlogreader
page_read callback to read unflushed/not-yet-flushed WAL directly from
WAL buffers. FWIW, a separate test module to explicitly test the new
function is suggested here -
https://www.postgresql.org/message-id/CAFiTN-sE7CJn-ZFj%2B-0Wv6TNytv_fp4n%2BeCszspxJ3mt77t5ig%40mail.gmail.com.

Please have a look at the attached patches.

I will register this for the next commit fest.

-- 
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

---

From: Rahila Syed <[email-masked]>
Date: Fri, 19 Sep 2025 18:42:32 +0530

Hi Bharath,

Hi,
> attaching a 0001 patch for this.
Thank you for working on this. It seems like a useful optimization. Do you
have any information
on how much this improves the performance of the new callers of
WALReadFromBuffers?

Regarding the v3 version of the patch, do you also intend to include other
callers of WALRead,
such as walsummarizer.c and pg_waldump.c?

In WALReadFromBuffers, the buffer scan currently stops when it encounters a
buffer that
doesn't have the needed WAL page. However, it's possible to continue
scanning past the
missing page and find other relevant pages further along. By keeping track
of which pages
are missing, we could read only those specific pages from files, instead of
reading everything
after the first missing page. I am wondering if this was taken into account
during the design
of the function.

Thank you,
Rahila Syed