MIME-Version: 1.0
References: <CALj2ACW3cT5NPrBWzdmXgjw59M4xLL7jcL8C3xu9W3i7wdOyaw@mail.gmail.com>
 <20220316142658.GS10577@tamriel.snowman.net> <CAE9k0PmFePyata6MDom_5Fp10gD2pa-yguLGfF6fZx+c9Jzz+w@mail.gmail.com>
 <20220317.141838.2298774060802316987.horikyota.ntt@gmail.com>
In-Reply-To: <20220317.141838.2298774060802316987.horikyota.ntt@gmail.com>
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Thu, 17 Mar 2022 13:53:55 +0530
Message-ID: <CALj2ACVBST5Us6-eDz4q_Gem3rUHSC7AYNOB7tmp9Yqq6PHsXw@mail.gmail.com>
Subject: Re: pg_walinspect - a new extension to get raw WAL data and WAL stats
To: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: Ashutosh Sharma <ashu.coek88@gmail.com>, Stephen Frost <sfrost@snowman.net>, 
	Jeff Davis <pgsql@j-davis.com>, Robert Haas <robertmhaas@gmail.com>, 
	Andrew Dunstan <andrew@dunslane.net>, Greg Stark <stark@mit.edu>, 
	Jeremy Schneider <schneider@ardentperf.com>, Bruce Momjian <bruce@momjian.us>, 
	PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, 
	SATYANARAYANA NARLAPURAM <satyanarlapuram@gmail.com>, marvin_liang@qq.com, actyzhang@outlook.com
Content-Type: text/plain; charset="UTF-8"
Archived-At: <https://www.postgresql.org/message-id/CALj2ACVBST5Us6-eDz4q_Gem3rUHSC7AYNOB7tmp9Yqq6PHsXw%40mail.gmail.com>
Precedence: bulk

On Thu, Mar 17, 2022 at 10:48 AM Kyotaro Horiguchi
<horikyota.ntt@gmail.com> wrote:
>
> It still suggests unspecifiable end-LSN..
>
> > select * from pg_get_wal_records_info('4/4B28EB68', '4/4C000060');
> > ERROR:  cannot accept future end LSN
> > DETAIL:  Last known WAL LSN on the database system is 4/4C000060.

Thanks Kyotaro-san. We can change the detail message to show (current
flush lsn/last replayed lsn - 1), that's what I've done in v11 posted
upthread at [1]. The problem is that all the pg_walinspect functions
would wait for the first valid record in read_local_xlog_page() via
InitXLogReaderState()->XLogFindNextRecord(), see[2].

We have two things to do:
1) Just document the behaviour "pg_walinspect functions will wait for
the first valid WAL record if there is none found after the specified
input LSN/start LSN.". This seems easier but some may see it as a
problem.
2) Have read_local_xlog_page_2 which doesn't wait for future WAL LSN
unlike read_local_xlog_page and like pg_waldump's WALDumpReadPage. It
requires a new function read_local_xlog_page_2 that almost looks like
read_local_xlog_page except wait (pg_usleep) loop, we can avoid code
duplication by moving the read_local_xlog_page code to a static
function read_local_xlog_page_guts(existing params, bool wait):

read_local_xlog_page(params)
 read_local_xlog_page_guts(existing params, false);

read_local_xlog_page_2(params)
  read_local_xlog_page_guts(existing params, true);

read_local_xlog_page_guts:
  if (wait) wait for future wal; ---> existing pg_usleep code in
read_local_xlog_page.
  else return;

I'm fine either way, please let me know your thoughts on this?

[1] https://www.postgresql.org/message-id/CALj2ACU8XjbYbMwh5x6hEUJdpRoG9%3DPO52_tuOSf1%3DMO7WtsmQ%40mail.gmail.com
[2]
postgres=# select pg_current_wal_flush_lsn();
 pg_current_wal_flush_lsn
--------------------------
 0/1624430
(1 row)

postgres=# select * from pg_get_wal_record_info('0/1624430');
ERROR:  cannot accept future input LSN
DETAIL:  Last known WAL LSN on the database system is 0/162442F.
postgres=# select * from pg_get_wal_record_info('0/162442f');   --->
waits for the first valid record in read_local_xlog_page.

Regards,
Bharath Rupireddy.