public inbox for [email protected]  
help / color / mirror / Atom feed
[PATCH v3 2/2] windows: Improve crash / assert / exception handling.
9+ messages / 4 participants
[nested] [flat]

* [PATCH v3 2/2] windows: Improve crash / assert / exception handling.
@ 2021-09-10 00:49  Andres Freund <[email protected]>
  0 siblings, 0 replies; 9+ messages in thread

From: Andres Freund @ 2021-09-10 00:49 UTC (permalink / raw)

---
 src/backend/main/main.c | 45 +++++++++++++++++++++++++++++++++++++++--
 .cirrus.yml             |  7 +++++++
 2 files changed, 50 insertions(+), 2 deletions(-)

diff --git a/src/backend/main/main.c b/src/backend/main/main.c
index ad84a45e28c..5115ad91898 100644
--- a/src/backend/main/main.c
+++ b/src/backend/main/main.c
@@ -26,6 +26,10 @@
 #include <sys/param.h>
 #endif
 
+#if defined(WIN32)
+#include <crtdbg.h>
+#endif
+
 #if defined(_M_AMD64) && _MSC_VER == 1800
 #include <math.h>
 #include <versionhelpers.h>
@@ -237,8 +241,45 @@ startup_hacks(const char *progname)
 			exit(1);
 		}
 
-		/* In case of general protection fault, don't show GUI popup box */
-		SetErrorMode(SEM_FAILCRITICALERRORS | SEM_NOGPFAULTERRORBOX);
+		/*
+		 * SEM_FAILCRITICALERRORS causes more errors to be reported to
+		 * callers.
+		 *
+		 * We used to also specify SEM_NOGPFAULTERRORBOX, but that prevents
+		 * windows crash reporting from working. Which includes registered
+		 * just-in-time debuggers. Now we try to disable sources of popups
+		 * separately below (note that SEM_NOGPFAULTERRORBOX didn't actually
+		 * prevent several sources of popups).
+		 */
+		SetErrorMode(SEM_FAILCRITICALERRORS);
+
+		/*
+		 * Show errors on stderr instead of popup box (note this doesn't
+		 * affect errors originating in the C runtime, see below).
+		 */
+		_set_error_mode(_OUT_TO_STDERR);
+
+
+		/*
+		 * By default abort() only generates a crash-dump in *non* debug
+		 * builds. As our Assert() / ExceptionalCondition() uses abort(),
+		 * leaving the default in place would make debugging harder.
+		 */
+#ifndef __MINGW64__
+		_set_abort_behavior(_CALL_REPORTFAULT | _WRITE_ABORT_MSG,
+							_CALL_REPORTFAULT | _WRITE_ABORT_MSG);
+#endif
+
+		/*
+		 * In case of errors (including assertions) on the C runtime level,
+		 * report them to stderr (and the debugger) instead of displaying a
+		 * popup. This is C runtime specific and thus the above incantations
+		 * aren't sufficient to suppress these popups.
+		 */
+		_CrtSetReportMode(_CRT_ASSERT, _CRTDBG_MODE_FILE | _CRTDBG_MODE_DEBUG);
+		_CrtSetReportMode(_CRT_ERROR, _CRTDBG_MODE_FILE | _CRTDBG_MODE_DEBUG);
+		_CrtSetReportFile(_CRT_ASSERT, _CRTDBG_FILE_STDERR);
+		_CrtSetReportFile(_CRT_ERROR, _CRTDBG_FILE_STDERR);
 
 #if defined(_M_AMD64) && _MSC_VER == 1800
 
diff --git a/.cirrus.yml b/.cirrus.yml
index 7a068650206..90c9c802d80 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -363,6 +363,13 @@ task:
     - cd src\tools\msvc
     - perl vcregress.pl ecpgcheck
 
+  always:
+    cores_script:
+      - cat crashlog.txt || true
+    dump_artifacts:
+      path: "crashlog.txt"
+      type: text/plain
+
   on_failure: *on_failure
 
 
-- 
2.34.0


--rbj3wvz2ukg3lhpx--





^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* [PATCH v6 1/2] windows: Improve crash / assert / exception handling.
@ 2021-09-10 00:49  Andres Freund <[email protected]>
  0 siblings, 0 replies; 9+ messages in thread

From: Andres Freund @ 2021-09-10 00:49 UTC (permalink / raw)

---
 src/backend/main/main.c | 48 +++++++++++++++++++++++++++++++++++++++--
 .cirrus.yml             |  6 +++++-
 2 files changed, 51 insertions(+), 3 deletions(-)

diff --git a/src/backend/main/main.c b/src/backend/main/main.c
index 9124060bde7..111e7867cc7 100644
--- a/src/backend/main/main.c
+++ b/src/backend/main/main.c
@@ -22,6 +22,10 @@
 
 #include <unistd.h>
 
+#if defined(WIN32)
+#include <crtdbg.h>
+#endif
+
 #if defined(__NetBSD__)
 #include <sys/param.h>
 #endif
@@ -237,8 +241,48 @@ startup_hacks(const char *progname)
 			exit(1);
 		}
 
-		/* In case of general protection fault, don't show GUI popup box */
-		SetErrorMode(SEM_FAILCRITICALERRORS | SEM_NOGPFAULTERRORBOX);
+		/*
+		 * By default abort() only generates a crash-dump in *non* debug
+		 * builds. As our Assert() / ExceptionalCondition() uses abort(),
+		 * leaving the default in place would make debugging harder.
+		 */
+#if !defined(__MINGW32__) && !defined(__MINGW64__)
+		_set_abort_behavior(_CALL_REPORTFAULT | _WRITE_ABORT_MSG,
+							_CALL_REPORTFAULT | _WRITE_ABORT_MSG);
+#endif /* MINGW */
+
+		/*
+		 * SEM_FAILCRITICALERRORS causes more errors to be reported to
+		 * callers.
+		 *
+		 * We used to also specify SEM_NOGPFAULTERRORBOX, but that prevents
+		 * windows crash reporting from working. Which includes registered
+		 * just-in-time debuggers. Now we try to disable sources of popups
+		 * separately below (note that SEM_NOGPFAULTERRORBOX didn't actually
+		 * prevent all sources of such popups).
+		 */
+		SetErrorMode(SEM_FAILCRITICALERRORS);
+
+		/*
+		 * Show errors on stderr instead of popup box (note this doesn't
+		 * affect errors originating in the C runtime, see below).
+		 */
+		_set_error_mode(_OUT_TO_STDERR);
+
+		/*
+		 * In DEBUG builds, errors, including assertions, C runtime errors are
+		 * reported via _CrtDbgReport. By default such errors are displayed
+		 * with a popup (even with NOGPFAULTERRORBOX), preventing forward
+		 * progress. Instead report such errors stderr (and the
+		 * debugger). This is C runtime specific and thus the above
+		 * incantations aren't sufficient to suppress these popups.
+		 */
+		_CrtSetReportMode(_CRT_ERROR, _CRTDBG_MODE_FILE | _CRTDBG_MODE_DEBUG);
+		_CrtSetReportFile(_CRT_ERROR, _CRTDBG_FILE_STDERR);
+		_CrtSetReportMode(_CRT_ASSERT, _CRTDBG_MODE_FILE | _CRTDBG_MODE_DEBUG);
+		_CrtSetReportFile(_CRT_ASSERT, _CRTDBG_FILE_STDERR);
+		_CrtSetReportMode(_CRT_WARN, _CRTDBG_MODE_FILE | _CRTDBG_MODE_DEBUG);
+		_CrtSetReportFile(_CRT_WARN, _CRTDBG_FILE_STDERR);
 
 #if defined(_M_AMD64) && _MSC_VER == 1800
 
diff --git a/.cirrus.yml b/.cirrus.yml
index 19b3737fa11..89fbff14abb 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -423,7 +423,11 @@ task:
     - cd src/tools/msvc
     - perl vcregress.pl ecpgcheck
 
-  on_failure: *on_failure
+  on_failure:
+    <<: *on_failure
+    crashlog_artifacts:
+      path: "crashlog-**.txt"
+      type: text/plain
 
 
 task:
-- 
2.34.0


--tzyf6sfbyi7t6jzz--





^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* [PATCH v2 2/2] windows: Improve crash / assert / exception handling.
@ 2021-09-10 00:49  Andres Freund <[email protected]>
  0 siblings, 0 replies; 9+ messages in thread

From: Andres Freund @ 2021-09-10 00:49 UTC (permalink / raw)

---
 .cirrus.yml             |  7 +++++++
 src/backend/main/main.c | 45 +++++++++++++++++++++++++++++++++++++++--
 2 files changed, 50 insertions(+), 2 deletions(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index b3321a91ae6..f75bdce6dec 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -337,6 +337,13 @@ task:
     - cd src\tools\msvc
     - perl vcregress.pl ecpgcheck
 
+  always:
+    cores_script:
+      - cat crashlog.txt || true
+    dump_artifacts:
+      path: "crashlog.txt"
+      type: text/plain
+
   on_failure:
     log_artifacts:
       path: "**/**.log"
diff --git a/src/backend/main/main.c b/src/backend/main/main.c
index ad84a45e28c..5115ad91898 100644
--- a/src/backend/main/main.c
+++ b/src/backend/main/main.c
@@ -26,6 +26,10 @@
 #include <sys/param.h>
 #endif
 
+#if defined(WIN32)
+#include <crtdbg.h>
+#endif
+
 #if defined(_M_AMD64) && _MSC_VER == 1800
 #include <math.h>
 #include <versionhelpers.h>
@@ -237,8 +241,45 @@ startup_hacks(const char *progname)
 			exit(1);
 		}
 
-		/* In case of general protection fault, don't show GUI popup box */
-		SetErrorMode(SEM_FAILCRITICALERRORS | SEM_NOGPFAULTERRORBOX);
+		/*
+		 * SEM_FAILCRITICALERRORS causes more errors to be reported to
+		 * callers.
+		 *
+		 * We used to also specify SEM_NOGPFAULTERRORBOX, but that prevents
+		 * windows crash reporting from working. Which includes registered
+		 * just-in-time debuggers. Now we try to disable sources of popups
+		 * separately below (note that SEM_NOGPFAULTERRORBOX didn't actually
+		 * prevent several sources of popups).
+		 */
+		SetErrorMode(SEM_FAILCRITICALERRORS);
+
+		/*
+		 * Show errors on stderr instead of popup box (note this doesn't
+		 * affect errors originating in the C runtime, see below).
+		 */
+		_set_error_mode(_OUT_TO_STDERR);
+
+
+		/*
+		 * By default abort() only generates a crash-dump in *non* debug
+		 * builds. As our Assert() / ExceptionalCondition() uses abort(),
+		 * leaving the default in place would make debugging harder.
+		 */
+#ifndef __MINGW64__
+		_set_abort_behavior(_CALL_REPORTFAULT | _WRITE_ABORT_MSG,
+							_CALL_REPORTFAULT | _WRITE_ABORT_MSG);
+#endif
+
+		/*
+		 * In case of errors (including assertions) on the C runtime level,
+		 * report them to stderr (and the debugger) instead of displaying a
+		 * popup. This is C runtime specific and thus the above incantations
+		 * aren't sufficient to suppress these popups.
+		 */
+		_CrtSetReportMode(_CRT_ASSERT, _CRTDBG_MODE_FILE | _CRTDBG_MODE_DEBUG);
+		_CrtSetReportMode(_CRT_ERROR, _CRTDBG_MODE_FILE | _CRTDBG_MODE_DEBUG);
+		_CrtSetReportFile(_CRT_ASSERT, _CRTDBG_FILE_STDERR);
+		_CrtSetReportFile(_CRT_ERROR, _CRTDBG_FILE_STDERR);
 
 #if defined(_M_AMD64) && _MSC_VER == 1800
 
-- 
2.23.0.385.gbc12974a89


--ep4fmvj5ysdv22gg--





^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* [PATCH v4 2/2] windows: Improve crash / assert / exception handling.
@ 2021-09-10 00:49  Andres Freund <[email protected]>
  0 siblings, 0 replies; 9+ messages in thread

From: Andres Freund @ 2021-09-10 00:49 UTC (permalink / raw)

---
 src/backend/main/main.c | 45 +++++++++++++++++++++++++++++++++++++++--
 .cirrus.yml             |  6 ++++++
 2 files changed, 49 insertions(+), 2 deletions(-)

diff --git a/src/backend/main/main.c b/src/backend/main/main.c
index ad84a45e28c..5115ad91898 100644
--- a/src/backend/main/main.c
+++ b/src/backend/main/main.c
@@ -26,6 +26,10 @@
 #include <sys/param.h>
 #endif
 
+#if defined(WIN32)
+#include <crtdbg.h>
+#endif
+
 #if defined(_M_AMD64) && _MSC_VER == 1800
 #include <math.h>
 #include <versionhelpers.h>
@@ -237,8 +241,45 @@ startup_hacks(const char *progname)
 			exit(1);
 		}
 
-		/* In case of general protection fault, don't show GUI popup box */
-		SetErrorMode(SEM_FAILCRITICALERRORS | SEM_NOGPFAULTERRORBOX);
+		/*
+		 * SEM_FAILCRITICALERRORS causes more errors to be reported to
+		 * callers.
+		 *
+		 * We used to also specify SEM_NOGPFAULTERRORBOX, but that prevents
+		 * windows crash reporting from working. Which includes registered
+		 * just-in-time debuggers. Now we try to disable sources of popups
+		 * separately below (note that SEM_NOGPFAULTERRORBOX didn't actually
+		 * prevent several sources of popups).
+		 */
+		SetErrorMode(SEM_FAILCRITICALERRORS);
+
+		/*
+		 * Show errors on stderr instead of popup box (note this doesn't
+		 * affect errors originating in the C runtime, see below).
+		 */
+		_set_error_mode(_OUT_TO_STDERR);
+
+
+		/*
+		 * By default abort() only generates a crash-dump in *non* debug
+		 * builds. As our Assert() / ExceptionalCondition() uses abort(),
+		 * leaving the default in place would make debugging harder.
+		 */
+#ifndef __MINGW64__
+		_set_abort_behavior(_CALL_REPORTFAULT | _WRITE_ABORT_MSG,
+							_CALL_REPORTFAULT | _WRITE_ABORT_MSG);
+#endif
+
+		/*
+		 * In case of errors (including assertions) on the C runtime level,
+		 * report them to stderr (and the debugger) instead of displaying a
+		 * popup. This is C runtime specific and thus the above incantations
+		 * aren't sufficient to suppress these popups.
+		 */
+		_CrtSetReportMode(_CRT_ASSERT, _CRTDBG_MODE_FILE | _CRTDBG_MODE_DEBUG);
+		_CrtSetReportMode(_CRT_ERROR, _CRTDBG_MODE_FILE | _CRTDBG_MODE_DEBUG);
+		_CrtSetReportFile(_CRT_ASSERT, _CRTDBG_FILE_STDERR);
+		_CrtSetReportFile(_CRT_ERROR, _CRTDBG_FILE_STDERR);
 
 #if defined(_M_AMD64) && _MSC_VER == 1800
 
diff --git a/.cirrus.yml b/.cirrus.yml
index 4116e0155dc..3ffb3f3d67f 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -415,6 +415,12 @@ task:
     - cd src/tools/msvc
     - perl vcregress.pl ecpgcheck
 
+  always:
+    cores_script: cat crashlog.txt || true
+    dump_artifacts:
+      path: "crashlog.txt"
+      type: text/plain
+
   on_failure: *on_failure
 
 
-- 
2.34.0


--mezqkkzp3cvymuis--





^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: Add pg_walinspect function with block info columns
@ 2023-03-07 05:44  Michael Paquier <[email protected]>
  0 siblings, 1 reply; 9+ messages in thread

From: Michael Paquier @ 2023-03-07 05:44 UTC (permalink / raw)
  To: Kyotaro Horiguchi <[email protected]>; +Cc: [email protected]; [email protected]; [email protected]; pgsql-hackers; [email protected]

On Tue, Mar 07, 2023 at 11:17:45AM +0900, Kyotaro Horiguchi wrote:
> Thus I'm inclined to agree with Michael's suggestion of creating a new
> normalized set-returning function that returns information of
> "blocks".

Just to be clear here, I am not suggesting to add a new function for
only the block information, just a rename of the existing
pg_get_wal_fpi_info() to something like pg_get_wal_block_info() that
includes both the FPI (if any or NULL if none) and the block data (if
any or NULL is none) so as all of them are governed by the same lookup
at pg_wal/.  The fpi information (aka compression type) is displayed
if there is a FPI in the block.
--
Michael


Attachments:

  [application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
  download

^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: Add pg_walinspect function with block info columns
@ 2023-03-07 06:49  Kyotaro Horiguchi <[email protected]>
  parent: Michael Paquier <[email protected]>
  0 siblings, 1 reply; 9+ messages in thread

From: Kyotaro Horiguchi @ 2023-03-07 06:49 UTC (permalink / raw)
  To: [email protected]; +Cc: [email protected]; [email protected]; [email protected]; pgsql-hackers; [email protected]

At Tue, 7 Mar 2023 14:44:49 +0900, Michael Paquier <[email protected]> wrote in 
> On Tue, Mar 07, 2023 at 11:17:45AM +0900, Kyotaro Horiguchi wrote:
> > Thus I'm inclined to agree with Michael's suggestion of creating a new
> > normalized set-returning function that returns information of
> > "blocks".
> 
> Just to be clear here, I am not suggesting to add a new function for
> only the block information, just a rename of the existing
> pg_get_wal_fpi_info() to something like pg_get_wal_block_info() that
> includes both the FPI (if any or NULL if none) and the block data (if
> any or NULL is none) so as all of them are governed by the same lookup
> at pg_wal/.  The fpi information (aka compression type) is displayed
> if there is a FPI in the block.

Ah. Yes, that expansion sounds sensible.

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center






^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: Add pg_walinspect function with block info columns
@ 2023-03-07 07:18  Michael Paquier <[email protected]>
  parent: Kyotaro Horiguchi <[email protected]>
  0 siblings, 2 replies; 9+ messages in thread

From: Michael Paquier @ 2023-03-07 07:18 UTC (permalink / raw)
  To: Kyotaro Horiguchi <[email protected]>; +Cc: [email protected]; [email protected]; [email protected]; pgsql-hackers; [email protected]

On Tue, Mar 07, 2023 at 03:49:02PM +0900, Kyotaro Horiguchi wrote:
> Ah. Yes, that expansion sounds sensible.

Okay, so, based on this idea, I have hacked on this stuff and finish
with the attached that shows block data if it exists, as well as FPI
stuff if any.  bimg_info is showed as a text[] for its flags.

I guess that I'd better add a test that shows correctly a record with
some block data attached to it, on top of the existing one for FPIs..
Any suggestions?  Perhaps just a heap/heap2 record?

Thoughts?
--
Michael


Attachments:

  [text/x-diff] 0001-Rework-pg_walinspect-to-retrieve-more-block-informat.patch (18.2K, ../../[email protected]/2-0001-Rework-pg_walinspect-to-retrieve-more-block-informat.patch)
  download | inline diff:
From 237be9d768c20f9c8fec0a11a8fcbee6c4e9bf8a Mon Sep 17 00:00:00 2001
From: Michael Paquier <[email protected]>
Date: Tue, 7 Mar 2023 16:06:28 +0900
Subject: [PATCH] Rework pg_walinspect to retrieve more block information

---
 doc/src/sgml/pgwalinspect.sgml                |  36 +++--
 .../pg_walinspect/expected/pg_walinspect.out  |  16 +--
 .../pg_walinspect/pg_walinspect--1.0--1.1.sql |  16 ++-
 contrib/pg_walinspect/pg_walinspect.c         | 134 +++++++++++++-----
 contrib/pg_walinspect/sql/pg_walinspect.sql   |  16 +--
 5 files changed, 145 insertions(+), 73 deletions(-)

diff --git a/doc/src/sgml/pgwalinspect.sgml b/doc/src/sgml/pgwalinspect.sgml
index 3d7cdb95cc..3a65beb08d 100644
--- a/doc/src/sgml/pgwalinspect.sgml
+++ b/doc/src/sgml/pgwalinspect.sgml
@@ -190,31 +190,39 @@ combined_size_percentage     | 2.8634072910530795
 
    <varlistentry>
     <term>
-     <function>pg_get_wal_fpi_info(start_lsn pg_lsn, end_lsn pg_lsn) returns setof record</function>
+     <function>pg_get_wal_block_info(start_lsn pg_lsn, end_lsn pg_lsn) returns setof record</function>
     </term>
 
     <listitem>
      <para>
-      Gets a copy of full page images as <type>bytea</type> values (after
-      applying decompression when necessary) and their information associated
-      with all the valid WAL records between
+      Gets a copy of the block information stored in WAL records. This includes
+      copies of the block data (<literal>NULL</literal> if none) and full page
+      images as <type>bytea</type> values (after
+      applying decompression when necessary, or <literal>NULL</literal> if none)
+      and their information associated with all the valid WAL records between
       <replaceable>start_lsn</replaceable> and
-      <replaceable>end_lsn</replaceable>. Returns one row per full page image.
-      If <replaceable>start_lsn</replaceable> or
+      <replaceable>end_lsn</replaceable>. Returns one row per block registered
+      in a WAL record. If <replaceable>start_lsn</replaceable> or
       <replaceable>end_lsn</replaceable> are not yet available, the function
       will raise an error. For example:
 <screen>
-postgres=# SELECT lsn, reltablespace, reldatabase, relfilenode, relblocknumber,
-                  forkname, substring(fpi for 24) as fpi_trimmed
-             FROM pg_get_wal_fpi_info('0/1801690', '0/1825C60');
+postgres=# SELECT lsn, blockid, reltablespace, reldatabase, relfilenode,
+                  relblocknumber, forkname,
+                  substring(blockdata for 24) as block_trimmed,
+                  substring(fpi for 24) as fpi_trimmed, fpilen, fpiinfo
+             FROM pg_get_wal_fpi_info('0/20BC498', '0/20BCD80');
 -[ RECORD 1 ]--+---------------------------------------------------
-lsn            | 0/1807E20
+lsn            | 0/20BC498
+blockid        | 0
 reltablespace  | 1663
-reldatabase    | 5
-relfilenode    | 16396
-relblocknumber | 43
+reldatabase    | 16384
+relfilenode    | 24576
+relblocknumber | 44
 forkname       | main
-fpi_trimmed    | \x00000000b89e660100000000a003c0030020042000000000
+block_trimmed  | null
+fpi_trimmed    | \x0000000018ed0a02000000000401a0180020042000000000
+fpilen         | 2148
+fpiinfo        | {HAS_HOLE,APPLY}
 </screen>
      </para>
     </listitem>
diff --git a/contrib/pg_walinspect/expected/pg_walinspect.out b/contrib/pg_walinspect/expected/pg_walinspect.out
index 9bcb05354e..b1682582e8 100644
--- a/contrib/pg_walinspect/expected/pg_walinspect.out
+++ b/contrib/pg_walinspect/expected/pg_walinspect.out
@@ -74,7 +74,7 @@ SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2'
 (1 row)
 
 -- ===================================================================
--- Tests to get full page image (FPI) from WAL record
+-- Tests to get block information from WAL record
 -- ===================================================================
 SELECT pg_current_wal_lsn() AS wal_lsn3 \gset
 -- Force FPI on the next update.
@@ -83,8 +83,8 @@ CHECKPOINT;
 UPDATE sample_tbl SET col1 = col1 * 100 WHERE col1 = 1;
 SELECT pg_current_wal_lsn() AS wal_lsn4 \gset
 -- Check if we get FPI from WAL record.
-SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn3', :'wal_lsn4')
-  WHERE relfilenode = :'sample_tbl_oid';
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn3', :'wal_lsn4')
+  WHERE relfilenode = :'sample_tbl_oid' AND fpi IS NOT NULL;
  ok 
 ----
  t
@@ -116,7 +116,7 @@ SELECT has_function_privilege('regress_pg_walinspect',
 (1 row)
 
 SELECT has_function_privilege('regress_pg_walinspect',
-  'pg_get_wal_fpi_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- no
+  'pg_get_wal_block_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- no
  has_function_privilege 
 ------------------------
  f
@@ -146,7 +146,7 @@ SELECT has_function_privilege('regress_pg_walinspect',
 (1 row)
 
 SELECT has_function_privilege('regress_pg_walinspect',
-  'pg_get_wal_fpi_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- yes
+  'pg_get_wal_block_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- yes
  has_function_privilege 
 ------------------------
  t
@@ -160,7 +160,7 @@ GRANT EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn)
   TO regress_pg_walinspect;
 GRANT EXECUTE ON FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean)
   TO regress_pg_walinspect;
-GRANT EXECUTE ON FUNCTION pg_get_wal_fpi_info(pg_lsn, pg_lsn)
+GRANT EXECUTE ON FUNCTION pg_get_wal_block_info(pg_lsn, pg_lsn)
   TO regress_pg_walinspect;
 SELECT has_function_privilege('regress_pg_walinspect',
   'pg_get_wal_record_info(pg_lsn)', 'EXECUTE'); -- yes
@@ -184,7 +184,7 @@ SELECT has_function_privilege('regress_pg_walinspect',
 (1 row)
 
 SELECT has_function_privilege('regress_pg_walinspect',
-  'pg_get_wal_fpi_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- yes
+  'pg_get_wal_block_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- yes
  has_function_privilege 
 ------------------------
  t
@@ -196,7 +196,7 @@ REVOKE EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn)
   FROM regress_pg_walinspect;
 REVOKE EXECUTE ON FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean)
   FROM regress_pg_walinspect;
-REVOKE EXECUTE ON FUNCTION pg_get_wal_fpi_info(pg_lsn, pg_lsn)
+REVOKE EXECUTE ON FUNCTION pg_get_wal_block_info(pg_lsn, pg_lsn)
   FROM regress_pg_walinspect;
 -- ===================================================================
 -- Clean up
diff --git a/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql b/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
index 1e9e1e6115..f6f9c00281 100644
--- a/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
+++ b/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
@@ -4,21 +4,25 @@
 \echo Use "ALTER EXTENSION pg_walinspect UPDATE TO '1.1'" to load this file. \quit
 
 --
--- pg_get_wal_fpi_info()
+-- pg_get_wal_block_info()
 --
-CREATE FUNCTION pg_get_wal_fpi_info(IN start_lsn pg_lsn,
+CREATE FUNCTION pg_get_wal_block_info(IN start_lsn pg_lsn,
 	IN end_lsn pg_lsn,
 	OUT lsn pg_lsn,
+	OUT blockid int2,
 	OUT reltablespace oid,
 	OUT reldatabase oid,
 	OUT relfilenode oid,
 	OUT relblocknumber int8,
 	OUT forkname text,
-	OUT fpi bytea
+	OUT blockdata bytea,
+	OUT fpi bytea,
+	OUT fpilen int,
+	OUT fpiinfo text[]
 )
 RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'pg_get_wal_fpi_info'
+AS 'MODULE_PATHNAME', 'pg_get_wal_block_info'
 LANGUAGE C STRICT PARALLEL SAFE;
 
-REVOKE EXECUTE ON FUNCTION pg_get_wal_fpi_info(pg_lsn, pg_lsn) FROM PUBLIC;
-GRANT EXECUTE ON FUNCTION pg_get_wal_fpi_info(pg_lsn, pg_lsn) TO pg_read_server_files;
+REVOKE EXECUTE ON FUNCTION pg_get_wal_block_info(pg_lsn, pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_wal_block_info(pg_lsn, pg_lsn) TO pg_read_server_files;
diff --git a/contrib/pg_walinspect/pg_walinspect.c b/contrib/pg_walinspect/pg_walinspect.c
index b7b0a805ee..7cc3260022 100644
--- a/contrib/pg_walinspect/pg_walinspect.c
+++ b/contrib/pg_walinspect/pg_walinspect.c
@@ -20,6 +20,7 @@
 #include "access/xlogutils.h"
 #include "funcapi.h"
 #include "miscadmin.h"
+#include "utils/array.h"
 #include "utils/builtins.h"
 #include "utils/pg_lsn.h"
 
@@ -30,7 +31,7 @@
 
 PG_MODULE_MAGIC;
 
-PG_FUNCTION_INFO_V1(pg_get_wal_fpi_info);
+PG_FUNCTION_INFO_V1(pg_get_wal_block_info);
 PG_FUNCTION_INFO_V1(pg_get_wal_record_info);
 PG_FUNCTION_INFO_V1(pg_get_wal_records_info);
 PG_FUNCTION_INFO_V1(pg_get_wal_records_info_till_end_of_wal);
@@ -56,7 +57,7 @@ static void FillXLogStatsRow(const char *name, uint64 n, uint64 total_count,
 							 Datum *values, bool *nulls, uint32 ncols);
 static void GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
 						XLogRecPtr end_lsn, bool stats_per_record);
-static void GetWALFPIInfo(FunctionCallInfo fcinfo, XLogReaderState *record);
+static void GetWALBlockInfo(FunctionCallInfo fcinfo, XLogReaderState *record);
 
 /*
  * Check if the given LSN is in future. Also, return the LSN up to which the
@@ -221,49 +222,41 @@ GetWALRecordInfo(XLogReaderState *record, Datum *values,
 
 
 /*
- * Store a set of full page images from a single record.
+ * Store a set of block information from a single record (FPI and block
+ * information).
  */
 static void
-GetWALFPIInfo(FunctionCallInfo fcinfo, XLogReaderState *record)
+GetWALBlockInfo(FunctionCallInfo fcinfo, XLogReaderState *record)
 {
-#define PG_GET_WAL_FPI_INFO_COLS 7
+#define PG_GET_WAL_BLOCK_INFO_COLS 11
 	int			block_id;
 	ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
 
 	for (block_id = 0; block_id <= XLogRecMaxBlockId(record); block_id++)
 	{
-		PGAlignedBlock buf;
-		Page		page;
-		bytea	   *raw_page;
+		DecodedBkpBlock *block;
 		BlockNumber blk;
 		RelFileLocator rnode;
 		ForkNumber	fork;
-		Datum		values[PG_GET_WAL_FPI_INFO_COLS] = {0};
-		bool		nulls[PG_GET_WAL_FPI_INFO_COLS] = {0};
+		Datum		values[PG_GET_WAL_BLOCK_INFO_COLS] = {0};
+		bool		nulls[PG_GET_WAL_BLOCK_INFO_COLS] = {0};
 		int			i = 0;
 
 		if (!XLogRecHasBlockRef(record, block_id))
 			continue;
 
-		if (!XLogRecHasBlockImage(record, block_id))
-			continue;
-
-		page = (Page) buf.data;
-
-		if (!RestoreBlockImage(record, block_id, page))
-			ereport(ERROR,
-					(errcode(ERRCODE_INTERNAL_ERROR),
-					 errmsg_internal("%s", record->errormsg_buf)));
+		block = XLogRecGetBlock(record, block_id);
 
 		/* Full page exists, so let's save it. */
 		(void) XLogRecGetBlockTagExtended(record, block_id,
 										  &rnode, &fork, &blk, NULL);
 
 		values[i++] = LSNGetDatum(record->ReadRecPtr);
-		values[i++] = ObjectIdGetDatum(rnode.spcOid);
-		values[i++] = ObjectIdGetDatum(rnode.dbOid);
-		values[i++] = ObjectIdGetDatum(rnode.relNumber);
-		values[i++] = Int64GetDatum((int64) blk);
+		values[i++] = Int16GetDatum(block_id);
+		values[i++] = ObjectIdGetDatum(block->rlocator.spcOid);
+		values[i++] = ObjectIdGetDatum(block->rlocator.dbOid);
+		values[i++] = ObjectIdGetDatum(block->rlocator.relNumber);
+		values[i++] = Int64GetDatum((int64) block->blkno);
 
 		if (fork >= 0 && fork <= MAX_FORKNUM)
 			values[i++] = CStringGetTextDatum(forkNames[fork]);
@@ -272,34 +265,101 @@ GetWALFPIInfo(FunctionCallInfo fcinfo, XLogReaderState *record)
 					(errcode(ERRCODE_INTERNAL_ERROR),
 					 errmsg_internal("invalid fork number: %u", fork)));
 
-		/* Initialize bytea buffer to copy the FPI to. */
-		raw_page = (bytea *) palloc(BLCKSZ + VARHDRSZ);
-		SET_VARSIZE(raw_page, BLCKSZ + VARHDRSZ);
+		/* Block data */
+		if (block->has_data)
+		{
+			bytea	   *raw_data;
 
-		/* Take a verbatim copy of the FPI. */
-		memcpy(VARDATA(raw_page), page, BLCKSZ);
+			/* Initialize bytea buffer to copy the data to. */
+			raw_data = (bytea *) palloc(block->data_len + VARHDRSZ);
+			SET_VARSIZE(raw_data, block->data_len + VARHDRSZ);
 
-		values[i++] = PointerGetDatum(raw_page);
+			/* Copy the data */
+			memcpy(VARDATA(raw_data), block->data, block->data_len);
+			values[i++] = PointerGetDatum(raw_data);
+		}
+		else
+		{
+			/* no data, so set this field to NULL */
+			nulls[i++] = true;
+		}
 
-		Assert(i == PG_GET_WAL_FPI_INFO_COLS);
+		/* Full-page image */
+		if (block->has_image)
+		{
+			PGAlignedBlock buf;
+			Page		page;
+			bytea	   *raw_page;
+			int			bitcnt;
+			int			cnt = 0;
+			Datum	   *flags;
+			ArrayType  *a;
+
+			page = (Page) buf.data;
+
+			if (!RestoreBlockImage(record, block_id, page))
+				ereport(ERROR,
+						(errcode(ERRCODE_INTERNAL_ERROR),
+						 errmsg_internal("%s", record->errormsg_buf)));
+
+			/* Initialize bytea buffer to copy the FPI to. */
+			raw_page = (bytea *) palloc(BLCKSZ + VARHDRSZ);
+			SET_VARSIZE(raw_page, BLCKSZ + VARHDRSZ);
+
+			/* Take a verbatim copy of the FPI. */
+			memcpy(VARDATA(raw_page), page, BLCKSZ);
+
+			values[i++] = PointerGetDatum(raw_page);
+			values[i++] = Int32GetDatum(block->bimg_len);
+
+			/* FPI flags */
+			bitcnt = pg_popcount((const char *) &block->bimg_info,
+								 sizeof(uint8));
+			/* build set of raw flags */
+			flags = (Datum *) palloc0(sizeof(Datum) * bitcnt);
+			if ((block->bimg_info & BKPIMAGE_HAS_HOLE) != 0)
+				flags[cnt++] = CStringGetTextDatum("HAS_HOLE");
+			if ((block->bimg_info & BKPIMAGE_APPLY) != 0)
+				flags[cnt++] = CStringGetTextDatum("APPLY");
+			if ((block->bimg_info & BKPIMAGE_COMPRESS_PGLZ) != 0)
+				flags[cnt++] = CStringGetTextDatum("COMPRESS_PGLZ");
+			if ((block->bimg_info & BKPIMAGE_COMPRESS_LZ4) != 0)
+				flags[cnt++] = CStringGetTextDatum("COMPRESS_LZ4");
+			if ((block->bimg_info & BKPIMAGE_COMPRESS_ZSTD) != 0)
+				flags[cnt++] = CStringGetTextDatum("COMPRESS_ZSTD");
+
+			Assert(cnt <= bitcnt);
+			a = construct_array_builtin(flags, cnt, TEXTOID);
+			values[i++] = PointerGetDatum(a);
+		}
+		else
+		{
+			/* No full page image, so store NULLs for all its fields */
+			memset(&nulls[i], true, 3 * sizeof(bool));
+			i += 3;
+		}
+
+		Assert(i == PG_GET_WAL_BLOCK_INFO_COLS);
 
 		tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc,
 							 values, nulls);
 	}
 
-#undef PG_GET_WAL_FPI_INFO_COLS
+#undef PG_GET_WAL_FPI_BLOCK_COLS
 }
 
 /*
- * Get full page images with their relation information for all the WAL
- * records between start and end LSNs.  Decompression is applied to the
- * blocks, if necessary.
+ * Get information about all the blocks saved in WAL records between start
+ * and end LSNs.  This produces information about the full page images with
+ * their relation information, and the data saved in each block associated
+ * to a record.  Decompression is applied to the full-page images, if
+ * necessary.
  *
  * This function emits an error if a future start or end WAL LSN i.e. WAL LSN
  * the database system doesn't know about is specified.
  */
 Datum
-pg_get_wal_fpi_info(PG_FUNCTION_ARGS)
+pg_get_wal_block_info(PG_FUNCTION_ARGS)
 {
 	XLogRecPtr	start_lsn;
 	XLogRecPtr	end_lsn;
@@ -317,7 +377,7 @@ pg_get_wal_fpi_info(PG_FUNCTION_ARGS)
 	xlogreader = InitXLogReaderState(start_lsn);
 
 	tmp_cxt = AllocSetContextCreate(CurrentMemoryContext,
-									"pg_get_wal_fpi_info temporary cxt",
+									"pg_get_wal_block_info temporary cxt",
 									ALLOCSET_DEFAULT_SIZES);
 
 	while (ReadNextXLogRecord(xlogreader) &&
@@ -326,7 +386,7 @@ pg_get_wal_fpi_info(PG_FUNCTION_ARGS)
 		/* Use the tmp context so we can clean up after each tuple is done */
 		old_cxt = MemoryContextSwitchTo(tmp_cxt);
 
-		GetWALFPIInfo(fcinfo, xlogreader);
+		GetWALBlockInfo(fcinfo, xlogreader);
 
 		/* clean up and switch back */
 		MemoryContextSwitchTo(old_cxt);
diff --git a/contrib/pg_walinspect/sql/pg_walinspect.sql b/contrib/pg_walinspect/sql/pg_walinspect.sql
index 849201a1f8..10988b4782 100644
--- a/contrib/pg_walinspect/sql/pg_walinspect.sql
+++ b/contrib/pg_walinspect/sql/pg_walinspect.sql
@@ -53,7 +53,7 @@ SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2'
 			WHERE resource_manager = 'Heap' AND record_type = 'INSERT';
 
 -- ===================================================================
--- Tests to get full page image (FPI) from WAL record
+-- Tests to get block information from WAL record
 -- ===================================================================
 SELECT pg_current_wal_lsn() AS wal_lsn3 \gset
 
@@ -65,8 +65,8 @@ UPDATE sample_tbl SET col1 = col1 * 100 WHERE col1 = 1;
 SELECT pg_current_wal_lsn() AS wal_lsn4 \gset
 
 -- Check if we get FPI from WAL record.
-SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn3', :'wal_lsn4')
-  WHERE relfilenode = :'sample_tbl_oid';
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn3', :'wal_lsn4')
+  WHERE relfilenode = :'sample_tbl_oid' AND fpi IS NOT NULL;
 
 -- ===================================================================
 -- Tests for permissions
@@ -83,7 +83,7 @@ SELECT has_function_privilege('regress_pg_walinspect',
   'pg_get_wal_stats(pg_lsn, pg_lsn, boolean) ', 'EXECUTE'); -- no
 
 SELECT has_function_privilege('regress_pg_walinspect',
-  'pg_get_wal_fpi_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- no
+  'pg_get_wal_block_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- no
 
 -- Functions accessible by users with role pg_read_server_files
 
@@ -99,7 +99,7 @@ SELECT has_function_privilege('regress_pg_walinspect',
   'pg_get_wal_stats(pg_lsn, pg_lsn, boolean) ', 'EXECUTE'); -- yes
 
 SELECT has_function_privilege('regress_pg_walinspect',
-  'pg_get_wal_fpi_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- yes
+  'pg_get_wal_block_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- yes
 
 REVOKE pg_read_server_files FROM regress_pg_walinspect;
 
@@ -113,7 +113,7 @@ GRANT EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn)
 GRANT EXECUTE ON FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean)
   TO regress_pg_walinspect;
 
-GRANT EXECUTE ON FUNCTION pg_get_wal_fpi_info(pg_lsn, pg_lsn)
+GRANT EXECUTE ON FUNCTION pg_get_wal_block_info(pg_lsn, pg_lsn)
   TO regress_pg_walinspect;
 
 SELECT has_function_privilege('regress_pg_walinspect',
@@ -126,7 +126,7 @@ SELECT has_function_privilege('regress_pg_walinspect',
   'pg_get_wal_stats(pg_lsn, pg_lsn, boolean) ', 'EXECUTE'); -- yes
 
 SELECT has_function_privilege('regress_pg_walinspect',
-  'pg_get_wal_fpi_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- yes
+  'pg_get_wal_block_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- yes
 
 REVOKE EXECUTE ON FUNCTION pg_get_wal_record_info(pg_lsn)
   FROM regress_pg_walinspect;
@@ -137,7 +137,7 @@ REVOKE EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn)
 REVOKE EXECUTE ON FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean)
   FROM regress_pg_walinspect;
 
-REVOKE EXECUTE ON FUNCTION pg_get_wal_fpi_info(pg_lsn, pg_lsn)
+REVOKE EXECUTE ON FUNCTION pg_get_wal_block_info(pg_lsn, pg_lsn)
   FROM regress_pg_walinspect;
 
 -- ===================================================================
-- 
2.39.2



  [application/pgp-signature] signature.asc (833B, ../../[email protected]/3-signature.asc)
  download

^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: Add pg_walinspect function with block info columns
@ 2023-03-07 09:07  Kyotaro Horiguchi <[email protected]>
  parent: Michael Paquier <[email protected]>
  1 sibling, 0 replies; 9+ messages in thread

From: Kyotaro Horiguchi @ 2023-03-07 09:07 UTC (permalink / raw)
  To: [email protected]; +Cc: [email protected]; [email protected]; [email protected]; pgsql-hackers; [email protected]

At Tue, 7 Mar 2023 16:18:21 +0900, Michael Paquier <[email protected]> wrote in 
> On Tue, Mar 07, 2023 at 03:49:02PM +0900, Kyotaro Horiguchi wrote:
> > Ah. Yes, that expansion sounds sensible.
> 
> Okay, so, based on this idea, I have hacked on this stuff and finish
> with the attached that shows block data if it exists, as well as FPI
> stuff if any.  bimg_info is showed as a text[] for its flags.

# The naming convetion looks inconsistent between
# pg_get_wal_records_info and pg_get_wal_block_info but it's not an
# issue of this patch..

> I guess that I'd better add a test that shows correctly a record with
> some block data attached to it, on top of the existing one for FPIs..
> Any suggestions?  Perhaps just a heap/heap2 record?
> 
> Thoughts?

I thought that we needed a test for block data when I saw the patch.
I don't have great idea but a single insert should work.

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center






^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: Add pg_walinspect function with block info columns
@ 2023-03-07 10:26  Bharath Rupireddy <[email protected]>
  parent: Michael Paquier <[email protected]>
  1 sibling, 0 replies; 9+ messages in thread

From: Bharath Rupireddy @ 2023-03-07 10:26 UTC (permalink / raw)
  To: Michael Paquier <[email protected]>; +Cc: Kyotaro Horiguchi <[email protected]>; [email protected]; [email protected]; pgsql-hackers; [email protected]

On Tue, Mar 7, 2023 at 12:48 PM Michael Paquier <[email protected]> wrote:
>
> On Tue, Mar 07, 2023 at 03:49:02PM +0900, Kyotaro Horiguchi wrote:
> > Ah. Yes, that expansion sounds sensible.
>
> Okay, so, based on this idea, I have hacked on this stuff and finish
> with the attached that shows block data if it exists, as well as FPI
> stuff if any.  bimg_info is showed as a text[] for its flags.

+1.

> I guess that I'd better add a test that shows correctly a record with
> some block data attached to it, on top of the existing one for FPIs..
> Any suggestions?  Perhaps just a heap/heap2 record?
>
> Thoughts?

That would be a lot better. Not just the test, but also the
documentation can have it. Simple way to generate such a record (both
block data and FPI) is to just change the wal_level to logical in
walinspect.conf [1], see code around REGBUF_KEEP_DATA and
RelationIsLogicallyLogged in heapam.c

I had the following comments and fixed them in the attached v2 patch:

1. Still a trace of pg_get_wal_fpi_info in docs, removed it.

2. Used int4 instead of int for fpilen just to be in sync with
fpi_length of pg_get_wal_record_info.

3. Changed to be consistent and use just FPI or "F/full page".
            /* FPI flags */
            /* No full page image, so store NULLs for all its fields */
        /* Full-page image */
        /* Full page exists, so let's save it. */
 * and end LSNs.  This produces information about the full page images with
 * to a record.  Decompression is applied to the full-page images, if

4. I think we need to free raw_data, raw_page and flags as we loop
over multiple blocks (XLR_MAX_BLOCK_ID) and will leak memory which can
be a problem if we have many blocks assocated with a single WAL
record.
            flags = (Datum *) palloc0(sizeof(Datum) * bitcnt);
Also, we will leak all CStringGetTextDatum memory in the block_id for loop.
Another way is to use and reset temp memory context in the for loop
over block_ids. I prefer this approach over multiple pfree()s in
block_id for loop.

5. I think it'd be good to say if the FPI is for WAL_VERIFICATION, so
I changed it to the following. Feel free to ignore this if you think
it's not required.
            if (blk->apply_image)
                flags[cnt++] = CStringGetTextDatum("APPLY");
            else
                flags[cnt++] = CStringGetTextDatum("WAL_VERIFICATION");

6. Did minor wordsmithing.

7. Added test case which shows both block data and fpi in the documentation.

8. Changed wal_level to logical in walinspect.conf to test case with block data.

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com


Attachments:

  [application/octet-stream] v2-0001-Rework-pg_walinspect-to-retrieve-more-block-infor.patch (20.1K, ../../CALj2ACVoowu=14F9uLSPfT5p0tJRtssAN47p8Kpq55SLPT=ZxQ@mail.gmail.com/2-v2-0001-Rework-pg_walinspect-to-retrieve-more-block-infor.patch)
  download | inline diff:
From 1472b08f64acb398e2c9bf830939719ca8d722ea Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <[email protected]>
Date: Tue, 7 Mar 2023 09:41:22 +0000
Subject: [PATCH v2] Rework pg_walinspect to retrieve more block information

---
 .../pg_walinspect/expected/pg_walinspect.out  |  24 ++-
 .../pg_walinspect/pg_walinspect--1.0--1.1.sql |  16 +-
 contrib/pg_walinspect/pg_walinspect.c         | 168 ++++++++++++------
 contrib/pg_walinspect/sql/pg_walinspect.sql   |  20 ++-
 contrib/pg_walinspect/walinspect.conf         |   2 +-
 doc/src/sgml/pgwalinspect.sgml                |  48 +++--
 6 files changed, 189 insertions(+), 89 deletions(-)

diff --git a/contrib/pg_walinspect/expected/pg_walinspect.out b/contrib/pg_walinspect/expected/pg_walinspect.out
index 9bcb05354e..413046a7e3 100644
--- a/contrib/pg_walinspect/expected/pg_walinspect.out
+++ b/contrib/pg_walinspect/expected/pg_walinspect.out
@@ -74,7 +74,7 @@ SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2'
 (1 row)
 
 -- ===================================================================
--- Tests to get full page image (FPI) from WAL record
+-- Tests to get block information from WAL record
 -- ===================================================================
 SELECT pg_current_wal_lsn() AS wal_lsn3 \gset
 -- Force FPI on the next update.
@@ -82,9 +82,17 @@ CHECKPOINT;
 -- Update table to generate an FPI.
 UPDATE sample_tbl SET col1 = col1 * 100 WHERE col1 = 1;
 SELECT pg_current_wal_lsn() AS wal_lsn4 \gset
+-- Check if we get block data from WAL record.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn3', :'wal_lsn4')
+  WHERE relfilenode = :'sample_tbl_oid' AND blockdata IS NOT NULL;
+ ok 
+----
+ t
+(1 row)
+
 -- Check if we get FPI from WAL record.
-SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn3', :'wal_lsn4')
-  WHERE relfilenode = :'sample_tbl_oid';
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn3', :'wal_lsn4')
+  WHERE relfilenode = :'sample_tbl_oid' AND fpi IS NOT NULL;
  ok 
 ----
  t
@@ -116,7 +124,7 @@ SELECT has_function_privilege('regress_pg_walinspect',
 (1 row)
 
 SELECT has_function_privilege('regress_pg_walinspect',
-  'pg_get_wal_fpi_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- no
+  'pg_get_wal_block_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- no
  has_function_privilege 
 ------------------------
  f
@@ -146,7 +154,7 @@ SELECT has_function_privilege('regress_pg_walinspect',
 (1 row)
 
 SELECT has_function_privilege('regress_pg_walinspect',
-  'pg_get_wal_fpi_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- yes
+  'pg_get_wal_block_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- yes
  has_function_privilege 
 ------------------------
  t
@@ -160,7 +168,7 @@ GRANT EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn)
   TO regress_pg_walinspect;
 GRANT EXECUTE ON FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean)
   TO regress_pg_walinspect;
-GRANT EXECUTE ON FUNCTION pg_get_wal_fpi_info(pg_lsn, pg_lsn)
+GRANT EXECUTE ON FUNCTION pg_get_wal_block_info(pg_lsn, pg_lsn)
   TO regress_pg_walinspect;
 SELECT has_function_privilege('regress_pg_walinspect',
   'pg_get_wal_record_info(pg_lsn)', 'EXECUTE'); -- yes
@@ -184,7 +192,7 @@ SELECT has_function_privilege('regress_pg_walinspect',
 (1 row)
 
 SELECT has_function_privilege('regress_pg_walinspect',
-  'pg_get_wal_fpi_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- yes
+  'pg_get_wal_block_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- yes
  has_function_privilege 
 ------------------------
  t
@@ -196,7 +204,7 @@ REVOKE EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn)
   FROM regress_pg_walinspect;
 REVOKE EXECUTE ON FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean)
   FROM regress_pg_walinspect;
-REVOKE EXECUTE ON FUNCTION pg_get_wal_fpi_info(pg_lsn, pg_lsn)
+REVOKE EXECUTE ON FUNCTION pg_get_wal_block_info(pg_lsn, pg_lsn)
   FROM regress_pg_walinspect;
 -- ===================================================================
 -- Clean up
diff --git a/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql b/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
index 1e9e1e6115..e674ef25aa 100644
--- a/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
+++ b/contrib/pg_walinspect/pg_walinspect--1.0--1.1.sql
@@ -4,21 +4,25 @@
 \echo Use "ALTER EXTENSION pg_walinspect UPDATE TO '1.1'" to load this file. \quit
 
 --
--- pg_get_wal_fpi_info()
+-- pg_get_wal_block_info()
 --
-CREATE FUNCTION pg_get_wal_fpi_info(IN start_lsn pg_lsn,
+CREATE FUNCTION pg_get_wal_block_info(IN start_lsn pg_lsn,
 	IN end_lsn pg_lsn,
 	OUT lsn pg_lsn,
+	OUT blockid int2,
 	OUT reltablespace oid,
 	OUT reldatabase oid,
 	OUT relfilenode oid,
 	OUT relblocknumber int8,
 	OUT forkname text,
-	OUT fpi bytea
+	OUT blockdata bytea,
+	OUT fpi bytea,
+	OUT fpilen int4,
+	OUT fpiinfo text[]
 )
 RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'pg_get_wal_fpi_info'
+AS 'MODULE_PATHNAME', 'pg_get_wal_block_info'
 LANGUAGE C STRICT PARALLEL SAFE;
 
-REVOKE EXECUTE ON FUNCTION pg_get_wal_fpi_info(pg_lsn, pg_lsn) FROM PUBLIC;
-GRANT EXECUTE ON FUNCTION pg_get_wal_fpi_info(pg_lsn, pg_lsn) TO pg_read_server_files;
+REVOKE EXECUTE ON FUNCTION pg_get_wal_block_info(pg_lsn, pg_lsn) FROM PUBLIC;
+GRANT EXECUTE ON FUNCTION pg_get_wal_block_info(pg_lsn, pg_lsn) TO pg_read_server_files;
diff --git a/contrib/pg_walinspect/pg_walinspect.c b/contrib/pg_walinspect/pg_walinspect.c
index b7b0a805ee..4902d8d7d3 100644
--- a/contrib/pg_walinspect/pg_walinspect.c
+++ b/contrib/pg_walinspect/pg_walinspect.c
@@ -20,6 +20,7 @@
 #include "access/xlogutils.h"
 #include "funcapi.h"
 #include "miscadmin.h"
+#include "utils/array.h"
 #include "utils/builtins.h"
 #include "utils/pg_lsn.h"
 
@@ -30,7 +31,7 @@
 
 PG_MODULE_MAGIC;
 
-PG_FUNCTION_INFO_V1(pg_get_wal_fpi_info);
+PG_FUNCTION_INFO_V1(pg_get_wal_block_info);
 PG_FUNCTION_INFO_V1(pg_get_wal_record_info);
 PG_FUNCTION_INFO_V1(pg_get_wal_records_info);
 PG_FUNCTION_INFO_V1(pg_get_wal_records_info_till_end_of_wal);
@@ -56,7 +57,7 @@ static void FillXLogStatsRow(const char *name, uint64 n, uint64 total_count,
 							 Datum *values, bool *nulls, uint32 ncols);
 static void GetWalStats(FunctionCallInfo fcinfo, XLogRecPtr start_lsn,
 						XLogRecPtr end_lsn, bool stats_per_record);
-static void GetWALFPIInfo(FunctionCallInfo fcinfo, XLogReaderState *record);
+static void GetWALBlockInfo(FunctionCallInfo fcinfo, XLogReaderState *record);
 
 /*
  * Check if the given LSN is in future. Also, return the LSN up to which the
@@ -221,49 +222,49 @@ GetWALRecordInfo(XLogReaderState *record, Datum *values,
 
 
 /*
- * Store a set of full page images from a single record.
+ * Store a set of block information from a single record (FPI and block
+ * information).
  */
 static void
-GetWALFPIInfo(FunctionCallInfo fcinfo, XLogReaderState *record)
+GetWALBlockInfo(FunctionCallInfo fcinfo, XLogReaderState *record)
 {
-#define PG_GET_WAL_FPI_INFO_COLS 7
+#define PG_GET_WAL_BLOCK_INFO_COLS 11
 	int			block_id;
 	ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+	MemoryContext old_cxt;
+	MemoryContext tmp_cxt;
+
+	tmp_cxt = AllocSetContextCreate(CurrentMemoryContext,
+									"GetWALBlockInfo temporary cxt",
+									ALLOCSET_DEFAULT_SIZES);
 
 	for (block_id = 0; block_id <= XLogRecMaxBlockId(record); block_id++)
 	{
-		PGAlignedBlock buf;
-		Page		page;
-		bytea	   *raw_page;
-		BlockNumber blk;
+		DecodedBkpBlock *blk;
+		BlockNumber blkno;
 		RelFileLocator rnode;
 		ForkNumber	fork;
-		Datum		values[PG_GET_WAL_FPI_INFO_COLS] = {0};
-		bool		nulls[PG_GET_WAL_FPI_INFO_COLS] = {0};
+		Datum		values[PG_GET_WAL_BLOCK_INFO_COLS] = {0};
+		bool		nulls[PG_GET_WAL_BLOCK_INFO_COLS] = {0};
 		int			i = 0;
 
 		if (!XLogRecHasBlockRef(record, block_id))
 			continue;
 
-		if (!XLogRecHasBlockImage(record, block_id))
-			continue;
-
-		page = (Page) buf.data;
+		/* Use the tmp context so we can clean up after each tuple is done */
+		old_cxt = MemoryContextSwitchTo(tmp_cxt);
 
-		if (!RestoreBlockImage(record, block_id, page))
-			ereport(ERROR,
-					(errcode(ERRCODE_INTERNAL_ERROR),
-					 errmsg_internal("%s", record->errormsg_buf)));
+		blk = XLogRecGetBlock(record, block_id);
 
-		/* Full page exists, so let's save it. */
 		(void) XLogRecGetBlockTagExtended(record, block_id,
-										  &rnode, &fork, &blk, NULL);
+										  &rnode, &fork, &blkno, NULL);
 
 		values[i++] = LSNGetDatum(record->ReadRecPtr);
-		values[i++] = ObjectIdGetDatum(rnode.spcOid);
-		values[i++] = ObjectIdGetDatum(rnode.dbOid);
-		values[i++] = ObjectIdGetDatum(rnode.relNumber);
-		values[i++] = Int64GetDatum((int64) blk);
+		values[i++] = Int16GetDatum(block_id);
+		values[i++] = ObjectIdGetDatum(blk->rlocator.spcOid);
+		values[i++] = ObjectIdGetDatum(blk->rlocator.dbOid);
+		values[i++] = ObjectIdGetDatum(blk->rlocator.relNumber);
+		values[i++] = Int64GetDatum((int64) blkno);
 
 		if (fork >= 0 && fork <= MAX_FORKNUM)
 			values[i++] = CStringGetTextDatum(forkNames[fork]);
@@ -272,40 +273,115 @@ GetWALFPIInfo(FunctionCallInfo fcinfo, XLogReaderState *record)
 					(errcode(ERRCODE_INTERNAL_ERROR),
 					 errmsg_internal("invalid fork number: %u", fork)));
 
-		/* Initialize bytea buffer to copy the FPI to. */
-		raw_page = (bytea *) palloc(BLCKSZ + VARHDRSZ);
-		SET_VARSIZE(raw_page, BLCKSZ + VARHDRSZ);
+		/* Block data */
+		if (blk->has_data)
+		{
+			bytea	   *raw_data;
 
-		/* Take a verbatim copy of the FPI. */
-		memcpy(VARDATA(raw_page), page, BLCKSZ);
+			/* Initialize bytea buffer to copy the data to */
+			raw_data = (bytea *) palloc(blk->data_len + VARHDRSZ);
+			SET_VARSIZE(raw_data, blk->data_len + VARHDRSZ);
 
-		values[i++] = PointerGetDatum(raw_page);
+			/* Copy the data */
+			memcpy(VARDATA(raw_data), blk->data, blk->data_len);
+			values[i++] = PointerGetDatum(raw_data);
+		}
+		else
+		{
+			/* No data, so set this field to NULL */
+			nulls[i++] = true;
+		}
 
-		Assert(i == PG_GET_WAL_FPI_INFO_COLS);
+		if (blk->has_image)
+		{
+			PGAlignedBlock buf;
+			Page		page;
+			bytea	   *raw_page;
+			int			bitcnt;
+			int			cnt = 0;
+			Datum	   *flags;
+			ArrayType  *a;
+
+			page = (Page) buf.data;
+
+			/* Full page image exists, so let's save it */
+			if (!RestoreBlockImage(record, block_id, page))
+				ereport(ERROR,
+						(errcode(ERRCODE_INTERNAL_ERROR),
+						 errmsg_internal("%s", record->errormsg_buf)));
+
+			/* Initialize bytea buffer to copy the FPI to */
+			raw_page = (bytea *) palloc(BLCKSZ + VARHDRSZ);
+			SET_VARSIZE(raw_page, BLCKSZ + VARHDRSZ);
+
+			/* Take a verbatim copy of the FPI */
+			memcpy(VARDATA(raw_page), page, BLCKSZ);
+
+			values[i++] = PointerGetDatum(raw_page);
+			values[i++] = UInt32GetDatum(blk->bimg_len);
+
+			/* FPI flags */
+			bitcnt = pg_popcount((const char *) &blk->bimg_info,
+								 sizeof(uint8));
+			/* Build set of raw flags */
+			flags = (Datum *) palloc0(sizeof(Datum) * bitcnt);
+			if ((blk->bimg_info & BKPIMAGE_HAS_HOLE) != 0)
+				flags[cnt++] = CStringGetTextDatum("HAS_HOLE");
+
+			if (blk->apply_image)
+				flags[cnt++] = CStringGetTextDatum("APPLY");
+			else
+				flags[cnt++] = CStringGetTextDatum("WAL_VERIFICATION");
+
+			if ((blk->bimg_info & BKPIMAGE_COMPRESS_PGLZ) != 0)
+				flags[cnt++] = CStringGetTextDatum("COMPRESS_PGLZ");
+			if ((blk->bimg_info & BKPIMAGE_COMPRESS_LZ4) != 0)
+				flags[cnt++] = CStringGetTextDatum("COMPRESS_LZ4");
+			if ((blk->bimg_info & BKPIMAGE_COMPRESS_ZSTD) != 0)
+				flags[cnt++] = CStringGetTextDatum("COMPRESS_ZSTD");
+
+			Assert(cnt <= bitcnt);
+			a = construct_array_builtin(flags, cnt, TEXTOID);
+			values[i++] = PointerGetDatum(a);
+		}
+		else
+		{
+			/* No full page image, so store NULLs for all its fields */
+			memset(&nulls[i], true, 3 * sizeof(bool));
+			i += 3;
+		}
+
+		Assert(i == PG_GET_WAL_BLOCK_INFO_COLS);
 
 		tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc,
 							 values, nulls);
+
+		/* clean up and switch back */
+		MemoryContextSwitchTo(old_cxt);
+		MemoryContextReset(tmp_cxt);
 	}
 
-#undef PG_GET_WAL_FPI_INFO_COLS
+	MemoryContextDelete(tmp_cxt);
+
+#undef PG_GET_WAL_FPI_BLOCK_COLS
 }
 
 /*
- * Get full page images with their relation information for all the WAL
- * records between start and end LSNs.  Decompression is applied to the
- * blocks, if necessary.
+ * Get information about all the blocks saved in WAL records between start
+ * and end LSNs. This produces information about the full page images with
+ * their relation information, and the data saved in each block associated
+ * to a record. Decompression is applied to the full page images, if
+ * necessary.
  *
  * This function emits an error if a future start or end WAL LSN i.e. WAL LSN
  * the database system doesn't know about is specified.
  */
 Datum
-pg_get_wal_fpi_info(PG_FUNCTION_ARGS)
+pg_get_wal_block_info(PG_FUNCTION_ARGS)
 {
 	XLogRecPtr	start_lsn;
 	XLogRecPtr	end_lsn;
 	XLogReaderState *xlogreader;
-	MemoryContext old_cxt;
-	MemoryContext tmp_cxt;
 
 	start_lsn = PG_GETARG_LSN(0);
 	end_lsn = PG_GETARG_LSN(1);
@@ -316,26 +392,14 @@ pg_get_wal_fpi_info(PG_FUNCTION_ARGS)
 
 	xlogreader = InitXLogReaderState(start_lsn);
 
-	tmp_cxt = AllocSetContextCreate(CurrentMemoryContext,
-									"pg_get_wal_fpi_info temporary cxt",
-									ALLOCSET_DEFAULT_SIZES);
-
 	while (ReadNextXLogRecord(xlogreader) &&
 		   xlogreader->EndRecPtr <= end_lsn)
 	{
-		/* Use the tmp context so we can clean up after each tuple is done */
-		old_cxt = MemoryContextSwitchTo(tmp_cxt);
-
-		GetWALFPIInfo(fcinfo, xlogreader);
-
-		/* clean up and switch back */
-		MemoryContextSwitchTo(old_cxt);
-		MemoryContextReset(tmp_cxt);
+		GetWALBlockInfo(fcinfo, xlogreader);
 
 		CHECK_FOR_INTERRUPTS();
 	}
 
-	MemoryContextDelete(tmp_cxt);
 	pfree(xlogreader->private_data);
 	XLogReaderFree(xlogreader);
 
diff --git a/contrib/pg_walinspect/sql/pg_walinspect.sql b/contrib/pg_walinspect/sql/pg_walinspect.sql
index 849201a1f8..9ad0e29a5c 100644
--- a/contrib/pg_walinspect/sql/pg_walinspect.sql
+++ b/contrib/pg_walinspect/sql/pg_walinspect.sql
@@ -53,7 +53,7 @@ SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_records_info(:'wal_lsn1', :'wal_lsn2'
 			WHERE resource_manager = 'Heap' AND record_type = 'INSERT';
 
 -- ===================================================================
--- Tests to get full page image (FPI) from WAL record
+-- Tests to get block information from WAL record
 -- ===================================================================
 SELECT pg_current_wal_lsn() AS wal_lsn3 \gset
 
@@ -64,9 +64,13 @@ CHECKPOINT;
 UPDATE sample_tbl SET col1 = col1 * 100 WHERE col1 = 1;
 SELECT pg_current_wal_lsn() AS wal_lsn4 \gset
 
+-- Check if we get block data from WAL record.
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn3', :'wal_lsn4')
+  WHERE relfilenode = :'sample_tbl_oid' AND blockdata IS NOT NULL;
+
 -- Check if we get FPI from WAL record.
-SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_fpi_info(:'wal_lsn3', :'wal_lsn4')
-  WHERE relfilenode = :'sample_tbl_oid';
+SELECT COUNT(*) >= 1 AS ok FROM pg_get_wal_block_info(:'wal_lsn3', :'wal_lsn4')
+  WHERE relfilenode = :'sample_tbl_oid' AND fpi IS NOT NULL;
 
 -- ===================================================================
 -- Tests for permissions
@@ -83,7 +87,7 @@ SELECT has_function_privilege('regress_pg_walinspect',
   'pg_get_wal_stats(pg_lsn, pg_lsn, boolean) ', 'EXECUTE'); -- no
 
 SELECT has_function_privilege('regress_pg_walinspect',
-  'pg_get_wal_fpi_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- no
+  'pg_get_wal_block_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- no
 
 -- Functions accessible by users with role pg_read_server_files
 
@@ -99,7 +103,7 @@ SELECT has_function_privilege('regress_pg_walinspect',
   'pg_get_wal_stats(pg_lsn, pg_lsn, boolean) ', 'EXECUTE'); -- yes
 
 SELECT has_function_privilege('regress_pg_walinspect',
-  'pg_get_wal_fpi_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- yes
+  'pg_get_wal_block_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- yes
 
 REVOKE pg_read_server_files FROM regress_pg_walinspect;
 
@@ -113,7 +117,7 @@ GRANT EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn)
 GRANT EXECUTE ON FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean)
   TO regress_pg_walinspect;
 
-GRANT EXECUTE ON FUNCTION pg_get_wal_fpi_info(pg_lsn, pg_lsn)
+GRANT EXECUTE ON FUNCTION pg_get_wal_block_info(pg_lsn, pg_lsn)
   TO regress_pg_walinspect;
 
 SELECT has_function_privilege('regress_pg_walinspect',
@@ -126,7 +130,7 @@ SELECT has_function_privilege('regress_pg_walinspect',
   'pg_get_wal_stats(pg_lsn, pg_lsn, boolean) ', 'EXECUTE'); -- yes
 
 SELECT has_function_privilege('regress_pg_walinspect',
-  'pg_get_wal_fpi_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- yes
+  'pg_get_wal_block_info(pg_lsn, pg_lsn) ', 'EXECUTE'); -- yes
 
 REVOKE EXECUTE ON FUNCTION pg_get_wal_record_info(pg_lsn)
   FROM regress_pg_walinspect;
@@ -137,7 +141,7 @@ REVOKE EXECUTE ON FUNCTION pg_get_wal_records_info(pg_lsn, pg_lsn)
 REVOKE EXECUTE ON FUNCTION pg_get_wal_stats(pg_lsn, pg_lsn, boolean)
   FROM regress_pg_walinspect;
 
-REVOKE EXECUTE ON FUNCTION pg_get_wal_fpi_info(pg_lsn, pg_lsn)
+REVOKE EXECUTE ON FUNCTION pg_get_wal_block_info(pg_lsn, pg_lsn)
   FROM regress_pg_walinspect;
 
 -- ===================================================================
diff --git a/contrib/pg_walinspect/walinspect.conf b/contrib/pg_walinspect/walinspect.conf
index 67ceb2bb07..367f706651 100644
--- a/contrib/pg_walinspect/walinspect.conf
+++ b/contrib/pg_walinspect/walinspect.conf
@@ -1,2 +1,2 @@
-wal_level = replica
+wal_level = logical
 max_replication_slots = 4
diff --git a/doc/src/sgml/pgwalinspect.sgml b/doc/src/sgml/pgwalinspect.sgml
index 3d7cdb95cc..d26115d6f2 100644
--- a/doc/src/sgml/pgwalinspect.sgml
+++ b/doc/src/sgml/pgwalinspect.sgml
@@ -190,31 +190,51 @@ combined_size_percentage     | 2.8634072910530795
 
    <varlistentry>
     <term>
-     <function>pg_get_wal_fpi_info(start_lsn pg_lsn, end_lsn pg_lsn) returns setof record</function>
+     <function>pg_get_wal_block_info(start_lsn pg_lsn, end_lsn pg_lsn) returns setof record</function>
     </term>
 
     <listitem>
      <para>
-      Gets a copy of full page images as <type>bytea</type> values (after
-      applying decompression when necessary) and their information associated
-      with all the valid WAL records between
+      Gets a copy of the block information stored in WAL records. This includes
+      copies of the block data (<literal>NULL</literal> if none) and full page
+      images as <type>bytea</type> values (after
+      applying decompression when necessary, or <literal>NULL</literal> if none)
+      and their information associated with all the valid WAL records between
       <replaceable>start_lsn</replaceable> and
-      <replaceable>end_lsn</replaceable>. Returns one row per full page image.
-      If <replaceable>start_lsn</replaceable> or
+      <replaceable>end_lsn</replaceable>. Returns one row per block registered
+      in a WAL record. If <replaceable>start_lsn</replaceable> or
       <replaceable>end_lsn</replaceable> are not yet available, the function
       will raise an error. For example:
 <screen>
-postgres=# SELECT lsn, reltablespace, reldatabase, relfilenode, relblocknumber,
-                  forkname, substring(fpi for 24) as fpi_trimmed
-             FROM pg_get_wal_fpi_info('0/1801690', '0/1825C60');
+postgres=# SELECT lsn, blockid, reltablespace, reldatabase, relfilenode,
+                  relblocknumber, forkname,
+                  substring(blockdata for 24) as block_trimmed,
+                  substring(fpi for 24) as fpi_trimmed, fpilen, fpiinfo
+             FROM pg_get_wal_block_info('0/1871080', '0/1871440');
 -[ RECORD 1 ]--+---------------------------------------------------
-lsn            | 0/1807E20
+lsn            | 0/1871208
+blockid        | 0
 reltablespace  | 1663
-reldatabase    | 5
-relfilenode    | 16396
-relblocknumber | 43
+reldatabase    | 16384
+relfilenode    | 1255
+relblocknumber | 96
 forkname       | main
-fpi_trimmed    | \x00000000b89e660100000000a003c0030020042000000000
+block_trimmed  | \x0700080009000a00
+fpi_trimmed    |
+fpilen         |
+fpiinfo        |
+-[ RECORD 2 ]--+---------------------------------------------------
+lsn            | 0/18712F8
+blockid        | 0
+reltablespace  | 1663
+reldatabase    | 16384
+relfilenode    | 16392
+relblocknumber | 0
+forkname       | main
+block_trimmed  | \x02800128180164000000
+fpi_trimmed    | \x0000000050108701000000002c00601f00200420e0020000
+fpilen         | 204
+fpiinfo        | {HAS_HOLE,APPLY}
 </screen>
      </para>
     </listitem>
-- 
2.34.1



^ permalink  raw  reply  [nested|flat] 9+ messages in thread


end of thread, other threads:[~2023-03-07 10:26 UTC | newest]

Thread overview: 9+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-09-10 00:49 [PATCH v3 2/2] windows: Improve crash / assert / exception handling. Andres Freund <[email protected]>
2021-09-10 00:49 [PATCH v6 1/2] windows: Improve crash / assert / exception handling. Andres Freund <[email protected]>
2021-09-10 00:49 [PATCH v4 2/2] windows: Improve crash / assert / exception handling. Andres Freund <[email protected]>
2021-09-10 00:49 [PATCH v2 2/2] windows: Improve crash / assert / exception handling. Andres Freund <[email protected]>
2023-03-07 05:44 Re: Add pg_walinspect function with block info columns Michael Paquier <[email protected]>
2023-03-07 06:49 ` Re: Add pg_walinspect function with block info columns Kyotaro Horiguchi <[email protected]>
2023-03-07 07:18   ` Re: Add pg_walinspect function with block info columns Michael Paquier <[email protected]>
2023-03-07 09:07     ` Re: Add pg_walinspect function with block info columns Kyotaro Horiguchi <[email protected]>
2023-03-07 10:26     ` Re: Add pg_walinspect function with block info columns Bharath Rupireddy <[email protected]>

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox