Date: Fri, 2 Sep 2022 06:55:11 -0500
From: Justin Pryzby <pryzby@telsasoft.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Andrey Borodin <x4mmm@yandex-team.ru>, pgsql-hackers@postgresql.org,
	Thomas Munro <thomas.munro@gmail.com>, Peter Geoghegan <pg@bowt.ie>,
	Andres Freund <andres@anarazel.de>
Subject: Re: Different compression methods for FPI
Message-ID: <20220902115511.GY31833@telsasoft.com>
References: <20210315180918.GW29463@telsasoft.com>
 <20210321193004.GA4203@telsasoft.com>
 <YKIelANwse+k2JxT@paquier.xyz>
 <20210519030618.GS373@telsasoft.com>
 <YKTa4+kTFkb3EIBs@paquier.xyz>
 <20210525044444.GV3676@telsasoft.com>
 <YKymshLnYZeD0cue@paquier.xyz>
 <3252174C-C21D-4711-A038-5230DE465BAE@yandex-team.ru>
 <YLWWPaq/KnVS24J4@paquier.xyz>
 <20210614012412.GA31772@telsasoft.com>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="um2V5WpqCyd73IVb"
Content-Disposition: inline
In-Reply-To: <20210614012412.GA31772@telsasoft.com>
User-Agent: Mutt/1.9.4 (2018-02-28)
Archived-At: 
 <https://www.postgresql.org/message-id/20220902115511.GY31833%40telsasoft.com>
Precedence: bulk


--um2V5WpqCyd73IVb
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

On Sun, Jun 13, 2021 at 08:24:12PM -0500, Justin Pryzby wrote:
> In this patch series, I added compression information to the errcontext from
> xlog_block_info(), and allow specifying compression levels like zlib-2.  I'll
> rearrange that commit earlier if we decide that's desirable to include.

4035cd5d4 added wal_compress=lz4 and
e9537321a added wal_compress=zstd

Since 4035cd5d4, pg_waldump has shown compression info (and walinspect
shows the same since 2258e76f9).

But if you try to replay WAL on a server which wasn't compiled with
support for the requisite compression methods, it's a bit crummy that it
doesn't include in the error message the reason *why* it failed to
restore the image, if that's caused by the missing compression method.

This hunk was from my patch in June, 2021, but wasn't included in the
final patches.  This would include the compression info algorithm: 1)
when failing to apply WAL in rm_redo_error_callback(); and, 2) in
wal_debug.

< 2022-08-31 21:37:53.325 CDT  >FATAL:  failed to restore block image
< 2022-08-31 21:37:53.325 CDT  >CONTEXT:  WAL redo at 1201/1B931F50 for XLOG/FPI_FOR_HINT: ; blkref #0: rel 1663/16888/164320567, blk 8186 FPW, compression method: zstd

In addition to cases where someone re/compiles postgres locally, I guess this
would also improve the situation for PITR and standbys, which might reasonably
be run on a different OS, with different OS packages, or with postgres compiled
separately.

> diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
> index 17eeff0720..1ccc51575a 100644
> --- a/src/backend/access/transam/xlog.c
> +++ b/src/backend/access/transam/xlog.c
> @@ -10470,7 +10470,17 @@ xlog_block_info(StringInfo buf, XLogReaderState *record)
>  							 rnode.spcNode, rnode.dbNode, rnode.relNode,
>  							 blk);
>  		if (XLogRecHasBlockImage(record, block_id))
> -			appendStringInfoString(buf, " FPW");
> +		{
> +			int compression =
> +				BKPIMAGE_IS_COMPRESSED(record->blocks[block_id].bimg_info) ?
> +				BKPIMAGE_COMPRESSION(record->blocks[block_id].bimg_info) : -1;
> +			if (compression == -1)
> +				appendStringInfoString(buf, " FPW");
> +			else
> +				appendStringInfo(buf, " FPW, compression method %d/%s",
> +					compression, wal_compression_name(compression));
> +		}
> +


--um2V5WpqCyd73IVb
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="0001-xlog_block_info-show-compression-method.patch"

From ce76fbaa0c0c9126a8b233891b9b32001b07129b Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 19 Feb 2022 11:35:53 -0600
Subject: [PATCH] xlog_block_info: show compression method

---
 src/backend/access/transam/xlogrecovery.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c
index 4ad145dd167..75a72b47f78 100644
--- a/src/backend/access/transam/xlogrecovery.c
+++ b/src/backend/access/transam/xlogrecovery.c
@@ -2238,7 +2238,20 @@ xlog_block_info(StringInfo buf, XLogReaderState *record)
 							 rlocator.relNumber,
 							 blk);
 		if (XLogRecHasBlockImage(record, block_id))
-			appendStringInfoString(buf, " FPW");
+		{
+			if (!BKPIMAGE_COMPRESSED(record->blocks[block_id].bimg_info))
+				appendStringInfoString(buf, " FPW");
+			else
+			{
+				uint8 info = record->blocks[block_id].bimg_info;
+				char *compression =
+					(BKPIMAGE_COMPRESS_PGLZ & info) ? "pglz" :
+					(BKPIMAGE_COMPRESS_LZ4 & info) ? "lz4" :
+					(BKPIMAGE_COMPRESS_ZSTD & info) ? "zstd" : "unknown";
+
+				appendStringInfo(buf, " FPW, compression method: %s", compression);
+			}
+		}
 	}
 }
 
-- 
2.17.1


--um2V5WpqCyd73IVb--