From: Justin Pryzby <pryzby@telsasoft.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Andrey Borodin <x4mmm@yandex-team.ru>
Cc: pgsql-hackers@postgresql.org, Thomas Munro <thomas.munro@gmail.com>
Cc: Peter Geoghegan <pg@bowt.ie>
Cc: Andres Freund <andres@anarazel.de>
Subject: Re: Different compression methods for FPI
Date: Fri, 2 Sep 2022 06:55:11 -0500
Message-ID: <20220902115511.GY31833@telsasoft.com> (raw)
In-Reply-To: <20210614012412.GA31772@telsasoft.com>
References: <20210315180918.GW29463@telsasoft.com>
<20210321193004.GA4203@telsasoft.com>
<YKIelANwse+k2JxT@paquier.xyz>
<20210519030618.GS373@telsasoft.com>
<YKTa4+kTFkb3EIBs@paquier.xyz>
<20210525044444.GV3676@telsasoft.com>
<YKymshLnYZeD0cue@paquier.xyz>
<3252174C-C21D-4711-A038-5230DE465BAE@yandex-team.ru>
<YLWWPaq/KnVS24J4@paquier.xyz>
<20210614012412.GA31772@telsasoft.com>
On Sun, Jun 13, 2021 at 08:24:12PM -0500, Justin Pryzby wrote:
> In this patch series, I added compression information to the errcontext from
> xlog_block_info(), and allow specifying compression levels like zlib-2. I'll
> rearrange that commit earlier if we decide that's desirable to include.
4035cd5d4 added wal_compress=lz4 and
e9537321a added wal_compress=zstd
Since 4035cd5d4, pg_waldump has shown compression info (and walinspect
shows the same since 2258e76f9).
But if you try to replay WAL on a server which wasn't compiled with
support for the requisite compression methods, it's a bit crummy that it
doesn't include in the error message the reason *why* it failed to
restore the image, if that's caused by the missing compression method.
This hunk was from my patch in June, 2021, but wasn't included in the
final patches. This would include the compression info algorithm: 1)
when failing to apply WAL in rm_redo_error_callback(); and, 2) in
wal_debug.
< 2022-08-31 21:37:53.325 CDT >FATAL: failed to restore block image
< 2022-08-31 21:37:53.325 CDT >CONTEXT: WAL redo at 1201/1B931F50 for XLOG/FPI_FOR_HINT: ; blkref #0: rel 1663/16888/164320567, blk 8186 FPW, compression method: zstd
In addition to cases where someone re/compiles postgres locally, I guess this
would also improve the situation for PITR and standbys, which might reasonably
be run on a different OS, with different OS packages, or with postgres compiled
separately.
> diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
> index 17eeff0720..1ccc51575a 100644
> --- a/src/backend/access/transam/xlog.c
> +++ b/src/backend/access/transam/xlog.c
> @@ -10470,7 +10470,17 @@ xlog_block_info(StringInfo buf, XLogReaderState *record)
> rnode.spcNode, rnode.dbNode, rnode.relNode,
> blk);
> if (XLogRecHasBlockImage(record, block_id))
> - appendStringInfoString(buf, " FPW");
> + {
> + int compression =
> + BKPIMAGE_IS_COMPRESSED(record->blocks[block_id].bimg_info) ?
> + BKPIMAGE_COMPRESSION(record->blocks[block_id].bimg_info) : -1;
> + if (compression == -1)
> + appendStringInfoString(buf, " FPW");
> + else
> + appendStringInfo(buf, " FPW, compression method %d/%s",
> + compression, wal_compression_name(compression));
> + }
> +
Attachments:0001-xlog_block_info-show-compression-method.patch (text/x-diff, 1.2K) download
From ce76fbaa0c0c9126a8b233891b9b32001b07129b Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 19 Feb 2022 11:35:53 -0600
Subject: [PATCH] xlog_block_info: show compression method
---
src/backend/access/transam/xlogrecovery.c | 15 ++++++++++++++-
1 file changed, 14 insertions(+), 1 deletion(-)
diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.cindex 4ad145dd167..75a72b47f78 100644--- a/src/backend/access/transam/xlogrecovery.c+++ b/src/backend/access/transam/xlogrecovery.c@@ -2238,7 +2238,20 @@ xlog_block_info(StringInfo buf, XLogReaderState *record)
rlocator.relNumber,
blk);
if (XLogRecHasBlockImage(record, block_id))
- appendStringInfoString(buf, " FPW");+ {+ if (!BKPIMAGE_COMPRESSED(record->blocks[block_id].bimg_info))+ appendStringInfoString(buf, " FPW");+ else+ {+ uint8 info = record->blocks[block_id].bimg_info;+ char *compression =+ (BKPIMAGE_COMPRESS_PGLZ & info) ? "pglz" :+ (BKPIMAGE_COMPRESS_LZ4 & info) ? "lz4" :+ (BKPIMAGE_COMPRESS_ZSTD & info) ? "zstd" : "unknown";++ appendStringInfo(buf, " FPW, compression method: %s", compression);+ }+ }
}
}
--
2.17.1
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: pryzby@telsasoft.com, michael@paquier.xyz, x4mmm@yandex-team.ru, thomas.munro@gmail.com, pg@bowt.ie, andres@anarazel.de
Subject: Re: Different compression methods for FPI
In-Reply-To: <20220902115511.GY31833@telsasoft.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox