Date: Mon, 5 Sep 2022 08:02:35 -0500
From: Justin Pryzby <pryzby@telsasoft.com>
To: Matthias van de Meent <boekewurm+postgres@gmail.com>
Cc: pgsql-hackers@postgresql.org
Subject: Re: Different compression methods for FPI
Message-ID: <20220905130235.GJ31833@telsasoft.com>
References: <20210519030618.GS373@telsasoft.com>
 <YKTa4+kTFkb3EIBs@paquier.xyz>
 <20210525044444.GV3676@telsasoft.com>
 <YKymshLnYZeD0cue@paquier.xyz>
 <3252174C-C21D-4711-A038-5230DE465BAE@yandex-team.ru>
 <YLWWPaq/KnVS24J4@paquier.xyz>
 <20210614012412.GA31772@telsasoft.com>
 <20220902115511.GY31833@telsasoft.com>
 <20220905002320.GD31833@telsasoft.com>
 <CAEze2Wjndd_idVQYcs=K2DW+0-j8eBTr8O0T+d6zz4i9-o+mWw@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
 <CAEze2Wjndd_idVQYcs=K2DW+0-j8eBTr8O0T+d6zz4i9-o+mWw@mail.gmail.com>
User-Agent: Mutt/1.9.4 (2018-02-28)
Archived-At: 
 <https://www.postgresql.org/message-id/20220905130235.GJ31833%40telsasoft.com>
Precedence: bulk

On Mon, Sep 05, 2022 at 02:45:57PM +0200, Matthias van de Meent wrote:
> Hi,
> 
> I have a small question for those involved:

I suggest to "reply all"

> Context: I'm trying to get the smallest BKPIMAGE size possible
> regardless of CPU cost, which means I'm trying multiple compressions
> to get the smallest data possible with the options available. This
> means ignoring the wal_compression GUC in XLogCompressBackupBlock and
> brute-forcing the smallest compression available, potentially layering
> compression methods, and returning the bimg_info compression flags
> that will be stored in XLogCompressBackupBlock and used to decompress
> the block's data.

I think once you apply one compression method/algorithm, you shouldn't
expect other "layered" methods to be able to compress it at all.  I
think you'll get better compression by using a higher compression level
in zstd (or zlib) than with any combination of methods.  A patch for
configurable compression level was here:

https://postgr.es/m/20220222231948.GJ9008@telsasoft.com

> Is there a prescribed order of compression algorithms to apply when
> (de)compressing full page images in WAL, when the bimg_info has more
> than one BKPIMAGE_COMPRESS_*-flags set? That is, when I want to check
> the compression of the block image with both ZSTD and LZ4, which order
> is the ordering indicated by bimg_info = (COMPRESS_LZ4 |
> COMPRESS_ZSTD)?

There's currently a separate bit for each method, but it's not supported
to "stack" them (See also the "Save bits" patch, above).

This came up before when Greg asked about it.
https://www.postgresql.org/message-id/20210622031358.GF29179@telsasoft.com
https://www.postgresql.org/message-id/20220131222800.GY23027@telsasoft.com

-- 
Justin