pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Justin Pryzby <pryzby@telsasoft.com>
To: Matthias van de Meent <boekewurm+postgres@gmail.com>
Cc: pgsql-hackers@postgresql.org
Subject: Re: Different compression methods for FPI
Date: Mon, 5 Sep 2022 08:02:35 -0500
Message-ID: <20220905130235.GJ31833@telsasoft.com> (raw)
In-Reply-To: <CAEze2Wjndd_idVQYcs=K2DW+0-j8eBTr8O0T+d6zz4i9-o+mWw@mail.gmail.com>
References: <20210519030618.GS373@telsasoft.com>
	<YKTa4+kTFkb3EIBs@paquier.xyz>
	<20210525044444.GV3676@telsasoft.com>
	<YKymshLnYZeD0cue@paquier.xyz>
	<3252174C-C21D-4711-A038-5230DE465BAE@yandex-team.ru>
	<YLWWPaq/KnVS24J4@paquier.xyz>
	<20210614012412.GA31772@telsasoft.com>
	<20220902115511.GY31833@telsasoft.com>
	<20220905002320.GD31833@telsasoft.com>
	<CAEze2Wjndd_idVQYcs=K2DW+0-j8eBTr8O0T+d6zz4i9-o+mWw@mail.gmail.com>

On Mon, Sep 05, 2022 at 02:45:57PM +0200, Matthias van de Meent wrote:
> Hi,
> 
> I have a small question for those involved:

I suggest to "reply all"

> Context: I'm trying to get the smallest BKPIMAGE size possible
> regardless of CPU cost, which means I'm trying multiple compressions
> to get the smallest data possible with the options available. This
> means ignoring the wal_compression GUC in XLogCompressBackupBlock and
> brute-forcing the smallest compression available, potentially layering
> compression methods, and returning the bimg_info compression flags
> that will be stored in XLogCompressBackupBlock and used to decompress
> the block's data.

I think once you apply one compression method/algorithm, you shouldn't
expect other "layered" methods to be able to compress it at all.  I
think you'll get better compression by using a higher compression level
in zstd (or zlib) than with any combination of methods.  A patch for
configurable compression level was here:

https://postgr.es/m/20220222231948.GJ9008@telsasoft.com

> Is there a prescribed order of compression algorithms to apply when
> (de)compressing full page images in WAL, when the bimg_info has more
> than one BKPIMAGE_COMPRESS_*-flags set? That is, when I want to check
> the compression of the block image with both ZSTD and LZ4, which order
> is the ordering indicated by bimg_info = (COMPRESS_LZ4 |
> COMPRESS_ZSTD)?

There's currently a separate bit for each method, but it's not supported
to "stack" them (See also the "Save bits" patch, above).

This came up before when Greg asked about it.
https://www.postgresql.org/message-id/20210622031358.GF29179@telsasoft.com
https://www.postgresql.org/message-id/20220131222800.GY23027@telsasoft.com

-- 
Justin





view thread (65+ messages)  latest in thread

Message-ID: <20220905130235.GJ31833@telsasoft.com>
Permalink:  ../20220905130235.GJ31833@telsasoft.com/
Also on:    postgresql.org/message-id/20220905130235.GJ31833@telsasoft.com

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: pryzby@telsasoft.com, boekewurm+postgres@gmail.com
  Subject: Re: Different compression methods for FPI
  In-Reply-To: <20220905130235.GJ31833@telsasoft.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox