pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Michael Paquier <michael@paquier.xyz>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Add support for zstd with compression of full-page writes in WAL
Date: Fri, 11 Mar 2022 03:19:54 +0000
Message-ID: <E1nSVoo-000CqO-5E@gemulon.postgresql.org> (raw)

Add support for zstd with compression of full-page writes in WAL

wal_compression gains a new value, "zstd", to allow the compression of
full-page images using the compression method of the same name.

Compression is done using the default level recommended by the library,
as of ZSTD_CLEVEL_DEFAULT = 3.  Some benchmarking has shown that it
could make sense to use a level lower for the FPI compression, like 1 or
2, as the compression rate did not change much with a bit less CPU
consumed, but any tests done would only cover few scenarios so it is
hard to come to a clear conclusion.  Anyway, there is no reason to not
use the default level instead, which is the level recommended by the
library so it should be fine for most cases.

zstd outclasses easily pglz, and is better than LZ4 where one wants to
have more compression at the cost of extra CPU but both are good enough
in their own scenarios, so the choice between one or the other of these
comes to a study of the workload patterns and the schema involved,
mainly.

This commit relies heavily on 4035cd5, that reshaped the code creating
and restoring full-page writes to be aware of the compression type,
making this integration straight-forward.

This patch borrows some early work from Andrey Borodin, though the patch
got a complete rewrite.

Author: Justin Pryzby
Discussion: https://postgr.es/m/20220222231948.GJ9008@telsasoft.com

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/e9537321a74a2b062c8f7a452314b4570913f780

Modified Files
--------------
doc/src/sgml/config.sgml                      | 11 +++++----
doc/src/sgml/installation.sgml                |  8 +++++++
src/backend/access/transam/xloginsert.c       | 32 ++++++++++++++++++++++++++-
src/backend/access/transam/xlogreader.c       | 20 +++++++++++++++++
src/backend/utils/misc/guc.c                  |  3 +++
src/backend/utils/misc/postgresql.conf.sample |  2 +-
src/bin/pg_waldump/pg_waldump.c               |  2 ++
src/include/access/xlog.h                     |  3 ++-
src/include/access/xlogrecord.h               |  5 ++++-
9 files changed, 78 insertions(+), 8 deletions(-)



Message-ID: <E1nSVoo-000CqO-5E@gemulon.postgresql.org>
Permalink:  ../E1nSVoo-000CqO-5E@gemulon.postgresql.org/
Also on:    postgresql.org/message-id/E1nSVoo-000CqO-5E@gemulon.postgresql.org

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-committers@postgresql.org
  Cc: michael@paquier.xyz
  Subject: Re: pgsql: Add support for zstd with compression of full-page writes in WAL
  In-Reply-To: <E1nSVoo-000CqO-5E@gemulon.postgresql.org>
  References: <E1nSVoo-000CqO-5E@gemulon.postgresql.org>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox