From: Michael Paquier <michael@paquier.xyz>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Fix bug in archive streamer with LZ4 decompression
Date: Wed, 02 Jul 2025 04:49:57 +0000
Message-ID: <E1uWpPx-004ITD-1m@gemulon.postgresql.org> (raw)
Fix bug in archive streamer with LZ4 decompression
When decompressing some input data, the calculation for the initial
starting point and the initial size were incorrect, potentially leading
to failures when decompressing contents with LZ4. These initialization
points are fixed in this commit, bringing the logic closer to what
exists for gzip and zstd.
The contents of the compressed data is clear (for example backups taken
with LZ4 can still be decompressed with a "lz4" command), only the
decompression part reading the input data was impacted by this issue.
This code path impacts pg_basebackup and pg_verifybackup, which can use
the LZ4 decompression routines with an archive streamer, or any tools
that try to use the archive streamers in src/fe_utils/.
The issue is easier to reproduce with files that have a low-compression
rate, like ones filled with random data, for a size of at least 512kB,
but this could happen with anything as long as it is stored in a data
folder. Some tests are added based on this idea, with a file filled
with random bytes grabbed from the backend, written at the root of the
data folder. This is proving good enough to reproduce the original
problem.
Author: Mikhail Gribkov <youzhick@gmail.com>
Discussion: https://postgr.es/m/CAMEv5_uQS1Hg6KCaEP2JkrTBbZ-nXQhxomWrhYQvbdzR-zy-wA@mail.gmail.com
Backpatch-through: 15
Branch
------
REL_16_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/5c639523f721e9bd6139916200470e151d690728
Modified Files
--------------
src/bin/pg_basebackup/bbstreamer_lz4.c | 4 ++--
src/bin/pg_verifybackup/t/008_untar.pl | 24 ++++++++++++++++++++++++
src/bin/pg_verifybackup/t/010_client_untar.pl | 25 +++++++++++++++++++++++++
3 files changed, 51 insertions(+), 2 deletions(-)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-committers@postgresql.org
Cc: michael@paquier.xyz, pgsql-committers@lists.postgresql.org
Subject: Re: pgsql: Fix bug in archive streamer with LZ4 decompression
In-Reply-To: <E1uWpPx-004ITD-1m@gemulon.postgresql.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox