pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: PG Bug reporting form <noreply@postgresql.org>
To: pgsql-bugs@lists.postgresql.org
Cc: idrissaajalloh@gmail.com
Subject: BUG #19890: pg_receivewal advances past a truncated LZ4 WAL segment
Date: Wed, 11 Nov 2026 15:50:43 +0000
Message-ID: <19890-293babc93c520dc8@postgresql.org> (raw)

The following bug has been logged on the website:

Bug reference:      19890
Logged by:          Wren
Email address:      idrissaajalloh@gmail.com
PostgreSQL version: 18.6
Operating system:   Linux/amd64
Description:        

pg_receivewal treats an incomplete LZ4 frame with a completed WAL filename
as a complete segment when decompression has already produced exactly
WalSegSz bytes. On restart, it advances to the following segment, skipping
validation or reacquisition of the damaged segment.

A completed-name LZ4 segment should be accepted only if decompression
produces exactly WalSegSz bytes and reaches the LZ4 frame-completion state.
A frame truncated after its final output bytes but before its footer is
complete should be rejected or skipped.

How to repeat:

Save the following as run.sh in an empty directory. Run it as `sh run.sh
/absolute/path/to/postgresql-source`. The source tree needs the normal build
prerequisites and the LZ4 development library.

#!/bin/sh
set -eu

if test "$#" -ne 1; then
        echo "usage: $0 /absolute/path/to/postgresql-source" >&2
        exit 2
fi
src=$1
case "$src" in /*) ;; *) echo "source path must be absolute" >&2; exit 2;;
esac
test -x "$src/configure" || { echo "not a PostgreSQL source tree: $src" >&2;
exit 2; }

base=$(pwd)/work.$$
build=$base/build
inst=$base/install
data=$base/data
sock=$base/socket
wal=$base/wal
mkdir -p "$build" "$inst" "$data" "$sock" "$wal"

cd "$build"
"$src/configure" --prefix="$inst" --with-lz4
make -j2
make install

"$inst/bin/initdb" -D "$data" -A trust --no-locale
cat >>"$data/postgresql.conf" <<EOF
listen_addresses = ''
unix_socket_directories = '$sock'
wal_level = replica
max_wal_senders = 5
EOF
"$inst/bin/pg_ctl" -D "$data" -l "$base/server.log" start
trap '"$inst/bin/pg_ctl" -D "$data" stop -m immediate >/dev/null 2>&1 ||
true' EXIT HUP INT TERM

export PGHOST=$sock PGDATABASE=postgres
# Receive two whole segments. Repeated switches are used because the first
# switch can finish the segment in which initdb/startup left the server.
endpos=$("$inst/bin/psql" -AtX -c "SELECT '0/0'::pg_lsn +
((((pg_current_wal_insert_lsn()-'0/0')::bigint/16777216)+2)*16777216)")
(
        sleep 1
        "$inst/bin/psql" -X -c 'SELECT pg_switch_wal()' >/dev/null
        "$inst/bin/psql" -X -c 'CREATE TABLE lz4_frame_probe(a text); INSERT
INTO lz4_frame_probe SELECT repeat(md5(g::text),100) FROM
generate_series(1,20000) g; SELECT pg_switch_wal(); DROP TABLE
lz4_frame_probe; SELECT pg_switch_wal()' >/dev/null
) &
producer=$!
"$inst/bin/pg_receivewal" -D "$wal" -Z lz4 -E "$endpos" -n -v
>"$base/first.log" 2>&1
wait "$producer"

set -- "$wal"/????????????????????????.lz4
test "$#" -ge 2 || { echo "SETUP ERROR: fewer than two completed LZ4
segments"; exit 2; }
highest=
previous=
for f in $(printf '%s\n' "$wal"/????????????????????????.lz4 | sort); do
previous=$highest; highest=$f; done
test -n "$previous" && test -n "$highest"
before=$(wc -c <"$highest")
truncate -s $((before - 1)) "$highest"

# With the lower complete segment retained, rejection has an unambiguous
# oracle: restart at the corrupted segment, not its successor.
set +e
"$inst/bin/pg_receivewal" -D "$wal" -Z lz4 -E 0/1 -n -v >"$base/second.log"
2>&1
rc=$?
set -e
cat "$base/second.log"

highhex=$(basename "$highest" .lz4)
logid=$(printf '%d' "0x$(printf '%s' "$highhex" | cut -c9-16)")
segid=$(printf '%d' "0x$(printf '%s' "$highhex" | cut -c17-24)")
nextno=$((logid * 256 + segid + 1))
ptr_hi=$((nextno / 256))
ptr_lo=$(((nextno % 256) * 16777216))
successor=$(printf '%X/%08X' "$ptr_hi" "$ptr_lo")
if grep -q "starting log streaming at $successor" "$base/second.log"; then
        echo "BUG REPRODUCED: truncated $(basename "$highest") was accepted;
rc=$rc"
        exit 1
fi
echo "NOT REPRODUCED: truncated frame was not used as a completed segment;
rc=$rc"
exit 0

The affected behavior is reported when the second pg_receivewal invocation
starts at the successor of the truncated highest segment and the script
prints:

BUG REPRODUCED: truncated <WAL filename>.lz4 was accepted; rc=0

The script then exits with status 1. Exit status 2 indicates a setup
failure. A corrected implementation should reject or skip the truncated
frame, avoid starting at its successor, print `NOT REPRODUCED`, and exit
with status 0.

In the recorded PostgreSQL 18.6 run, a completed segment was truncated by
one byte and pg_receivewal produced:

TRUNCATED_FILE=00000001000000000000001A.lz4 BEFORE=140130 AFTER=140129
pg_receivewal: starting log streaming at 0/1B000000 (timeline 1)
pg_receivewal: stopped log streaming at 0/1B000000 (timeline 1)
SECOND_RC=0

Here 0/1B000000 is the boundary following the truncated
00000001000000000000001A.lz4 segment, showing that directory scanning
classified the incomplete frame as complete.

The issue was reproduced on PostgreSQL 18.6 on Linux/amd64 with LZ4 support.
The LZ4 path used to find the streaming start counts decompressed bytes but
does not require LZ4F_decompress() to return 0, the frame-completion signal.
Consequently, truncating the frame footer can leave the decompressed size
equal to WalSegSz and allow the file to be accepted.

This occurs when pg_receivewal restarts from a directory whose highest
completed-name LZ4 WAL file is truncated after producing a full segment but
before frame completion.








Message-ID: <19890-293babc93c520dc8@postgresql.org>
Permalink:  ../19890-293babc93c520dc8@postgresql.org/
Also on:    postgresql.org/message-id/19890-293babc93c520dc8@postgresql.org

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-bugs@postgresql.org
  Cc: noreply@postgresql.org, idrissaajalloh@gmail.com
  Subject: Re: BUG #19890: pg_receivewal advances past a truncated LZ4 WAL segment
  In-Reply-To: <19890-293babc93c520dc8@postgresql.org>
  References: <19890-293babc93c520dc8@postgresql.org>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox