Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xRYdu-000000014up-2EJG for pgsql-bugs@arkaria.postgresql.org; Thu, 12 Nov 2026 17:31:22 +0000 Received: from localhost ([127.0.0.1]:47716 helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xRYds-0000000450q-10Vl for pgsql-bugs@arkaria.postgresql.org; Thu, 12 Nov 2026 17:31:20 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]:36128) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xRAbM-0000000BtRy-2Hg9 for pgsql-bugs@lists.postgresql.org; Wed, 11 Nov 2026 15:51:09 +0000 Received: from mahout.postgresql.org ([2001:4800:3e1:1::227]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xRAbK-00000000UYm-2svj for pgsql-bugs@lists.postgresql.org; Wed, 11 Nov 2026 15:51:08 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=postgresql.org; s=20171124; h=Message-ID:Date:Reply-To:Cc:From:To:Subject: Content-Transfer-Encoding:MIME-Version:Content-Type:Sender:Content-ID: Content-Description:In-Reply-To:References; bh=1XOFsUuLoRY+ecE+LvdIDj2Y3jLL/3ZulvxgAW6Q5iI=; b=ewEUjumz3hPf7gX7fZvyM2DD2o /n6LebsqOtpBEBaDp9yJa33zyFak3MN38PirEoPVDq6CyjwApOEisTzQHp9Vi2EEiVPCaMScPajSk 2h7YCc+DRNfObdTwGASSpLHx96rZSH714gjNWe/tl6r2Tv2BFtjqDxAOBpaP4yD3lSNgnfRa3+9dT RqTuUHA1Q063kbvui0Q1spl7RjqnqDS01Kt2yJUSHA9v7MGqOS+v6+wCYndBP7/hvHK840xjBlERk BC1bbo8CPLY4PquQbxkmsgTsxUJtqeG70EzcDOu4f88+VlEyQTZJuhxBINCM/0/aSRFxOfmJiqlBG TL72me0g==; Received: from wrigleys.postgresql.org ([2a02:16a8:dc51::60]) by mahout.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xRAbJ-000000015cz-0JHI for pgsql-bugs@lists.postgresql.org; Wed, 11 Nov 2026 15:51:05 +0000 Received: from localhost ([127.0.0.1] helo=wrigleys.postgresql.org) by wrigleys.postgresql.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xRAbH-00000002ndv-3KJG for pgsql-bugs@lists.postgresql.org; Wed, 11 Nov 2026 15:51:03 +0000 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Subject: BUG #19890: pg_receivewal advances past a truncated LZ4 WAL segment To: pgsql-bugs@lists.postgresql.org From: PG Bug reporting form Cc: idrissaajalloh@gmail.com Reply-To: idrissaajalloh@gmail.com, pgsql-bugs@lists.postgresql.org Date: Wed, 11 Nov 2026 15:50:43 +0000 Message-ID: <19890-293babc93c520dc8@postgresql.org> X-Auto-Response-Suppress: All Auto-Submitted: auto-generated List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk The following bug has been logged on the website: Bug reference: 19890 Logged by: Wren Email address: idrissaajalloh@gmail.com PostgreSQL version: 18.6 Operating system: Linux/amd64 Description: =20 pg_receivewal treats an incomplete LZ4 frame with a completed WAL filename as a complete segment when decompression has already produced exactly WalSegSz bytes. On restart, it advances to the following segment, skipping validation or reacquisition of the damaged segment. A completed-name LZ4 segment should be accepted only if decompression produces exactly WalSegSz bytes and reaches the LZ4 frame-completion state. A frame truncated after its final output bytes but before its footer is complete should be rejected or skipped. How to repeat: Save the following as run.sh in an empty directory. Run it as `sh run.sh /absolute/path/to/postgresql-source`. The source tree needs the normal build prerequisites and the LZ4 development library. #!/bin/sh set -eu if test "$#" -ne 1; then echo "usage: $0 /absolute/path/to/postgresql-source" >&2 exit 2 fi src=3D$1 case "$src" in /*) ;; *) echo "source path must be absolute" >&2; exit 2;; esac test -x "$src/configure" || { echo "not a PostgreSQL source tree: $src" >&2; exit 2; } base=3D$(pwd)/work.$$ build=3D$base/build inst=3D$base/install data=3D$base/data sock=3D$base/socket wal=3D$base/wal mkdir -p "$build" "$inst" "$data" "$sock" "$wal" cd "$build" "$src/configure" --prefix=3D"$inst" --with-lz4 make -j2 make install "$inst/bin/initdb" -D "$data" -A trust --no-locale cat >>"$data/postgresql.conf" </dev/null 2>&1 || true' EXIT HUP INT TERM export PGHOST=3D$sock PGDATABASE=3Dpostgres # Receive two whole segments. Repeated switches are used because the first # switch can finish the segment in which initdb/startup left the server. endpos=3D$("$inst/bin/psql" -AtX -c "SELECT '0/0'::pg_lsn + ((((pg_current_wal_insert_lsn()-'0/0')::bigint/16777216)+2)*16777216)") ( sleep 1 "$inst/bin/psql" -X -c 'SELECT pg_switch_wal()' >/dev/null "$inst/bin/psql" -X -c 'CREATE TABLE lz4_frame_probe(a text); INSERT INTO lz4_frame_probe SELECT repeat(md5(g::text),100) FROM generate_series(1,20000) g; SELECT pg_switch_wal(); DROP TABLE lz4_frame_probe; SELECT pg_switch_wal()' >/dev/null ) & producer=3D$! "$inst/bin/pg_receivewal" -D "$wal" -Z lz4 -E "$endpos" -n -v >"$base/first.log" 2>&1 wait "$producer" set -- "$wal"/????????????????????????.lz4 test "$#" -ge 2 || { echo "SETUP ERROR: fewer than two completed LZ4 segments"; exit 2; } highest=3D previous=3D for f in $(printf '%s\n' "$wal"/????????????????????????.lz4 | sort); do previous=3D$highest; highest=3D$f; done test -n "$previous" && test -n "$highest" before=3D$(wc -c <"$highest") truncate -s $((before - 1)) "$highest" # With the lower complete segment retained, rejection has an unambiguous # oracle: restart at the corrupted segment, not its successor. set +e "$inst/bin/pg_receivewal" -D "$wal" -Z lz4 -E 0/1 -n -v >"$base/second.log" 2>&1 rc=3D$? set -e cat "$base/second.log" highhex=3D$(basename "$highest" .lz4) logid=3D$(printf '%d' "0x$(printf '%s' "$highhex" | cut -c9-16)") segid=3D$(printf '%d' "0x$(printf '%s' "$highhex" | cut -c17-24)") nextno=3D$((logid * 256 + segid + 1)) ptr_hi=3D$((nextno / 256)) ptr_lo=3D$(((nextno % 256) * 16777216)) successor=3D$(printf '%X/%08X' "$ptr_hi" "$ptr_lo") if grep -q "starting log streaming at $successor" "$base/second.log"; then echo "BUG REPRODUCED: truncated $(basename "$highest") was accepted; rc=3D$rc" exit 1 fi echo "NOT REPRODUCED: truncated frame was not used as a completed segment; rc=3D$rc" exit 0 The affected behavior is reported when the second pg_receivewal invocation starts at the successor of the truncated highest segment and the script prints: BUG REPRODUCED: truncated .lz4 was accepted; rc=3D0 The script then exits with status 1. Exit status 2 indicates a setup failure. A corrected implementation should reject or skip the truncated frame, avoid starting at its successor, print `NOT REPRODUCED`, and exit with status 0. In the recorded PostgreSQL 18.6 run, a completed segment was truncated by one byte and pg_receivewal produced: TRUNCATED_FILE=3D00000001000000000000001A.lz4 BEFORE=3D140130 AFTER=3D140129 pg_receivewal: starting log streaming at 0/1B000000 (timeline 1) pg_receivewal: stopped log streaming at 0/1B000000 (timeline 1) SECOND_RC=3D0 Here 0/1B000000 is the boundary following the truncated 00000001000000000000001A.lz4 segment, showing that directory scanning classified the incomplete frame as complete. The issue was reproduced on PostgreSQL 18.6 on Linux/amd64 with LZ4 support. The LZ4 path used to find the streaming start counts decompressed bytes but does not require LZ4F_decompress() to return 0, the frame-completion signal. Consequently, truncating the frame footer can leave the decompressed size equal to WalSegSz and allow the file to be accepted. This occurs when pg_receivewal restarts from a directory whose highest completed-name LZ4 WAL file is truncated after producing a full segment but before frame completion.