From: rahul@rhyadav.dev
To: Pgsql Hackers <pgsql-hackers@lists.postgresql.org>
Cc: Thomas Munro <thomas.munro@gmail.com>
Cc: Ju Grigorev <ju.grigorev@ftdata.ru>
Subject: Wrong LSN in WAL decoding error messages
Date: Thu, 1 Oct 2026 09:17:46 +0200 (CEST)
Message-ID: <P2qIkq4--F-9@rhyadav.dev> (raw)
Hi,
DecodeXLogRecord() is passed the LSN of the record it decodes, but its
error messages print state->ReadRecPtr instead. Before the circular
WAL decoding buffer went in (3f1ce97346, v15), the two were the same.
Now records are decoded ahead of the one the reader last returned, so
these messages name an earlier record. 0001 makes them use the lsn
argument.
I noticed this while testing Yuriy's fix for bug #19599 [1], and he
suggested sending it separately. These checks only see records that
passed the CRC check, so they fire when some code writes a malformed
record, which is when you most want the right LSN.
To trigger one, I set hole_offset to 0 in a full-page image that has
BKPIMAGE_HAS_HOLE set, and recomputed the record CRC. For that
record, at 0/01C03DC0, master reports:
pg_waldump: error in WAL record at 0/01C03D38: BKPIMAGE_HAS_HOLE
set, but hole offset 0 length 4208 block image length 3984 at
0/01C03D38
crash recovery: BKPIMAGE_HAS_HOLE set, but hole offset 0 length
4208 block image length 3984 at 0/01C03CD8
0/01C03D38 is the record just before it and 0/01C03CD8 is three
records back. With the patches applied, both report 0/01C03DC0, and
crash recovery still ends at the same place.
0002 is separate because it changes long-standing output.
pg_waldump's "error in WAL record at" prefix also uses ReadRecPtr,
which is the last record read successfully rather than where reading
failed. At the normal end of WAL that gives:
error in WAL record at 0/017AC370: invalid record length at
0/017AC3F8: expected at least 24, got 0
With 0002 both LSNs are 0/017AC3F8. It uses EndRecPtr, as pg_rewind
does. v14 behaves the same at the end of WAL, so if 0002 is wanted
at all I'd keep it to master. I can add a check to pg_waldump's TAP
test that the two LSNs match.
0001 applies to REL_19_STABLE as is. On 15 to 18 the messages still
use %X/%X, so the context needs a small adjustment.
[1] https://postgr.es/m/cd00a0536c674fac9e5dd9f4e27ae868@localhost.localdomain
Regards,
Rahul
Attachments:download all (2 files, .tar.gz)v1-0002-pg_waldump-Report-the-position-of-a-WAL-read-erro.patch (application/octet-stream, 1.4K) download
From 27363ec05340a56435843345ac75585fb7521dfa Mon Sep 17 00:00:00 2001
From: Rahul Yadav <rahul@rhyadav.dev>
Date: Thu, 1 Oct 2026 07:46:25 +0100
Subject: [PATCH v1 2/2] pg_waldump: Report the position of a WAL read error
When reading a record fails, pg_waldump prints "error in WAL record at
%X/%08X" followed by the reader's message, using ReadRecPtr for the
position. ReadRecPtr is the start of the last record read
successfully, not the place where reading failed. At the end of WAL,
the most common case, the prefix names the last valid record while the
message names the end of WAL:
error in WAL record at 0/017AC370: invalid record length at
0/017AC3F8: expected at least 24, got 0
Since 3f1ce97346 the same happens when a record is rejected while it
is being decoded.
Use EndRecPtr instead, the position where the failing record starts,
as pg_rewind already does.
Author: Rahul Yadav <rahul@rhyadav.dev>
---
src/bin/pg_waldump/pg_waldump.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/bin/pg_waldump/pg_waldump.c b/src/bin/pg_waldump/pg_waldump.cindex 6f0574a876..34ca41a7e6 100644--- a/src/bin/pg_waldump/pg_waldump.c+++ b/src/bin/pg_waldump/pg_waldump.c@@ -1521,7 +1521,7 @@ main(int argc, char **argv)
if (errormsg)
pg_fatal("error in WAL record at %X/%08X: %s",
- LSN_FORMAT_ARGS(xlogreader_state->ReadRecPtr),+ LSN_FORMAT_ARGS(xlogreader_state->EndRecPtr),
errormsg);
/*
--
2.50.1 (Apple Git-155)
v1-0001-Report-the-decoded-record-s-LSN-in-DecodeXLogReco.patch (application/octet-stream, 4.4K) download
From 542ff1da0f42ad21189418f33db68bf618a873e0 Mon Sep 17 00:00:00 2001
From: Rahul Yadav <rahul@rhyadav.dev>
Date: Thu, 1 Oct 2026 07:46:25 +0100
Subject: [PATCH v1 1/2] Report the decoded record's LSN in DecodeXLogRecord()
errors
DecodeXLogRecord() is passed the LSN of the record it decodes, but its
error messages printed state->ReadRecPtr instead. Before 3f1ce97346
added the circular WAL decoding buffer, the two were the same, since
XLogReadRecord() set ReadRecPtr before decoding. Now records are
decoded ahead of the one the reader last returned, so these messages
named an earlier record: in pg_waldump the record just before the
failing one, and in crash recovery, which reads further ahead, one
several records back.
Use the lsn argument instead. The only other caller, the WAL_DEBUG
code in XLogInsertRecord(), passes the end of the record it has just
inserted, which matches the LSN that code logs for it.
Noticed while testing a fix for bug #19599.
Author: Rahul Yadav <rahul@rhyadav.dev>
Backpatch-through: 15
---
src/backend/access/transam/xlogreader.c | 20 ++++++++++----------
1 file changed, 10 insertions(+), 10 deletions(-)
diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.cindex 7db7c273b0..6573270dab 100644--- a/src/backend/access/transam/xlogreader.c+++ b/src/backend/access/transam/xlogreader.c@@ -1818,7 +1818,7 @@ DecodeXLogRecord(XLogReaderState *state,
report_invalid_record(state,
"out-of-order block_id %u at %X/%08X",
block_id,
- LSN_FORMAT_ARGS(state->ReadRecPtr));+ LSN_FORMAT_ARGS(lsn));
goto err;
}
decoded->max_block_id = block_id;
@@ -1841,7 +1841,7 @@ DecodeXLogRecord(XLogReaderState *state,
{
report_invalid_record(state,
"BKPBLOCK_HAS_DATA set, but no data included at %X/%08X",
- LSN_FORMAT_ARGS(state->ReadRecPtr));+ LSN_FORMAT_ARGS(lsn));
goto err;
}
if (!blk->has_data && blk->data_len != 0)
@@ -1849,7 +1849,7 @@ DecodeXLogRecord(XLogReaderState *state,
report_invalid_record(state,
"BKPBLOCK_HAS_DATA not set, but data length is %d at %X/%08X",
blk->data_len,
- LSN_FORMAT_ARGS(state->ReadRecPtr));+ LSN_FORMAT_ARGS(lsn));
goto err;
}
datatotal += blk->data_len;
@@ -1887,7 +1887,7 @@ DecodeXLogRecord(XLogReaderState *state,
blk->hole_offset,
blk->hole_length,
blk->bimg_len,
- LSN_FORMAT_ARGS(state->ReadRecPtr));+ LSN_FORMAT_ARGS(lsn));
goto err;
}
@@ -1902,7 +1902,7 @@ DecodeXLogRecord(XLogReaderState *state,
"BKPIMAGE_HAS_HOLE not set, but hole offset %d length %d at %X/%08X",
blk->hole_offset,
blk->hole_length,
- LSN_FORMAT_ARGS(state->ReadRecPtr));+ LSN_FORMAT_ARGS(lsn));
goto err;
}
@@ -1915,7 +1915,7 @@ DecodeXLogRecord(XLogReaderState *state,
report_invalid_record(state,
"BKPIMAGE_COMPRESSED set, but block image length %d at %X/%08X",
blk->bimg_len,
- LSN_FORMAT_ARGS(state->ReadRecPtr));+ LSN_FORMAT_ARGS(lsn));
goto err;
}
@@ -1930,7 +1930,7 @@ DecodeXLogRecord(XLogReaderState *state,
report_invalid_record(state,
"neither BKPIMAGE_HAS_HOLE nor BKPIMAGE_COMPRESSED set, but block image length is %d at %X/%08X",
blk->bimg_len,
- LSN_FORMAT_ARGS(state->ReadRecPtr));+ LSN_FORMAT_ARGS(lsn));
goto err;
}
}
@@ -1945,7 +1945,7 @@ DecodeXLogRecord(XLogReaderState *state,
{
report_invalid_record(state,
"BKPBLOCK_SAME_REL set but no previous rel at %X/%08X",
- LSN_FORMAT_ARGS(state->ReadRecPtr));+ LSN_FORMAT_ARGS(lsn));
goto err;
}
@@ -1957,7 +1957,7 @@ DecodeXLogRecord(XLogReaderState *state,
{
report_invalid_record(state,
"invalid block_id %u at %X/%08X",
- block_id, LSN_FORMAT_ARGS(state->ReadRecPtr));+ block_id, LSN_FORMAT_ARGS(lsn));
goto err;
}
}
@@ -2024,7 +2024,7 @@ DecodeXLogRecord(XLogReaderState *state,
shortdata_err:
report_invalid_record(state,
"record with invalid length at %X/%08X",
- LSN_FORMAT_ARGS(state->ReadRecPtr));+ LSN_FORMAT_ARGS(lsn));
err:
*errormsg = state->errormsg_buf;
--
2.50.1 (Apple Git-155)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-hackers@postgresql.org
Cc: rahul@rhyadav.dev, thomas.munro@gmail.com, ju.grigorev@ftdata.ru
Subject: Re: Wrong LSN in WAL decoding error messages
In-Reply-To: <P2qIkq4--F-9@rhyadav.dev>
References: <P2qIkq4--F-9@rhyadav.dev>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox