pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: rahul@rhyadav.dev
To: Pgsql Hackers <pgsql-hackers@lists.postgresql.org>
Cc: Thomas Munro <thomas.munro@gmail.com>
Cc: Ju Grigorev <ju.grigorev@ftdata.ru>
Subject: Wrong LSN in WAL decoding error messages
Date: Thu, 1 Oct 2026 09:17:46 +0200 (CEST)
Message-ID: <P2qIkq4--F-9@rhyadav.dev> (raw)


Hi,

DecodeXLogRecord() is passed the LSN of the record it decodes, but its
error messages print state->ReadRecPtr instead.  Before the circular
WAL decoding buffer went in (3f1ce97346, v15), the two were the same.
Now records are decoded ahead of the one the reader last returned, so
these messages name an earlier record.  0001 makes them use the lsn
argument.

I noticed this while testing Yuriy's fix for bug #19599 [1], and he
suggested sending it separately.  These checks only see records that
passed the CRC check, so they fire when some code writes a malformed
record, which is when you most want the right LSN.

To trigger one, I set hole_offset to 0 in a full-page image that has
BKPIMAGE_HAS_HOLE set, and recomputed the record CRC.  For that
record, at 0/01C03DC0, master reports:

  pg_waldump: error in WAL record at 0/01C03D38: BKPIMAGE_HAS_HOLE
    set, but hole offset 0 length 4208 block image length 3984 at
    0/01C03D38

  crash recovery: BKPIMAGE_HAS_HOLE set, but hole offset 0 length
    4208 block image length 3984 at 0/01C03CD8

0/01C03D38 is the record just before it and 0/01C03CD8 is three
records back.  With the patches applied, both report 0/01C03DC0, and
crash recovery still ends at the same place.

0002 is separate because it changes long-standing output.
pg_waldump's "error in WAL record at" prefix also uses ReadRecPtr,
which is the last record read successfully rather than where reading
failed.  At the normal end of WAL that gives:

  error in WAL record at 0/017AC370: invalid record length at
  0/017AC3F8: expected at least 24, got 0

With 0002 both LSNs are 0/017AC3F8.  It uses EndRecPtr, as pg_rewind
does.  v14 behaves the same at the end of WAL, so if 0002 is wanted
at all I'd keep it to master.  I can add a check to pg_waldump's TAP
test that the two LSNs match.

0001 applies to REL_19_STABLE as is.  On 15 to 18 the messages still
use %X/%X, so the context needs a small adjustment.

[1] https://postgr.es/m/cd00a0536c674fac9e5dd9f4e27ae868@localhost.localdomain


Regards,
Rahul

Attachments: download all (2 files, .tar.gz)

v1-0002-pg_waldump-Report-the-position-of-a-WAL-read-erro.patch (application/octet-stream, 1.4K) download From 27363ec05340a56435843345ac75585fb7521dfa Mon Sep 17 00:00:00 2001 From: Rahul Yadav <rahul@rhyadav.dev> Date: Thu, 1 Oct 2026 07:46:25 +0100 Subject: [PATCH v1 2/2] pg_waldump: Report the position of a WAL read error When reading a record fails, pg_waldump prints "error in WAL record at %X/%08X" followed by the reader's message, using ReadRecPtr for the position. ReadRecPtr is the start of the last record read successfully, not the place where reading failed. At the end of WAL, the most common case, the prefix names the last valid record while the message names the end of WAL: error in WAL record at 0/017AC370: invalid record length at 0/017AC3F8: expected at least 24, got 0 Since 3f1ce97346 the same happens when a record is rejected while it is being decoded. Use EndRecPtr instead, the position where the failing record starts, as pg_rewind already does. Author: Rahul Yadav <rahul@rhyadav.dev> --- src/bin/pg_waldump/pg_waldump.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/bin/pg_waldump/pg_waldump.c b/src/bin/pg_waldump/pg_waldump.c index 6f0574a876..34ca41a7e6 100644 --- a/src/bin/pg_waldump/pg_waldump.c +++ b/src/bin/pg_waldump/pg_waldump.c @@ -1521,7 +1521,7 @@ main(int argc, char **argv) if (errormsg) pg_fatal("error in WAL record at %X/%08X: %s", - LSN_FORMAT_ARGS(xlogreader_state->ReadRecPtr), + LSN_FORMAT_ARGS(xlogreader_state->EndRecPtr), errormsg); /* -- 2.50.1 (Apple Git-155)
v1-0001-Report-the-decoded-record-s-LSN-in-DecodeXLogReco.patch (application/octet-stream, 4.4K) download From 542ff1da0f42ad21189418f33db68bf618a873e0 Mon Sep 17 00:00:00 2001 From: Rahul Yadav <rahul@rhyadav.dev> Date: Thu, 1 Oct 2026 07:46:25 +0100 Subject: [PATCH v1 1/2] Report the decoded record's LSN in DecodeXLogRecord() errors DecodeXLogRecord() is passed the LSN of the record it decodes, but its error messages printed state->ReadRecPtr instead. Before 3f1ce97346 added the circular WAL decoding buffer, the two were the same, since XLogReadRecord() set ReadRecPtr before decoding. Now records are decoded ahead of the one the reader last returned, so these messages named an earlier record: in pg_waldump the record just before the failing one, and in crash recovery, which reads further ahead, one several records back. Use the lsn argument instead. The only other caller, the WAL_DEBUG code in XLogInsertRecord(), passes the end of the record it has just inserted, which matches the LSN that code logs for it. Noticed while testing a fix for bug #19599. Author: Rahul Yadav <rahul@rhyadav.dev> Backpatch-through: 15 --- src/backend/access/transam/xlogreader.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c index 7db7c273b0..6573270dab 100644 --- a/src/backend/access/transam/xlogreader.c +++ b/src/backend/access/transam/xlogreader.c @@ -1818,7 +1818,7 @@ DecodeXLogRecord(XLogReaderState *state, report_invalid_record(state, "out-of-order block_id %u at %X/%08X", block_id, - LSN_FORMAT_ARGS(state->ReadRecPtr)); + LSN_FORMAT_ARGS(lsn)); goto err; } decoded->max_block_id = block_id; @@ -1841,7 +1841,7 @@ DecodeXLogRecord(XLogReaderState *state, { report_invalid_record(state, "BKPBLOCK_HAS_DATA set, but no data included at %X/%08X", - LSN_FORMAT_ARGS(state->ReadRecPtr)); + LSN_FORMAT_ARGS(lsn)); goto err; } if (!blk->has_data && blk->data_len != 0) @@ -1849,7 +1849,7 @@ DecodeXLogRecord(XLogReaderState *state, report_invalid_record(state, "BKPBLOCK_HAS_DATA not set, but data length is %d at %X/%08X", blk->data_len, - LSN_FORMAT_ARGS(state->ReadRecPtr)); + LSN_FORMAT_ARGS(lsn)); goto err; } datatotal += blk->data_len; @@ -1887,7 +1887,7 @@ DecodeXLogRecord(XLogReaderState *state, blk->hole_offset, blk->hole_length, blk->bimg_len, - LSN_FORMAT_ARGS(state->ReadRecPtr)); + LSN_FORMAT_ARGS(lsn)); goto err; } @@ -1902,7 +1902,7 @@ DecodeXLogRecord(XLogReaderState *state, "BKPIMAGE_HAS_HOLE not set, but hole offset %d length %d at %X/%08X", blk->hole_offset, blk->hole_length, - LSN_FORMAT_ARGS(state->ReadRecPtr)); + LSN_FORMAT_ARGS(lsn)); goto err; } @@ -1915,7 +1915,7 @@ DecodeXLogRecord(XLogReaderState *state, report_invalid_record(state, "BKPIMAGE_COMPRESSED set, but block image length %d at %X/%08X", blk->bimg_len, - LSN_FORMAT_ARGS(state->ReadRecPtr)); + LSN_FORMAT_ARGS(lsn)); goto err; } @@ -1930,7 +1930,7 @@ DecodeXLogRecord(XLogReaderState *state, report_invalid_record(state, "neither BKPIMAGE_HAS_HOLE nor BKPIMAGE_COMPRESSED set, but block image length is %d at %X/%08X", blk->bimg_len, - LSN_FORMAT_ARGS(state->ReadRecPtr)); + LSN_FORMAT_ARGS(lsn)); goto err; } } @@ -1945,7 +1945,7 @@ DecodeXLogRecord(XLogReaderState *state, { report_invalid_record(state, "BKPBLOCK_SAME_REL set but no previous rel at %X/%08X", - LSN_FORMAT_ARGS(state->ReadRecPtr)); + LSN_FORMAT_ARGS(lsn)); goto err; } @@ -1957,7 +1957,7 @@ DecodeXLogRecord(XLogReaderState *state, { report_invalid_record(state, "invalid block_id %u at %X/%08X", - block_id, LSN_FORMAT_ARGS(state->ReadRecPtr)); + block_id, LSN_FORMAT_ARGS(lsn)); goto err; } } @@ -2024,7 +2024,7 @@ DecodeXLogRecord(XLogReaderState *state, shortdata_err: report_invalid_record(state, "record with invalid length at %X/%08X", - LSN_FORMAT_ARGS(state->ReadRecPtr)); + LSN_FORMAT_ARGS(lsn)); err: *errormsg = state->errormsg_buf; -- 2.50.1 (Apple Git-155)

view thread (9+ messages)  latest in thread

Message-ID: <P2qIkq4--F-9@rhyadav.dev>
Permalink:  ../P2qIkq4--F-9@rhyadav.dev/
Also on:    postgresql.org/message-id/P2qIkq4--F-9@rhyadav.dev

 ·  · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-hackers@postgresql.org
  Cc: rahul@rhyadav.dev, thomas.munro@gmail.com, ju.grigorev@ftdata.ru
  Subject: Re: Wrong LSN in WAL decoding error messages
  In-Reply-To: <P2qIkq4--F-9@rhyadav.dev>
  References: <P2qIkq4--F-9@rhyadav.dev>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox