From: Jan Nidzwetzki <jan@planetscale.com>
To: Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
To: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
To: ♂π≌26218 <1991230470@qq.com>
Cc: pgsql-bugs <pgsql-bugs@lists.postgresql.org>
Cc: Fujii Masao <masao.fujii@gmail.com>
Subject: Re: START_REPLICATION silently truncates an overlong LSN component
Date: Fri, 16 Oct 2026 15:03:59 +0200
Message-ID: <8b1abbd0-7335-4e68-93ec-91cdbbfd9568@planetscale.com> (raw)
In-Reply-To: <OS9PR01MB1214972D4AF13C0B23FB2129EF5BE2@OS9PR01MB12149.jpnprd01.prod.outlook.com>
References: <tencent_C6ED19DF1759CDD37E786F7EE0B5207D0009@qq.com>
<CAJTYsWWN82LQzaVPX4D-P_yhooMf18OZ_6uO7jxdbrMaq1KexA@mail.gmail.com>
<OS9PR01MB1214972D4AF13C0B23FB2129EF5BE2@OS9PR01MB12149.jpnprd01.prod.outlook.com>
Hi Ayush, Hayato,
Thanks for the patches. I looked into them, and both apply cleanly to
master, and check-world passes. So, these changes look good to me.
On 11.09.26 04:35, Hayato Kuroda (Fujitsu) wrote:
>> I wonder if we should make the same change in libpqwalreceiver too.
>
> I feel it should be, see attached. It could be combined with your patch.
> I grepped the source with the keyword "scanf\(.*%X/%08X", and I could
> not find other places where pg_parse_lsn() is usable as-is.
While looking for other places that need the same change, I found that
readTimeLineHistory() in src/backend/access/transam/timeline.c still
parses the switchpoint with sscanf("%X/%08X"). f31d6fbc31d fixed the
frontend copy of this code in pg_rewind, but not the backend one. The
attached 0003 patch converts it the same way and adds a TAP test to
004_timeline_switch.pl, including a case for the trailing characters
that are now rejected. As with the test in 0001, I'm not sure this one
needs to be committed.
In the attached v2 series, your patches are unchanged; I only added the
new 0003.
Best regards,
Jan
--
Jan Nidzwetzki
PlanetScale Postgres Core Team
Attachments:download all (3 files, .tar.gz)v2-0001-Use-pg_parse_lsn-for-START_REPLICATION.patch (text/plain, 2.5K) download
From 071838f48cdc12ce4ac6e4ac4a5eec4294276cbe Mon Sep 17 00:00:00 2001
From: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Date: Thu, 10 Sep 2026 19:02:10 +0530
Subject: [PATCH v2 1/3] Use pg_parse_lsn() for START_REPLICATION
The replication command scanner parses LSNs with sscanf() into uint32
variables. This can silently change an LSN with an overlong component:
the high component wraps around, while the low component is truncated to
the first eight digits because of the field width in "%08X".
Use pg_parse_lsn() instead. This enforces the same limit of one to eight
hexadecimal digits per component as pg_lsn input and rejects invalid LSNs
rather than starting replication at a different location.
Test overlong high and low components in the existing logical decoding
TAP test.
---
src/backend/replication/repl_scanner.l | 6 ++----
src/test/recovery/t/006_logical_decoding.pl | 12 ++++++++++++
2 files changed, 14 insertions(+), 4 deletions(-)
diff --git a/src/backend/replication/repl_scanner.l b/src/backend/replication/repl_scanner.lindex 70b137acedc..af1615f9dbf 100644--- a/src/backend/replication/repl_scanner.l+++ b/src/backend/replication/repl_scanner.l@@ -15,6 +15,7 @@
*/
#include "postgres.h"
+#include "common/pg_parse_lsn.h"
#include "nodes/parsenodes.h"
#include "utils/builtins.h"
#include "parser/scansup.h"
@@ -153,11 +154,8 @@ UPLOAD_MANIFEST { return K_UPLOAD_MANIFEST; }
}
{hexdigit}+\/{hexdigit}+ {
- uint32 hi,- lo;- if (sscanf(yytext, "%X/%08X", &hi, &lo) != 2)+ if (!pg_parse_lsn(yytext, &yylval->recptr))
replication_yyerror(NULL, yyscanner, "invalid streaming start location");
- yylval->recptr = ((uint64) hi) << 32 | lo;
return RECPTR;
}
diff --git a/src/test/recovery/t/006_logical_decoding.pl b/src/test/recovery/t/006_logical_decoding.plindex ab8364fee7a..d8adb4c3050 100644--- a/src/test/recovery/t/006_logical_decoding.pl+++ b/src/test/recovery/t/006_logical_decoding.pl@@ -45,6 +45,18 @@ like(
qr/replication slot "test_slot" was not created in this database/,
"Logical decoding correctly fails to start");
+for my $lsn ('123456789/0', '0/123456789')+{+ ($result, $stdout, $stderr) = $node_primary->psql(+ 'template1',+ qq[START_REPLICATION $lsn],+ replication => 'database');+ like(+ $stderr,+ qr/invalid streaming start location/,+ "START_REPLICATION rejects overlong LSN $lsn");+}+
($result, $stdout, $stderr) = $node_primary->psql(
'template1',
qq[READ_REPLICATION_SLOT test_slot;],
--
2.47.3
v2-0002-Use-pg_paser_lsn-in-libpqwalreceiver.patch (text/plain, 1.4K) download
From 635293f464a6d77fc287b596eb75be270dc41daf Mon Sep 17 00:00:00 2001
From: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Date: Fri, 11 Sep 2026 11:31:51 +0900
Subject: [PATCH v2 2/3] Use pg_paser_lsn in libpqwalreceiver
---
.../replication/libpqwalreceiver/libpqwalreceiver.c | 7 ++-----
1 file changed, 2 insertions(+), 5 deletions(-)
diff --git a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.cindex 029990d9fce..df47ae535ce 100644--- a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c+++ b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c@@ -23,6 +23,7 @@
#include <sys/time.h>
#include "common/connect.h"
+#include "common/pg_parse_lsn.h"
#include "funcapi.h"
#include "libpq-fe.h"
#include "libpq/libpq-be-fe-helpers.h"
@@ -458,15 +459,11 @@ libpqrcv_identify_system(WalReceiverConn *conn, TimeLineID *primary_tli,
/* Column 2 is the server's current WAL flush position */
if (server_lsn)
{
- uint32 hi,- lo;-- if (sscanf(PQgetvalue(res, 0, 2), "%X/%X", &hi, &lo) != 2)+ if (!pg_parse_lsn(PQgetvalue(res, 0, 2), server_lsn))
ereport(ERROR,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
errmsg("could not parse WAL location \"%s\"",
PQgetvalue(res, 0, 2))));
- *server_lsn = ((uint64) hi) << 32 | lo;
}
PQclear(res);
--
2.47.3
v2-0003-Use-pg_parse_lsn-for-timeline-history-switchpoint.patch (text/plain, 4.6K) download
From ed08aad4ef065d91560238ffb14c7f18976b7e15 Mon Sep 17 00:00:00 2001
From: Jan Nidzwetzki <jan@planetscale.com>
Date: Fri, 16 Oct 2026 14:24:05 +0200
Subject: [PATCH v2 3/3] Use pg_parse_lsn() for timeline history switchpoints
readTimeLineHistory() parsed the switchpoint on each history file line
with sscanf("%X/%08X"). Commit f31d6fbc31d converted the frontend copy
of this code in pg_rewind, but not the backend one. Convert it in the
same way. As in pg_rewind, characters directly after the switchpoint are
now rejected rather than ignored.
Add a TAP test covering overlong components and trailing characters.
---
src/backend/access/transam/timeline.c | 37 +++++++++++++++++-----
src/test/recovery/t/004_timeline_switch.pl | 28 ++++++++++++++++
2 files changed, 57 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/transam/timeline.c b/src/backend/access/transam/timeline.cindex d80c8ffe0a7..cc27587a07c 100644--- a/src/backend/access/transam/timeline.c+++ b/src/backend/access/transam/timeline.c@@ -39,6 +39,7 @@
#include "access/xlog_internal.h"
#include "access/xlogarchive.h"
#include "access/xlogdefs.h"
+#include "common/pg_parse_lsn.h"
#include "pgstat.h"
#include "storage/fd.h"
#include "utils/wait_event.h"
@@ -128,10 +129,13 @@ readTimeLineHistory(TimeLineID targetTLI)
char fline[MAXPGPATH];
char *res;
char *ptr;
+ char *token_end;+ char save;
TimeLineID tli;
- uint32 switchpoint_hi;- uint32 switchpoint_lo;- int nfields;+ XLogRecPtr switchpoint;+ bool valid_switchpoint = false;+ int nchars;+ size_t nspaces;
pgstat_report_wait_start(WAIT_EVENT_TIMELINE_HISTORY_READ);
res = fgets(fline, sizeof(fline), fd);
@@ -155,16 +159,33 @@ readTimeLineHistory(TimeLineID targetTLI)
if (*ptr == '\0' || *ptr == '#')
continue;
- nfields = sscanf(fline, "%u\t%X/%08X", &tli, &switchpoint_hi, &switchpoint_lo);-- if (nfields < 1)+ if (sscanf(fline, "%u%n", &tli, &nchars) != 1)
{
/* expect a numeric timeline ID as first field of line */
ereport(FATAL,
(errmsg("syntax error in history file: %s", fline),
errhint("Expected a numeric timeline ID.")));
}
- if (nfields != 3)++ /* the switchpoint location follows, separated by whitespace */+ ptr = fline + nchars;+ nspaces = strspn(ptr, " \t\n\r\f\v");+ if (nspaces > 0)+ {+ ptr += nspaces;++ /*+ * isolate the location from the rest of the line before parsing+ * it+ */+ token_end = ptr + strcspn(ptr, " \t\n\r\f\v");+ save = *token_end;+ *token_end = '\0';+ valid_switchpoint = pg_parse_lsn(ptr, &switchpoint);+ *token_end = save;+ }++ if (!valid_switchpoint)
ereport(FATAL,
(errmsg("syntax error in history file: %s", fline),
errhint("Expected a write-ahead log switchpoint location.")));
@@ -179,7 +200,7 @@ readTimeLineHistory(TimeLineID targetTLI)
entry = palloc_object(TimeLineHistoryEntry);
entry->tli = tli;
entry->begin = prevend;
- entry->end = ((uint64) (switchpoint_hi)) << 32 | (uint64) switchpoint_lo;+ entry->end = switchpoint;
prevend = entry->end;
/* Build list with newest item first */
diff --git a/src/test/recovery/t/004_timeline_switch.pl b/src/test/recovery/t/004_timeline_switch.plindex 4085d7d04d1..acb479a3b5c 100644--- a/src/test/recovery/t/004_timeline_switch.pl+++ b/src/test/recovery/t/004_timeline_switch.pl@@ -205,4 +205,32 @@ my $temp_slot_name = $node_standby_4->safe_psql('postgres',
like($temp_slot_name, qr/^pg_walreceiver_[0-9]+$/,
'pg_stat_wal_receiver.slot_name remains set across timeline jumps');
+# Check that a history file with a bogus switchpoint is rejected.+my $node_bad_history = PostgreSQL::Test::Cluster->new('bad_history');+$node_bad_history->init(allows_streaming => 1);+# Start and stop the node once before switching it to standby mode. This is+# required so that the control file records the test settings.+$node_bad_history->start;+$node_bad_history->stop;+$node_bad_history->set_standby_mode;+$node_bad_history->append_conf('postgresql.conf',+ "recovery_target_timeline = '2'");++for my $switchpoint ('123456789/0', '0/123456789', '0/1x')+{+ my $history_file =+ $node_bad_history->data_dir . '/pg_wal/00000002.history';+ my $log_offset = -s $node_bad_history->logfile;++ unlink $history_file;+ append_to_file($history_file, "1\t$switchpoint\tbogus switchpoint\n");++ ok(!$node_bad_history->start(fail_ok => 1),+ "server fails to start with history switchpoint $switchpoint");+ ok( $node_bad_history->log_contains(+ qr/FATAL: .*syntax error in history file: 1\t\Q$switchpoint\E/,+ $log_offset),+ "history switchpoint $switchpoint is rejected");+}+
done_testing();
--
2.47.3
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-bugs@postgresql.org
Cc: jan@planetscale.com, kuroda.hayato@fujitsu.com, ayushtiwari.slg01@gmail.com, 1991230470@qq.com, masao.fujii@gmail.com
Subject: Re: START_REPLICATION silently truncates an overlong LSN component
In-Reply-To: <8b1abbd0-7335-4e68-93ec-91cdbbfd9568@planetscale.com>
References: <tencent_C6ED19DF1759CDD37E786F7EE0B5207D0009@qq.com> <CAJTYsWWN82LQzaVPX4D-P_yhooMf18OZ_6uO7jxdbrMaq1KexA@mail.gmail.com> <OS9PR01MB1214972D4AF13C0B23FB2129EF5BE2@OS9PR01MB12149.jpnprd01.prod.outlook.com> <8b1abbd0-7335-4e68-93ec-91cdbbfd9568@planetscale.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox