agora inbox for pgsql-committers@postgresql.orghelp / color / mirror / Atom feed
pgsql: Fix cascading standby reconnect failure after archive fallback 7+ messages / 1 participants [nested] [flat]
* pgsql: Fix cascading standby reconnect failure after archive fallback @ 2026-07-29 15:17 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 7+ messages in thread From: Álvaro Herrera @ 2026-07-29 15:17 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix cascading standby reconnect failure after archive fallback A cascading standby could fail to reconnect to its upstream standby with "requested starting point ... is ahead of the WAL flush position" after falling back to archive recovery. This happened because archive recovery processes whole segment files, so after replaying a segment the cascade's next read position lands at the start of the following segment, which is ahead of the upstream's flush position reported by GetStandbyFlushRecPtr() (still inside the just-replayed segment). Fix by having the walreceiver check the upstream's current WAL flush position via IDENTIFY_SYSTEM before issuing START_REPLICATION. IDENTIFY_SYSTEM already returns this position (as xlogpos), but walrcv_identify_system() previously discarded it; now we have a use for it. If the requested start point exceeds the upstream's flush position on the same timeline, the walreceiver waits for wal_retrieve_retry_interval and retries. The wait is limited to gaps of at most one WAL segment, which is the expected case from the segment-granularity of archive recovery. Larger gaps indicate the upstream is genuinely behind, so START_REPLICATION is allowed to proceed (and fail) normally, letting the startup process fall back to other WAL sources. The first wait is logged at LOG level; subsequent waits are demoted to DEBUG1 to avoid log noise. The walreceiver honors wal_receiver_timeout during the wait, so it will exit if the upstream doesn't catch up in time. To preserve ABI compatibility on back branches, the flush position from IDENTIFY_SYSTEM is communicated via a new global variable (WalRcvIdentifySystemLsn) rather than changing the signature of walrcv_identify_system(). The bug was introduced in Postgres 9.3 by commit abfd192b1b5b, which added a flush-position check in StartReplication() that rejects requests ahead of the upstream server's WAL flush position. Author: Marco Nenciarini <marco.nenciarini@enterprisedb.com> Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com> Backpatch-through: 14 Discussion: https://postgr.es/m/CA+nrD2cTuTkkX5WXVZengTYYZbAO6zV8K+Tri-R0fbLFuoyMBA@mail.gmail.com Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/b614de4876bb4e4264a99ec4c6f2582ad4e0ca8d Modified Files -------------- .../libpqwalreceiver/libpqwalreceiver.c | 21 ++- src/backend/replication/logical/worker.c | 2 +- src/backend/replication/walreceiver.c | 72 +++++++++- src/backend/utils/activity/wait_event_names.txt | 1 + src/include/replication/walreceiver.h | 9 +- src/test/recovery/meson.build | 1 + src/test/recovery/t/055_cascade_reconnect.pl | 148 +++++++++++++++++++++ 7 files changed, 247 insertions(+), 7 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix cascading standby reconnect failure after archive fallback @ 2026-07-29 15:17 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 7+ messages in thread From: Álvaro Herrera @ 2026-07-29 15:17 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix cascading standby reconnect failure after archive fallback A cascading standby could fail to reconnect to its upstream standby with "requested starting point ... is ahead of the WAL flush position" after falling back to archive recovery. This happened because archive recovery processes whole segment files, so after replaying a segment the cascade's next read position lands at the start of the following segment, which is ahead of the upstream's flush position reported by GetStandbyFlushRecPtr() (still inside the just-replayed segment). Fix by having the walreceiver check the upstream's current WAL flush position via IDENTIFY_SYSTEM before issuing START_REPLICATION. IDENTIFY_SYSTEM already returns this position (as xlogpos), but walrcv_identify_system() previously discarded it; now we have a use for it. If the requested start point exceeds the upstream's flush position on the same timeline, the walreceiver waits for wal_retrieve_retry_interval and retries. The wait is limited to gaps of at most one WAL segment, which is the expected case from the segment-granularity of archive recovery. Larger gaps indicate the upstream is genuinely behind, so START_REPLICATION is allowed to proceed (and fail) normally, letting the startup process fall back to other WAL sources. The first wait is logged at LOG level; subsequent waits are demoted to DEBUG1 to avoid log noise. The walreceiver honors wal_receiver_timeout during the wait, so it will exit if the upstream doesn't catch up in time. To preserve ABI compatibility on back branches, the flush position from IDENTIFY_SYSTEM is communicated via a new global variable (WalRcvIdentifySystemLsn) rather than changing the signature of walrcv_identify_system(). The bug was introduced in Postgres 9.3 by commit abfd192b1b5b, which added a flush-position check in StartReplication() that rejects requests ahead of the upstream server's WAL flush position. Author: Marco Nenciarini <marco.nenciarini@enterprisedb.com> Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com> Backpatch-through: 14 Discussion: https://postgr.es/m/CA+nrD2cTuTkkX5WXVZengTYYZbAO6zV8K+Tri-R0fbLFuoyMBA@mail.gmail.com Branch ------ REL_19_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/bef46aed394917be95f6572f64af90e24b1c0652 Modified Files -------------- .../libpqwalreceiver/libpqwalreceiver.c | 21 ++- src/backend/replication/logical/worker.c | 2 +- src/backend/replication/walreceiver.c | 72 +++++++++- src/backend/utils/activity/wait_event_names.txt | 1 + src/include/replication/walreceiver.h | 9 +- src/test/recovery/meson.build | 1 + src/test/recovery/t/055_cascade_reconnect.pl | 148 +++++++++++++++++++++ 7 files changed, 247 insertions(+), 7 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix cascading standby reconnect failure after archive fallback @ 2026-07-29 15:17 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 7+ messages in thread From: Álvaro Herrera @ 2026-07-29 15:17 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix cascading standby reconnect failure after archive fallback A cascading standby could fail to reconnect to its upstream standby with "requested starting point ... is ahead of the WAL flush position" after falling back to archive recovery. This happened because archive recovery processes whole segment files, so after replaying a segment the cascade's next read position lands at the start of the following segment, which is ahead of the upstream's flush position reported by GetStandbyFlushRecPtr() (still inside the just-replayed segment). Fix by having the walreceiver check the upstream's current WAL flush position via IDENTIFY_SYSTEM before issuing START_REPLICATION. IDENTIFY_SYSTEM already returns this position (as xlogpos), but walrcv_identify_system() previously discarded it; now we have a use for it. If the requested start point exceeds the upstream's flush position on the same timeline, the walreceiver waits for wal_retrieve_retry_interval and retries. The wait is limited to gaps of at most one WAL segment, which is the expected case from the segment-granularity of archive recovery. Larger gaps indicate the upstream is genuinely behind, so START_REPLICATION is allowed to proceed (and fail) normally, letting the startup process fall back to other WAL sources. The first wait is logged at LOG level; subsequent waits are demoted to DEBUG1 to avoid log noise. The walreceiver honors wal_receiver_timeout during the wait, so it will exit if the upstream doesn't catch up in time. To preserve ABI compatibility on back branches, the flush position from IDENTIFY_SYSTEM is communicated via a new global variable (WalRcvIdentifySystemLsn) rather than changing the signature of walrcv_identify_system(). The bug was introduced in Postgres 9.3 by commit abfd192b1b5b, which added a flush-position check in StartReplication() that rejects requests ahead of the upstream server's WAL flush position. Author: Marco Nenciarini <marco.nenciarini@enterprisedb.com> Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com> Backpatch-through: 14 Discussion: https://postgr.es/m/CA+nrD2cTuTkkX5WXVZengTYYZbAO6zV8K+Tri-R0fbLFuoyMBA@mail.gmail.com Branch ------ REL_18_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/33101632235ad064b2bd7bc04a5066048dc48023 Modified Files -------------- .../libpqwalreceiver/libpqwalreceiver.c | 14 ++ src/backend/replication/walreceiver.c | 79 ++++++++++- src/backend/utils/activity/wait_event_names.txt | 1 + src/include/replication/walreceiver.h | 7 + src/test/recovery/meson.build | 1 + src/test/recovery/t/055_cascade_reconnect.pl | 148 +++++++++++++++++++++ 6 files changed, 249 insertions(+), 1 deletion(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix cascading standby reconnect failure after archive fallback @ 2026-07-29 15:17 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 7+ messages in thread From: Álvaro Herrera @ 2026-07-29 15:17 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix cascading standby reconnect failure after archive fallback A cascading standby could fail to reconnect to its upstream standby with "requested starting point ... is ahead of the WAL flush position" after falling back to archive recovery. This happened because archive recovery processes whole segment files, so after replaying a segment the cascade's next read position lands at the start of the following segment, which is ahead of the upstream's flush position reported by GetStandbyFlushRecPtr() (still inside the just-replayed segment). Fix by having the walreceiver check the upstream's current WAL flush position via IDENTIFY_SYSTEM before issuing START_REPLICATION. IDENTIFY_SYSTEM already returns this position (as xlogpos), but walrcv_identify_system() previously discarded it; now we have a use for it. If the requested start point exceeds the upstream's flush position on the same timeline, the walreceiver waits for wal_retrieve_retry_interval and retries. The wait is limited to gaps of at most one WAL segment, which is the expected case from the segment-granularity of archive recovery. Larger gaps indicate the upstream is genuinely behind, so START_REPLICATION is allowed to proceed (and fail) normally, letting the startup process fall back to other WAL sources. The first wait is logged at LOG level; subsequent waits are demoted to DEBUG1 to avoid log noise. The walreceiver honors wal_receiver_timeout during the wait, so it will exit if the upstream doesn't catch up in time. To preserve ABI compatibility on back branches, the flush position from IDENTIFY_SYSTEM is communicated via a new global variable (WalRcvIdentifySystemLsn) rather than changing the signature of walrcv_identify_system(). The bug was introduced in Postgres 9.3 by commit abfd192b1b5b, which added a flush-position check in StartReplication() that rejects requests ahead of the upstream server's WAL flush position. Author: Marco Nenciarini <marco.nenciarini@enterprisedb.com> Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com> Backpatch-through: 14 Discussion: https://postgr.es/m/CA+nrD2cTuTkkX5WXVZengTYYZbAO6zV8K+Tri-R0fbLFuoyMBA@mail.gmail.com Branch ------ REL_17_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/2cf28d1b9a273ba5627a04066746726bf3e54662 Modified Files -------------- .../libpqwalreceiver/libpqwalreceiver.c | 14 ++ src/backend/replication/walreceiver.c | 79 ++++++++++- src/backend/utils/activity/wait_event_names.txt | 1 + src/include/replication/walreceiver.h | 7 + src/test/recovery/meson.build | 1 + src/test/recovery/t/055_cascade_reconnect.pl | 148 +++++++++++++++++++++ 6 files changed, 249 insertions(+), 1 deletion(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix cascading standby reconnect failure after archive fallback @ 2026-07-29 15:17 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 7+ messages in thread From: Álvaro Herrera @ 2026-07-29 15:17 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix cascading standby reconnect failure after archive fallback A cascading standby could fail to reconnect to its upstream standby with "requested starting point ... is ahead of the WAL flush position" after falling back to archive recovery. This happened because archive recovery processes whole segment files, so after replaying a segment the cascade's next read position lands at the start of the following segment, which is ahead of the upstream's flush position reported by GetStandbyFlushRecPtr() (still inside the just-replayed segment). Fix by having the walreceiver check the upstream's current WAL flush position via IDENTIFY_SYSTEM before issuing START_REPLICATION. IDENTIFY_SYSTEM already returns this position (as xlogpos), but walrcv_identify_system() previously discarded it; now we have a use for it. If the requested start point exceeds the upstream's flush position on the same timeline, the walreceiver waits for wal_retrieve_retry_interval and retries. The wait is limited to gaps of at most one WAL segment, which is the expected case from the segment-granularity of archive recovery. Larger gaps indicate the upstream is genuinely behind, so START_REPLICATION is allowed to proceed (and fail) normally, letting the startup process fall back to other WAL sources. The first wait is logged at LOG level; subsequent waits are demoted to DEBUG1 to avoid log noise. The walreceiver honors wal_receiver_timeout during the wait, so it will exit if the upstream doesn't catch up in time. To preserve ABI compatibility on back branches, the flush position from IDENTIFY_SYSTEM is communicated via a new global variable (WalRcvIdentifySystemLsn) rather than changing the signature of walrcv_identify_system(). The bug was introduced in Postgres 9.3 by commit abfd192b1b5b, which added a flush-position check in StartReplication() that rejects requests ahead of the upstream server's WAL flush position. Author: Marco Nenciarini <marco.nenciarini@enterprisedb.com> Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com> Backpatch-through: 14 Discussion: https://postgr.es/m/CA+nrD2cTuTkkX5WXVZengTYYZbAO6zV8K+Tri-R0fbLFuoyMBA@mail.gmail.com Branch ------ REL_16_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/5dbeb69bcb0313f20731f43b682e04a663f39d27 Modified Files -------------- .../libpqwalreceiver/libpqwalreceiver.c | 14 ++ src/backend/replication/walreceiver.c | 79 ++++++++++- src/backend/utils/activity/wait_event.c | 3 + src/backend/utils/adt/formatting.c | 21 ++- src/include/replication/walreceiver.h | 7 + src/include/utils/wait_event.h | 3 +- src/test/recovery/meson.build | 1 + src/test/recovery/t/055_cascade_reconnect.pl | 154 +++++++++++++++++++++ 8 files changed, 277 insertions(+), 5 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix cascading standby reconnect failure after archive fallback @ 2026-07-29 15:17 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 7+ messages in thread From: Álvaro Herrera @ 2026-07-29 15:17 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix cascading standby reconnect failure after archive fallback A cascading standby could fail to reconnect to its upstream standby with "requested starting point ... is ahead of the WAL flush position" after falling back to archive recovery. This happened because archive recovery processes whole segment files, so after replaying a segment the cascade's next read position lands at the start of the following segment, which is ahead of the upstream's flush position reported by GetStandbyFlushRecPtr() (still inside the just-replayed segment). Fix by having the walreceiver check the upstream's current WAL flush position via IDENTIFY_SYSTEM before issuing START_REPLICATION. IDENTIFY_SYSTEM already returns this position (as xlogpos), but walrcv_identify_system() previously discarded it; now we have a use for it. If the requested start point exceeds the upstream's flush position on the same timeline, the walreceiver waits for wal_retrieve_retry_interval and retries. The wait is limited to gaps of at most one WAL segment, which is the expected case from the segment-granularity of archive recovery. Larger gaps indicate the upstream is genuinely behind, so START_REPLICATION is allowed to proceed (and fail) normally, letting the startup process fall back to other WAL sources. The first wait is logged at LOG level; subsequent waits are demoted to DEBUG1 to avoid log noise. The walreceiver honors wal_receiver_timeout during the wait, so it will exit if the upstream doesn't catch up in time. To preserve ABI compatibility on back branches, the flush position from IDENTIFY_SYSTEM is communicated via a new global variable (WalRcvIdentifySystemLsn) rather than changing the signature of walrcv_identify_system(). The bug was introduced in Postgres 9.3 by commit abfd192b1b5b, which added a flush-position check in StartReplication() that rejects requests ahead of the upstream server's WAL flush position. Author: Marco Nenciarini <marco.nenciarini@enterprisedb.com> Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com> Backpatch-through: 14 Discussion: https://postgr.es/m/CA+nrD2cTuTkkX5WXVZengTYYZbAO6zV8K+Tri-R0fbLFuoyMBA@mail.gmail.com Branch ------ REL_15_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/8f64cc83f9ddde828e99c13301c10d573f399baa Modified Files -------------- .../libpqwalreceiver/libpqwalreceiver.c | 14 ++ src/backend/replication/walreceiver.c | 79 +++++++++- src/backend/utils/activity/wait_event.c | 3 + src/backend/utils/adt/formatting.c | 21 ++- src/include/replication/walreceiver.h | 7 + src/include/utils/wait_event.h | 3 +- src/test/recovery/t/055_cascade_reconnect.pl | 164 +++++++++++++++++++++ 7 files changed, 286 insertions(+), 5 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix cascading standby reconnect failure after archive fallback @ 2026-07-29 15:17 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 7+ messages in thread From: Álvaro Herrera @ 2026-07-29 15:17 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix cascading standby reconnect failure after archive fallback A cascading standby could fail to reconnect to its upstream standby with "requested starting point ... is ahead of the WAL flush position" after falling back to archive recovery. This happened because archive recovery processes whole segment files, so after replaying a segment the cascade's next read position lands at the start of the following segment, which is ahead of the upstream's flush position reported by GetStandbyFlushRecPtr() (still inside the just-replayed segment). Fix by having the walreceiver check the upstream's current WAL flush position via IDENTIFY_SYSTEM before issuing START_REPLICATION. IDENTIFY_SYSTEM already returns this position (as xlogpos), but walrcv_identify_system() previously discarded it; now we have a use for it. If the requested start point exceeds the upstream's flush position on the same timeline, the walreceiver waits for wal_retrieve_retry_interval and retries. The wait is limited to gaps of at most one WAL segment, which is the expected case from the segment-granularity of archive recovery. Larger gaps indicate the upstream is genuinely behind, so START_REPLICATION is allowed to proceed (and fail) normally, letting the startup process fall back to other WAL sources. The first wait is logged at LOG level; subsequent waits are demoted to DEBUG1 to avoid log noise. The walreceiver honors wal_receiver_timeout during the wait, so it will exit if the upstream doesn't catch up in time. To preserve ABI compatibility on back branches, the flush position from IDENTIFY_SYSTEM is communicated via a new global variable (WalRcvIdentifySystemLsn) rather than changing the signature of walrcv_identify_system(). The bug was introduced in Postgres 9.3 by commit abfd192b1b5b, which added a flush-position check in StartReplication() that rejects requests ahead of the upstream server's WAL flush position. Author: Marco Nenciarini <marco.nenciarini@enterprisedb.com> Reviewed-by: Xuneng Zhou <xunengzhou@gmail.com> Backpatch-through: 14 Discussion: https://postgr.es/m/CA+nrD2cTuTkkX5WXVZengTYYZbAO6zV8K+Tri-R0fbLFuoyMBA@mail.gmail.com Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/5fb3c638910cdac2e00861e6f6806d9e03e5831d Modified Files -------------- .../libpqwalreceiver/libpqwalreceiver.c | 14 ++ src/backend/replication/walreceiver.c | 79 +++++++++- src/backend/utils/activity/wait_event.c | 3 + src/backend/utils/adt/formatting.c | 21 ++- src/include/replication/walreceiver.h | 7 + src/include/utils/wait_event.h | 3 +- src/test/recovery/t/055_cascade_reconnect.pl | 164 +++++++++++++++++++++ 7 files changed, 286 insertions(+), 5 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2026-07-29 15:17 UTC | newest] Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2026-07-29 15:17 pgsql: Fix cascading standby reconnect failure after archive fallback Álvaro Herrera <alvherre@kurilemu.de> 2026-07-29 15:17 pgsql: Fix cascading standby reconnect failure after archive fallback Álvaro Herrera <alvherre@kurilemu.de> 2026-07-29 15:17 pgsql: Fix cascading standby reconnect failure after archive fallback Álvaro Herrera <alvherre@kurilemu.de> 2026-07-29 15:17 pgsql: Fix cascading standby reconnect failure after archive fallback Álvaro Herrera <alvherre@kurilemu.de> 2026-07-29 15:17 pgsql: Fix cascading standby reconnect failure after archive fallback Álvaro Herrera <alvherre@kurilemu.de> 2026-07-29 15:17 pgsql: Fix cascading standby reconnect failure after archive fallback Álvaro Herrera <alvherre@kurilemu.de> 2026-07-29 15:17 pgsql: Fix cascading standby reconnect failure after archive fallback Álvaro Herrera <alvherre@kurilemu.de>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox