pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: Fix header check for continuation records where standbys could b 6+ messages / 1 participants [nested] [flat]
* pgsql: Fix header check for continuation records where standbys could b @ 2025-01-20 00:31 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 6+ messages in thread From: Michael Paquier @ 2025-01-20 00:31 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix header check for continuation records where standbys could be stuck XLogPageRead() checks immediately for an invalid WAL record header on a standby, to be able to handle the case of continuation records that need to be read across two different sources. As written, the check was too generic, applying to any target LSN. Based on an analysis by Kyotaro Horiguchi, what really matters is to make sure that the page header is checked when attempting to read a LSN at the boundary of a segment, to handle the case of a continuation record that spawns across multiple pages when dealing with multiple segments, as WAL receivers are spawned they request WAL from the beginning of a segment. This fix has been proposed by Kyotaro Horiguchi. This could cause standbys to loop infinitely when dealing with a continuation record during a timeline jump, in the case where the contents of the record in the follow-up page are invalid. Some regression tests are added to check such scenarios, able to reproduce the original problem. In the test, the contents of a continuation record are overwritten with junk zeros on its follow-up page, and replayed on standbys. This is inspired by 039_end_of_wal.pl, and is enough to show how standbys should react on promotion by not being stuck. Without the fix, the test would fail with a timeout. The test to reproduce the problem has been written by Alexander Kukushkin. The original check has been introduced in 066871980183, for a similar problem. Author: Kyotaro Horiguchi, Alexander Kukushkin Reviewed-by: Michael Paquier Discussion: https://postgr.es/m/CAFh8B=mozC+e1wGJq0H=0O65goZju+6ab5AU7DEWCSUA2OtwDg@mail.gmail.com Backpatch-through: 13 Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/6cf1647d87e7cd423d71525a8759b75c4e4a47ec Modified Files -------------- src/backend/access/transam/xlogrecovery.c | 13 +- src/test/recovery/meson.build | 1 + src/test/recovery/t/043_no_contrecord_switch.pl | 154 ++++++++++++++++++++++++ 3 files changed, 162 insertions(+), 6 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix header check for continuation records where standbys could b @ 2025-01-20 00:31 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 6+ messages in thread From: Michael Paquier @ 2025-01-20 00:31 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix header check for continuation records where standbys could be stuck XLogPageRead() checks immediately for an invalid WAL record header on a standby, to be able to handle the case of continuation records that need to be read across two different sources. As written, the check was too generic, applying to any target LSN. Based on an analysis by Kyotaro Horiguchi, what really matters is to make sure that the page header is checked when attempting to read a LSN at the boundary of a segment, to handle the case of a continuation record that spawns across multiple pages when dealing with multiple segments, as WAL receivers are spawned they request WAL from the beginning of a segment. This fix has been proposed by Kyotaro Horiguchi. This could cause standbys to loop infinitely when dealing with a continuation record during a timeline jump, in the case where the contents of the record in the follow-up page are invalid. Some regression tests are added to check such scenarios, able to reproduce the original problem. In the test, the contents of a continuation record are overwritten with junk zeros on its follow-up page, and replayed on standbys. This is inspired by 039_end_of_wal.pl, and is enough to show how standbys should react on promotion by not being stuck. Without the fix, the test would fail with a timeout. The test to reproduce the problem has been written by Alexander Kukushkin. The original check has been introduced in 066871980183, for a similar problem. Author: Kyotaro Horiguchi, Alexander Kukushkin Reviewed-by: Michael Paquier Discussion: https://postgr.es/m/CAFh8B=mozC+e1wGJq0H=0O65goZju+6ab5AU7DEWCSUA2OtwDg@mail.gmail.com Backpatch-through: 13 Branch ------ REL_17_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/e6767c0ed16f34435b632a78b01d2d5397b3bb09 Modified Files -------------- src/backend/access/transam/xlogrecovery.c | 13 +- src/test/recovery/meson.build | 1 + src/test/recovery/t/043_no_contrecord_switch.pl | 153 ++++++++++++++++++++++++ 3 files changed, 161 insertions(+), 6 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix header check for continuation records where standbys could b @ 2025-01-20 00:31 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 6+ messages in thread From: Michael Paquier @ 2025-01-20 00:31 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix header check for continuation records where standbys could be stuck XLogPageRead() checks immediately for an invalid WAL record header on a standby, to be able to handle the case of continuation records that need to be read across two different sources. As written, the check was too generic, applying to any target LSN. Based on an analysis by Kyotaro Horiguchi, what really matters is to make sure that the page header is checked when attempting to read a LSN at the boundary of a segment, to handle the case of a continuation record that spawns across multiple pages when dealing with multiple segments, as WAL receivers are spawned they request WAL from the beginning of a segment. This fix has been proposed by Kyotaro Horiguchi. This could cause standbys to loop infinitely when dealing with a continuation record during a timeline jump, in the case where the contents of the record in the follow-up page are invalid. Some regression tests are added to check such scenarios, able to reproduce the original problem. In the test, the contents of a continuation record are overwritten with junk zeros on its follow-up page, and replayed on standbys. This is inspired by 039_end_of_wal.pl, and is enough to show how standbys should react on promotion by not being stuck. Without the fix, the test would fail with a timeout. The test to reproduce the problem has been written by Alexander Kukushkin. The original check has been introduced in 066871980183, for a similar problem. Author: Kyotaro Horiguchi, Alexander Kukushkin Reviewed-by: Michael Paquier Discussion: https://postgr.es/m/CAFh8B=mozC+e1wGJq0H=0O65goZju+6ab5AU7DEWCSUA2OtwDg@mail.gmail.com Backpatch-through: 13 Branch ------ REL_16_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/2c2e1d4f42c00784b46023ba123bd5cd38d914fb Modified Files -------------- src/backend/access/transam/xlogrecovery.c | 13 +- src/test/recovery/meson.build | 1 + src/test/recovery/t/043_no_contrecord_switch.pl | 153 ++++++++++++++++++++++++ 3 files changed, 161 insertions(+), 6 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix header check for continuation records where standbys could b @ 2025-01-20 00:31 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 6+ messages in thread From: Michael Paquier @ 2025-01-20 00:31 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix header check for continuation records where standbys could be stuck XLogPageRead() checks immediately for an invalid WAL record header on a standby, to be able to handle the case of continuation records that need to be read across two different sources. As written, the check was too generic, applying to any target LSN. Based on an analysis by Kyotaro Horiguchi, what really matters is to make sure that the page header is checked when attempting to read a LSN at the boundary of a segment, to handle the case of a continuation record that spawns across multiple pages when dealing with multiple segments, as WAL receivers are spawned they request WAL from the beginning of a segment. This fix has been proposed by Kyotaro Horiguchi. This could cause standbys to loop infinitely when dealing with a continuation record during a timeline jump, in the case where the contents of the record in the follow-up page are invalid. Some regression tests are added to check such scenarios, able to reproduce the original problem. In the test, the contents of a continuation record are overwritten with junk zeros on its follow-up page, and replayed on standbys. This is inspired by 039_end_of_wal.pl, and is enough to show how standbys should react on promotion by not being stuck. Without the fix, the test would fail with a timeout. The test to reproduce the problem has been written by Alexander Kukushkin. The original check has been introduced in 066871980183, for a similar problem. Author: Kyotaro Horiguchi, Alexander Kukushkin Reviewed-by: Michael Paquier Discussion: https://postgr.es/m/CAFh8B=mozC+e1wGJq0H=0O65goZju+6ab5AU7DEWCSUA2OtwDg@mail.gmail.com Backpatch-through: 13 Branch ------ REL_15_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/26554faccc975fbd23b7cc04c0b10079e5c839c9 Modified Files -------------- src/backend/access/transam/xlogrecovery.c | 13 +- src/test/recovery/t/043_no_contrecord_switch.pl | 153 ++++++++++++++++++++++++ 2 files changed, 160 insertions(+), 6 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix header check for continuation records where standbys could b @ 2025-01-20 00:31 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 6+ messages in thread From: Michael Paquier @ 2025-01-20 00:31 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix header check for continuation records where standbys could be stuck XLogPageRead() checks immediately for an invalid WAL record header on a standby, to be able to handle the case of continuation records that need to be read across two different sources. As written, the check was too generic, applying to any target LSN. Based on an analysis by Kyotaro Horiguchi, what really matters is to make sure that the page header is checked when attempting to read a LSN at the boundary of a segment, to handle the case of a continuation record that spawns across multiple pages when dealing with multiple segments, as WAL receivers are spawned they request WAL from the beginning of a segment. This fix has been proposed by Kyotaro Horiguchi. This could cause standbys to loop infinitely when dealing with a continuation record during a timeline jump, in the case where the contents of the record in the follow-up page are invalid. Some regression tests are added to check such scenarios, able to reproduce the original problem. In the test, the contents of a continuation record are overwritten with junk zeros on its follow-up page, and replayed on standbys. This is inspired by 039_end_of_wal.pl, and is enough to show how standbys should react on promotion by not being stuck. Without the fix, the test would fail with a timeout. The test to reproduce the problem has been written by Alexander Kukushkin. The original check has been introduced in 066871980183, for a similar problem. Author: Kyotaro Horiguchi, Alexander Kukushkin Reviewed-by: Michael Paquier Discussion: https://postgr.es/m/CAFh8B=mozC+e1wGJq0H=0O65goZju+6ab5AU7DEWCSUA2OtwDg@mail.gmail.com Backpatch-through: 13 Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/a2d4f806c4b91496f6055ea1200b5268ee550fca Modified Files -------------- src/backend/access/transam/xlog.c | 24 ++-- src/test/recovery/t/043_no_contrecord_switch.pl | 153 ++++++++++++++++++++++++ 2 files changed, 165 insertions(+), 12 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix header check for continuation records where standbys could b @ 2025-01-20 00:31 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 6+ messages in thread From: Michael Paquier @ 2025-01-20 00:31 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix header check for continuation records where standbys could be stuck XLogPageRead() checks immediately for an invalid WAL record header on a standby, to be able to handle the case of continuation records that need to be read across two different sources. As written, the check was too generic, applying to any target LSN. Based on an analysis by Kyotaro Horiguchi, what really matters is to make sure that the page header is checked when attempting to read a LSN at the boundary of a segment, to handle the case of a continuation record that spawns across multiple pages when dealing with multiple segments, as WAL receivers are spawned they request WAL from the beginning of a segment. This fix has been proposed by Kyotaro Horiguchi. This could cause standbys to loop infinitely when dealing with a continuation record during a timeline jump, in the case where the contents of the record in the follow-up page are invalid. Some regression tests are added to check such scenarios, able to reproduce the original problem. In the test, the contents of a continuation record are overwritten with junk zeros on its follow-up page, and replayed on standbys. This is inspired by 039_end_of_wal.pl, and is enough to show how standbys should react on promotion by not being stuck. Without the fix, the test would fail with a timeout. The test to reproduce the problem has been written by Alexander Kukushkin. The original check has been introduced in 066871980183, for a similar problem. Author: Kyotaro Horiguchi, Alexander Kukushkin Reviewed-by: Michael Paquier Discussion: https://postgr.es/m/CAFh8B=mozC+e1wGJq0H=0O65goZju+6ab5AU7DEWCSUA2OtwDg@mail.gmail.com Backpatch-through: 13 Branch ------ REL_13_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/0f0431e919f4a2889fbdbede5efc67443c51b93c Modified Files -------------- src/backend/access/transam/xlog.c | 13 +- src/test/recovery/t/043_no_contrecord_switch.pl | 153 ++++++++++++++++++++++++ 2 files changed, 160 insertions(+), 6 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2025-01-20 00:31 UTC | newest] Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2025-01-20 00:31 pgsql: Fix header check for continuation records where standbys could b Michael Paquier <michael@paquier.xyz> 2025-01-20 00:31 pgsql: Fix header check for continuation records where standbys could b Michael Paquier <michael@paquier.xyz> 2025-01-20 00:31 pgsql: Fix header check for continuation records where standbys could b Michael Paquier <michael@paquier.xyz> 2025-01-20 00:31 pgsql: Fix header check for continuation records where standbys could b Michael Paquier <michael@paquier.xyz> 2025-01-20 00:31 pgsql: Fix header check for continuation records where standbys could b Michael Paquier <michael@paquier.xyz> 2025-01-20 00:31 pgsql: Fix header check for continuation records where standbys could b Michael Paquier <michael@paquier.xyz>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox