pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: basebackup: do not verify checksums on pages from before enablin 2+ messages / 1 participants [nested] [flat]
* pgsql: basebackup: do not verify checksums on pages from before enablin @ 2026-08-18 20:40 Daniel Gustafsson <dgustafsson@postgresql.org> 0 siblings, 0 replies; 2+ messages in thread From: Daniel Gustafsson @ 2026-08-18 20:40 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org basebackup: do not verify checksums on pages from before enabling Enabling data checksums in a running cluster changes the state to "on" before the checkpoint which flushes the pages the worker rewrote. A base backup which started before that transition absorbs the barrier mid-run and starts verifying pages whose on-disk copies legitimately lack checksums, and whose LSNs predate the backup start, so the LSN check does not skip them either. The backup fails with bogus corruption warnings. The same applies to checksums being disabled and re-enabled while the backup runs: hint bits set while checksums were off reach disk without a checksum update and without moving the page LSN, tripping verification once the re-enabling completes. To fix, verify checksums only while they have been continuously enabled since the checkpoint the backup started from: track the location of the last XLOG2_CHECKSUMS record inserted or replayed, and verify only when the state is "on" and the last change predates the backup start. The starting checkpoint then guarantees that every page flushed before it has a checksum written, and any later change disables verification for the rest of the backup. A standby loses the tracked location when restarting, while pg_control already carries the new state, so it could reach consistency below the record and serve base backups with the location unknown. To prevent this, replaying XLOG2_CHECKSUMS advances minRecoveryPoint to the record, like XLOG_PARAMETER_CHANGE does. The tests hold the enabling between the state change and its final checkpoint with injection points, straddling it with backups on the primary and across a standby crash-restart. Author: Zsolt Parragi <zsolt.parragi@percona.com> Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com> Reviewed-by: Daniel Gustafsson <daniel@yesql.se> Discussion: https://postgr.es/m/CAN4CZFP=-cVVVPue+e8qqPtDfuLuQn=ZB4Mw_C9-Ncru2wqAsQ@mail.gmail.com Backpatch-through: 19 Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/55edc7907b87db6c0b3257b4d7083c2b9fcbfd69 Modified Files -------------- doc/src/sgml/ref/pg_basebackup.sgml | 6 + src/backend/access/transam/xlog.c | 58 +++++ src/backend/backup/basebackup.c | 73 +++++- src/include/access/xlog.h | 1 + src/test/modules/test_checksums/meson.build | 2 + .../test_checksums/t/010_backup_straddle.pl | 260 +++++++++++++++++++++ .../test_checksums/t/011_standby_straddle.pl | 249 ++++++++++++++++++++ 7 files changed, 640 insertions(+), 9 deletions(-) ^ permalink raw reply [nested|flat] 2+ messages in thread
* pgsql: basebackup: do not verify checksums on pages from before enablin @ 2026-08-18 20:41 Daniel Gustafsson <dgustafsson@postgresql.org> 0 siblings, 0 replies; 2+ messages in thread From: Daniel Gustafsson @ 2026-08-18 20:41 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org basebackup: do not verify checksums on pages from before enabling Enabling data checksums in a running cluster changes the state to "on" before the checkpoint which flushes the pages the worker rewrote. A base backup which started before that transition absorbs the barrier mid-run and starts verifying pages whose on-disk copies legitimately lack checksums, and whose LSNs predate the backup start, so the LSN check does not skip them either. The backup fails with bogus corruption warnings. The same applies to checksums being disabled and re-enabled while the backup runs: hint bits set while checksums were off reach disk without a checksum update and without moving the page LSN, tripping verification once the re-enabling completes. To fix, verify checksums only while they have been continuously enabled since the checkpoint the backup started from: track the location of the last XLOG2_CHECKSUMS record inserted or replayed, and verify only when the state is "on" and the last change predates the backup start. The starting checkpoint then guarantees that every page flushed before it has a checksum written, and any later change disables verification for the rest of the backup. A standby loses the tracked location when restarting, while pg_control already carries the new state, so it could reach consistency below the record and serve base backups with the location unknown. To prevent this, replaying XLOG2_CHECKSUMS advances minRecoveryPoint to the record, like XLOG_PARAMETER_CHANGE does. The tests hold the enabling between the state change and its final checkpoint with injection points, straddling it with backups on the primary and across a standby crash-restart. Author: Zsolt Parragi <zsolt.parragi@percona.com> Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com> Reviewed-by: Daniel Gustafsson <daniel@yesql.se> Discussion: https://postgr.es/m/CAN4CZFP=-cVVVPue+e8qqPtDfuLuQn=ZB4Mw_C9-Ncru2wqAsQ@mail.gmail.com Backpatch-through: 19 Branch ------ REL_19_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/0907112d3882192f3c14d5ce0735790341e073ed Modified Files -------------- doc/src/sgml/ref/pg_basebackup.sgml | 6 + src/backend/access/transam/xlog.c | 58 +++++ src/backend/backup/basebackup.c | 73 +++++- src/include/access/xlog.h | 1 + src/test/modules/test_checksums/meson.build | 2 + .../test_checksums/t/010_backup_straddle.pl | 260 +++++++++++++++++++++ .../test_checksums/t/011_standby_straddle.pl | 249 ++++++++++++++++++++ 7 files changed, 640 insertions(+), 9 deletions(-) ^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2026-08-18 20:41 UTC | newest] Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2026-08-18 20:40 pgsql: basebackup: do not verify checksums on pages from before enablin Daniel Gustafsson <dgustafsson@postgresql.org> 2026-08-18 20:41 pgsql: basebackup: do not verify checksums on pages from before enablin Daniel Gustafsson <dgustafsson@postgresql.org>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox