agora inbox for pgsql-committers@postgresql.orghelp / color / mirror / Atom feed
pgsql: Mark modified the FSM buffer as dirty during recovery 6+ messages / 1 participants [nested] [flat]
* pgsql: Mark modified the FSM buffer as dirty during recovery @ 2026-05-03 18:21 Alexander Korotkov <akorotkov@postgresql.org> 0 siblings, 0 replies; 6+ messages in thread From: Alexander Korotkov @ 2026-05-03 18:21 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Mark modified the FSM buffer as dirty during recovery The XLogRecordPageWithFreeSpace function updates the freespace map (FSM) data while replaying data-level WAL records during the recovery. If the FSM block is updated, it needs to be marked as modified. Currently, this is done with the MarkBufferDirtyHint call (as in all other cases for modifying FSM data). However, in the recovery context, this function will actually do nothing if checksums are enabled. It's assumed that the page should not be dirtied during recovery while modifying hints to protect against torn pages, since no new WAL data can be generated at this point to store FPI. Such logic does not seem fully aligned with the FSM case, as its blocks could be simply zeroed if a checksum mismatch is detected. Currently, changes to an FSM block could be lost if each change to that block occurs infrequently enough to allow it to be evicted from the cache. To persist the change, the modification needs to be performed while the FSM block is still kept in buffers and marked as dirty after receiving its FPI. If the block has already been cleaned, the change won't be persisted, so stored FSM blocks may remain in an obsolete state. If a large number of discrepancies between the data in leaf FSM blocks and the actual data blocks accumulate on the replica server, this could cause significant delays in insert operations after switchover. Such an insert operation may need to visit many data blocks marked as having sufficient space in the FSM, only to discover that the information is incorrect and the FSM records need to be corrected. In a heavily trafficked insert-only table with many concurrent clients performing inserts, this has been observed to cause several-second stalls, causing visible application malfunction. The desire to avoid such cases was the reason behind the commit ab7dbd681, which introduced an update of FSM data during the heap_xlog_visible invocation. However, an update to the FSM data on the standby side could be lost due to a missing 'dirty' flag, so there is still a possibility that a large number of FSM records will contain incorrect data. Note that having a zeroed FSM page in such a case (due to a checksum mismatch) is preferable, as a zero value will be interpreted as an indication of full data blocks, and the inserter will be routed to the next FSM block or to the end of the table. Given that FSM is ready to handle torn page writes and XLogRecordPageWithFreeSpace is called only during the recovery, there seems to be no reason to use MarkBufferDirtyHint here instead of a regular MarkBufferDirty call. Discussion: https://postgr.es/m/596c4f1c-f966-4512-b9c9-dd8fbcaf0928%40postgrespro.ru Author: Alexey Makhmutov <a.makhmutov@postgrespro.ru> Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru> Reviewed-by: Melanie Plageman <melanieplageman@gmail.com> Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com> Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/f5d1038d92cf102b30cd4429d10c97b5293f8115 Modified Files -------------- src/backend/storage/freespace/freespace.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Mark modified the FSM buffer as dirty during recovery @ 2026-05-03 18:21 Alexander Korotkov <akorotkov@postgresql.org> 0 siblings, 0 replies; 6+ messages in thread From: Alexander Korotkov @ 2026-05-03 18:21 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Mark modified the FSM buffer as dirty during recovery The XLogRecordPageWithFreeSpace function updates the freespace map (FSM) data while replaying data-level WAL records during the recovery. If the FSM block is updated, it needs to be marked as modified. Currently, this is done with the MarkBufferDirtyHint call (as in all other cases for modifying FSM data). However, in the recovery context, this function will actually do nothing if checksums are enabled. It's assumed that the page should not be dirtied during recovery while modifying hints to protect against torn pages, since no new WAL data can be generated at this point to store FPI. Such logic does not seem fully aligned with the FSM case, as its blocks could be simply zeroed if a checksum mismatch is detected. Currently, changes to an FSM block could be lost if each change to that block occurs infrequently enough to allow it to be evicted from the cache. To persist the change, the modification needs to be performed while the FSM block is still kept in buffers and marked as dirty after receiving its FPI. If the block has already been cleaned, the change won't be persisted, so stored FSM blocks may remain in an obsolete state. If a large number of discrepancies between the data in leaf FSM blocks and the actual data blocks accumulate on the replica server, this could cause significant delays in insert operations after switchover. Such an insert operation may need to visit many data blocks marked as having sufficient space in the FSM, only to discover that the information is incorrect and the FSM records need to be corrected. In a heavily trafficked insert-only table with many concurrent clients performing inserts, this has been observed to cause several-second stalls, causing visible application malfunction. The desire to avoid such cases was the reason behind the commit ab7dbd681, which introduced an update of FSM data during the heap_xlog_visible invocation. However, an update to the FSM data on the standby side could be lost due to a missing 'dirty' flag, so there is still a possibility that a large number of FSM records will contain incorrect data. Note that having a zeroed FSM page in such a case (due to a checksum mismatch) is preferable, as a zero value will be interpreted as an indication of full data blocks, and the inserter will be routed to the next FSM block or to the end of the table. Given that FSM is ready to handle torn page writes and XLogRecordPageWithFreeSpace is called only during the recovery, there seems to be no reason to use MarkBufferDirtyHint here instead of a regular MarkBufferDirty call. Discussion: https://postgr.es/m/596c4f1c-f966-4512-b9c9-dd8fbcaf0928%40postgrespro.ru Author: Alexey Makhmutov <a.makhmutov@postgrespro.ru> Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru> Reviewed-by: Melanie Plageman <melanieplageman@gmail.com> Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com> Branch ------ REL_15_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/ca259b08409096811b57b1c13dd5cccf9b2b83b1 Modified Files -------------- src/backend/storage/freespace/freespace.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Mark modified the FSM buffer as dirty during recovery @ 2026-05-03 18:21 Alexander Korotkov <akorotkov@postgresql.org> 0 siblings, 0 replies; 6+ messages in thread From: Alexander Korotkov @ 2026-05-03 18:21 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Mark modified the FSM buffer as dirty during recovery The XLogRecordPageWithFreeSpace function updates the freespace map (FSM) data while replaying data-level WAL records during the recovery. If the FSM block is updated, it needs to be marked as modified. Currently, this is done with the MarkBufferDirtyHint call (as in all other cases for modifying FSM data). However, in the recovery context, this function will actually do nothing if checksums are enabled. It's assumed that the page should not be dirtied during recovery while modifying hints to protect against torn pages, since no new WAL data can be generated at this point to store FPI. Such logic does not seem fully aligned with the FSM case, as its blocks could be simply zeroed if a checksum mismatch is detected. Currently, changes to an FSM block could be lost if each change to that block occurs infrequently enough to allow it to be evicted from the cache. To persist the change, the modification needs to be performed while the FSM block is still kept in buffers and marked as dirty after receiving its FPI. If the block has already been cleaned, the change won't be persisted, so stored FSM blocks may remain in an obsolete state. If a large number of discrepancies between the data in leaf FSM blocks and the actual data blocks accumulate on the replica server, this could cause significant delays in insert operations after switchover. Such an insert operation may need to visit many data blocks marked as having sufficient space in the FSM, only to discover that the information is incorrect and the FSM records need to be corrected. In a heavily trafficked insert-only table with many concurrent clients performing inserts, this has been observed to cause several-second stalls, causing visible application malfunction. The desire to avoid such cases was the reason behind the commit ab7dbd681, which introduced an update of FSM data during the heap_xlog_visible invocation. However, an update to the FSM data on the standby side could be lost due to a missing 'dirty' flag, so there is still a possibility that a large number of FSM records will contain incorrect data. Note that having a zeroed FSM page in such a case (due to a checksum mismatch) is preferable, as a zero value will be interpreted as an indication of full data blocks, and the inserter will be routed to the next FSM block or to the end of the table. Given that FSM is ready to handle torn page writes and XLogRecordPageWithFreeSpace is called only during the recovery, there seems to be no reason to use MarkBufferDirtyHint here instead of a regular MarkBufferDirty call. Discussion: https://postgr.es/m/596c4f1c-f966-4512-b9c9-dd8fbcaf0928%40postgrespro.ru Author: Alexey Makhmutov <a.makhmutov@postgrespro.ru> Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru> Reviewed-by: Melanie Plageman <melanieplageman@gmail.com> Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com> Branch ------ REL_16_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/54537de35873eaa4547ab2aa7168c6a935894643 Modified Files -------------- src/backend/storage/freespace/freespace.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Mark modified the FSM buffer as dirty during recovery @ 2026-05-03 18:21 Alexander Korotkov <akorotkov@postgresql.org> 0 siblings, 0 replies; 6+ messages in thread From: Alexander Korotkov @ 2026-05-03 18:21 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Mark modified the FSM buffer as dirty during recovery The XLogRecordPageWithFreeSpace function updates the freespace map (FSM) data while replaying data-level WAL records during the recovery. If the FSM block is updated, it needs to be marked as modified. Currently, this is done with the MarkBufferDirtyHint call (as in all other cases for modifying FSM data). However, in the recovery context, this function will actually do nothing if checksums are enabled. It's assumed that the page should not be dirtied during recovery while modifying hints to protect against torn pages, since no new WAL data can be generated at this point to store FPI. Such logic does not seem fully aligned with the FSM case, as its blocks could be simply zeroed if a checksum mismatch is detected. Currently, changes to an FSM block could be lost if each change to that block occurs infrequently enough to allow it to be evicted from the cache. To persist the change, the modification needs to be performed while the FSM block is still kept in buffers and marked as dirty after receiving its FPI. If the block has already been cleaned, the change won't be persisted, so stored FSM blocks may remain in an obsolete state. If a large number of discrepancies between the data in leaf FSM blocks and the actual data blocks accumulate on the replica server, this could cause significant delays in insert operations after switchover. Such an insert operation may need to visit many data blocks marked as having sufficient space in the FSM, only to discover that the information is incorrect and the FSM records need to be corrected. In a heavily trafficked insert-only table with many concurrent clients performing inserts, this has been observed to cause several-second stalls, causing visible application malfunction. The desire to avoid such cases was the reason behind the commit ab7dbd681, which introduced an update of FSM data during the heap_xlog_visible invocation. However, an update to the FSM data on the standby side could be lost due to a missing 'dirty' flag, so there is still a possibility that a large number of FSM records will contain incorrect data. Note that having a zeroed FSM page in such a case (due to a checksum mismatch) is preferable, as a zero value will be interpreted as an indication of full data blocks, and the inserter will be routed to the next FSM block or to the end of the table. Given that FSM is ready to handle torn page writes and XLogRecordPageWithFreeSpace is called only during the recovery, there seems to be no reason to use MarkBufferDirtyHint here instead of a regular MarkBufferDirty call. Discussion: https://postgr.es/m/596c4f1c-f966-4512-b9c9-dd8fbcaf0928%40postgrespro.ru Author: Alexey Makhmutov <a.makhmutov@postgrespro.ru> Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru> Reviewed-by: Melanie Plageman <melanieplageman@gmail.com> Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com> Branch ------ REL_17_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/1cf010f216138c6d580107589c9e42de72da37f2 Modified Files -------------- src/backend/storage/freespace/freespace.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Mark modified the FSM buffer as dirty during recovery @ 2026-05-03 18:21 Alexander Korotkov <akorotkov@postgresql.org> 0 siblings, 0 replies; 6+ messages in thread From: Alexander Korotkov @ 2026-05-03 18:21 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Mark modified the FSM buffer as dirty during recovery The XLogRecordPageWithFreeSpace function updates the freespace map (FSM) data while replaying data-level WAL records during the recovery. If the FSM block is updated, it needs to be marked as modified. Currently, this is done with the MarkBufferDirtyHint call (as in all other cases for modifying FSM data). However, in the recovery context, this function will actually do nothing if checksums are enabled. It's assumed that the page should not be dirtied during recovery while modifying hints to protect against torn pages, since no new WAL data can be generated at this point to store FPI. Such logic does not seem fully aligned with the FSM case, as its blocks could be simply zeroed if a checksum mismatch is detected. Currently, changes to an FSM block could be lost if each change to that block occurs infrequently enough to allow it to be evicted from the cache. To persist the change, the modification needs to be performed while the FSM block is still kept in buffers and marked as dirty after receiving its FPI. If the block has already been cleaned, the change won't be persisted, so stored FSM blocks may remain in an obsolete state. If a large number of discrepancies between the data in leaf FSM blocks and the actual data blocks accumulate on the replica server, this could cause significant delays in insert operations after switchover. Such an insert operation may need to visit many data blocks marked as having sufficient space in the FSM, only to discover that the information is incorrect and the FSM records need to be corrected. In a heavily trafficked insert-only table with many concurrent clients performing inserts, this has been observed to cause several-second stalls, causing visible application malfunction. The desire to avoid such cases was the reason behind the commit ab7dbd681, which introduced an update of FSM data during the heap_xlog_visible invocation. However, an update to the FSM data on the standby side could be lost due to a missing 'dirty' flag, so there is still a possibility that a large number of FSM records will contain incorrect data. Note that having a zeroed FSM page in such a case (due to a checksum mismatch) is preferable, as a zero value will be interpreted as an indication of full data blocks, and the inserter will be routed to the next FSM block or to the end of the table. Given that FSM is ready to handle torn page writes and XLogRecordPageWithFreeSpace is called only during the recovery, there seems to be no reason to use MarkBufferDirtyHint here instead of a regular MarkBufferDirty call. Discussion: https://postgr.es/m/596c4f1c-f966-4512-b9c9-dd8fbcaf0928%40postgrespro.ru Author: Alexey Makhmutov <a.makhmutov@postgrespro.ru> Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru> Reviewed-by: Melanie Plageman <melanieplageman@gmail.com> Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com> Branch ------ REL_18_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/ac3b97db380ad6295e6fe582c64dd80ae32d4b94 Modified Files -------------- src/backend/storage/freespace/freespace.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Mark modified the FSM buffer as dirty during recovery @ 2026-05-03 18:21 Alexander Korotkov <akorotkov@postgresql.org> 0 siblings, 0 replies; 6+ messages in thread From: Alexander Korotkov @ 2026-05-03 18:21 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Mark modified the FSM buffer as dirty during recovery The XLogRecordPageWithFreeSpace function updates the freespace map (FSM) data while replaying data-level WAL records during the recovery. If the FSM block is updated, it needs to be marked as modified. Currently, this is done with the MarkBufferDirtyHint call (as in all other cases for modifying FSM data). However, in the recovery context, this function will actually do nothing if checksums are enabled. It's assumed that the page should not be dirtied during recovery while modifying hints to protect against torn pages, since no new WAL data can be generated at this point to store FPI. Such logic does not seem fully aligned with the FSM case, as its blocks could be simply zeroed if a checksum mismatch is detected. Currently, changes to an FSM block could be lost if each change to that block occurs infrequently enough to allow it to be evicted from the cache. To persist the change, the modification needs to be performed while the FSM block is still kept in buffers and marked as dirty after receiving its FPI. If the block has already been cleaned, the change won't be persisted, so stored FSM blocks may remain in an obsolete state. If a large number of discrepancies between the data in leaf FSM blocks and the actual data blocks accumulate on the replica server, this could cause significant delays in insert operations after switchover. Such an insert operation may need to visit many data blocks marked as having sufficient space in the FSM, only to discover that the information is incorrect and the FSM records need to be corrected. In a heavily trafficked insert-only table with many concurrent clients performing inserts, this has been observed to cause several-second stalls, causing visible application malfunction. The desire to avoid such cases was the reason behind the commit ab7dbd681, which introduced an update of FSM data during the heap_xlog_visible invocation. However, an update to the FSM data on the standby side could be lost due to a missing 'dirty' flag, so there is still a possibility that a large number of FSM records will contain incorrect data. Note that having a zeroed FSM page in such a case (due to a checksum mismatch) is preferable, as a zero value will be interpreted as an indication of full data blocks, and the inserter will be routed to the next FSM block or to the end of the table. Given that FSM is ready to handle torn page writes and XLogRecordPageWithFreeSpace is called only during the recovery, there seems to be no reason to use MarkBufferDirtyHint here instead of a regular MarkBufferDirty call. Discussion: https://postgr.es/m/596c4f1c-f966-4512-b9c9-dd8fbcaf0928%40postgrespro.ru Author: Alexey Makhmutov <a.makhmutov@postgrespro.ru> Reviewed-by: Andrey Borodin <x4mmm@yandex-team.ru> Reviewed-by: Melanie Plageman <melanieplageman@gmail.com> Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com> Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/c06d1a4ba6b26eef27b04074683cccade6c277ee Modified Files -------------- src/backend/storage/freespace/freespace.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2026-05-03 18:21 UTC | newest] Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2026-05-03 18:21 pgsql: Mark modified the FSM buffer as dirty during recovery Alexander Korotkov <akorotkov@postgresql.org> 2026-05-03 18:21 pgsql: Mark modified the FSM buffer as dirty during recovery Alexander Korotkov <akorotkov@postgresql.org> 2026-05-03 18:21 pgsql: Mark modified the FSM buffer as dirty during recovery Alexander Korotkov <akorotkov@postgresql.org> 2026-05-03 18:21 pgsql: Mark modified the FSM buffer as dirty during recovery Alexander Korotkov <akorotkov@postgresql.org> 2026-05-03 18:21 pgsql: Mark modified the FSM buffer as dirty during recovery Alexander Korotkov <akorotkov@postgresql.org> 2026-05-03 18:21 pgsql: Mark modified the FSM buffer as dirty during recovery Alexander Korotkov <akorotkov@postgresql.org>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox