pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: Fix snapshot builds during promotion of hot standby node with 2P 7+ messages / 1 participants [nested] [flat]
* pgsql: Fix snapshot builds during promotion of hot standby node with 2P @ 2021-10-04 05:07 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 7+ messages in thread From: Michael Paquier @ 2021-10-04 05:07 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix snapshot builds during promotion of hot standby node with 2PC Some specific logic is done at the end of recovery when involving 2PC transactions: 1) Call RecoverPreparedTransactions(), to recover the state of 2PC transactions into memory (re-acquire locks, etc.). 2) ShutdownRecoveryTransactionEnvironment(), to move back to normal operations, mainly cleaning up recovery locks and KnownAssignedXids (including any 2PC transaction tracked previously). 3) Switch XLogCtl->SharedRecoveryState to RECOVERY_STATE_DONE, which is the tipping point for any process calling RecoveryInProgress() to check if the cluster is still in recovery or not. Any snapshot taken between steps 2) and 3) would be empty, causing any transaction relying on a snapshot at this point to potentially corrupt data as there could still be some 2PC transactions to track, with RecentXmin moving backwards on successive calls to GetSnapshotData() in the same transaction. As SharedRecoveryState is the point to take into account to know if it is safe to discard KnownAssignedXids, this commit moves step 2) after step 3), so as we can never finish with empty snapshots. This exists since the introduction of hot standby, so backpatch all the way down. The window with incorrect snapshots is extremely small, but I have seen it when running 023_pitr_prepared_xact.pl, as did buildfarm member fairywren. Thomas Munro also found it independently. Special thanks to Andres Freund for taking the time to analyze this issue. Reported-by: Thomas Munro, Michael Paquier Analyzed-by: Andres Freund Discussion: https://postgr.es/m/20210422203603.fdnh3fu2mmfp2iov@alap3.anarazel.de Backpatch-through: 9.6 Branch ------ REL_10_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/8a6a1fe07e062911f6bbcb395b673931cada1ccd Modified Files -------------- src/backend/access/transam/xlog.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix snapshot builds during promotion of hot standby node with 2P @ 2021-10-04 05:07 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 7+ messages in thread From: Michael Paquier @ 2021-10-04 05:07 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix snapshot builds during promotion of hot standby node with 2PC Some specific logic is done at the end of recovery when involving 2PC transactions: 1) Call RecoverPreparedTransactions(), to recover the state of 2PC transactions into memory (re-acquire locks, etc.). 2) ShutdownRecoveryTransactionEnvironment(), to move back to normal operations, mainly cleaning up recovery locks and KnownAssignedXids (including any 2PC transaction tracked previously). 3) Switch XLogCtl->SharedRecoveryState to RECOVERY_STATE_DONE, which is the tipping point for any process calling RecoveryInProgress() to check if the cluster is still in recovery or not. Any snapshot taken between steps 2) and 3) would be empty, causing any transaction relying on a snapshot at this point to potentially corrupt data as there could still be some 2PC transactions to track, with RecentXmin moving backwards on successive calls to GetSnapshotData() in the same transaction. As SharedRecoveryState is the point to take into account to know if it is safe to discard KnownAssignedXids, this commit moves step 2) after step 3), so as we can never finish with empty snapshots. This exists since the introduction of hot standby, so backpatch all the way down. The window with incorrect snapshots is extremely small, but I have seen it when running 023_pitr_prepared_xact.pl, as did buildfarm member fairywren. Thomas Munro also found it independently. Special thanks to Andres Freund for taking the time to analyze this issue. Reported-by: Thomas Munro, Michael Paquier Analyzed-by: Andres Freund Discussion: https://postgr.es/m/20210422203603.fdnh3fu2mmfp2iov@alap3.anarazel.de Backpatch-through: 9.6 Branch ------ REL9_6_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/e2b2a9e1c3d51c33f9b4a0d29d8bb7cad05c41ae Modified Files -------------- src/backend/access/transam/xlog.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix snapshot builds during promotion of hot standby node with 2P @ 2021-10-04 05:07 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 7+ messages in thread From: Michael Paquier @ 2021-10-04 05:07 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix snapshot builds during promotion of hot standby node with 2PC Some specific logic is done at the end of recovery when involving 2PC transactions: 1) Call RecoverPreparedTransactions(), to recover the state of 2PC transactions into memory (re-acquire locks, etc.). 2) ShutdownRecoveryTransactionEnvironment(), to move back to normal operations, mainly cleaning up recovery locks and KnownAssignedXids (including any 2PC transaction tracked previously). 3) Switch XLogCtl->SharedRecoveryState to RECOVERY_STATE_DONE, which is the tipping point for any process calling RecoveryInProgress() to check if the cluster is still in recovery or not. Any snapshot taken between steps 2) and 3) would be empty, causing any transaction relying on a snapshot at this point to potentially corrupt data as there could still be some 2PC transactions to track, with RecentXmin moving backwards on successive calls to GetSnapshotData() in the same transaction. As SharedRecoveryState is the point to take into account to know if it is safe to discard KnownAssignedXids, this commit moves step 2) after step 3), so as we can never finish with empty snapshots. This exists since the introduction of hot standby, so backpatch all the way down. The window with incorrect snapshots is extremely small, but I have seen it when running 023_pitr_prepared_xact.pl, as did buildfarm member fairywren. Thomas Munro also found it independently. Special thanks to Andres Freund for taking the time to analyze this issue. Reported-by: Thomas Munro, Michael Paquier Analyzed-by: Andres Freund Discussion: https://postgr.es/m/20210422203603.fdnh3fu2mmfp2iov@alap3.anarazel.de Backpatch-through: 9.6 Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/8a4237908c0fe73dd41d4d7c7a6314f17dfd7a6f Modified Files -------------- src/backend/access/transam/xlog.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix snapshot builds during promotion of hot standby node with 2P @ 2021-10-04 05:07 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 7+ messages in thread From: Michael Paquier @ 2021-10-04 05:07 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix snapshot builds during promotion of hot standby node with 2PC Some specific logic is done at the end of recovery when involving 2PC transactions: 1) Call RecoverPreparedTransactions(), to recover the state of 2PC transactions into memory (re-acquire locks, etc.). 2) ShutdownRecoveryTransactionEnvironment(), to move back to normal operations, mainly cleaning up recovery locks and KnownAssignedXids (including any 2PC transaction tracked previously). 3) Switch XLogCtl->SharedRecoveryState to RECOVERY_STATE_DONE, which is the tipping point for any process calling RecoveryInProgress() to check if the cluster is still in recovery or not. Any snapshot taken between steps 2) and 3) would be empty, causing any transaction relying on a snapshot at this point to potentially corrupt data as there could still be some 2PC transactions to track, with RecentXmin moving backwards on successive calls to GetSnapshotData() in the same transaction. As SharedRecoveryState is the point to take into account to know if it is safe to discard KnownAssignedXids, this commit moves step 2) after step 3), so as we can never finish with empty snapshots. This exists since the introduction of hot standby, so backpatch all the way down. The window with incorrect snapshots is extremely small, but I have seen it when running 023_pitr_prepared_xact.pl, as did buildfarm member fairywren. Thomas Munro also found it independently. Special thanks to Andres Freund for taking the time to analyze this issue. Reported-by: Thomas Munro, Michael Paquier Analyzed-by: Andres Freund Discussion: https://postgr.es/m/20210422203603.fdnh3fu2mmfp2iov@alap3.anarazel.de Backpatch-through: 9.6 Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/828f7f0009edda3f78dd789102cd02a394f94938 Modified Files -------------- src/backend/access/transam/xlog.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix snapshot builds during promotion of hot standby node with 2P @ 2021-10-04 05:07 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 7+ messages in thread From: Michael Paquier @ 2021-10-04 05:07 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix snapshot builds during promotion of hot standby node with 2PC Some specific logic is done at the end of recovery when involving 2PC transactions: 1) Call RecoverPreparedTransactions(), to recover the state of 2PC transactions into memory (re-acquire locks, etc.). 2) ShutdownRecoveryTransactionEnvironment(), to move back to normal operations, mainly cleaning up recovery locks and KnownAssignedXids (including any 2PC transaction tracked previously). 3) Switch XLogCtl->SharedRecoveryState to RECOVERY_STATE_DONE, which is the tipping point for any process calling RecoveryInProgress() to check if the cluster is still in recovery or not. Any snapshot taken between steps 2) and 3) would be empty, causing any transaction relying on a snapshot at this point to potentially corrupt data as there could still be some 2PC transactions to track, with RecentXmin moving backwards on successive calls to GetSnapshotData() in the same transaction. As SharedRecoveryState is the point to take into account to know if it is safe to discard KnownAssignedXids, this commit moves step 2) after step 3), so as we can never finish with empty snapshots. This exists since the introduction of hot standby, so backpatch all the way down. The window with incorrect snapshots is extremely small, but I have seen it when running 023_pitr_prepared_xact.pl, as did buildfarm member fairywren. Thomas Munro also found it independently. Special thanks to Andres Freund for taking the time to analyze this issue. Reported-by: Thomas Munro, Michael Paquier Analyzed-by: Andres Freund Discussion: https://postgr.es/m/20210422203603.fdnh3fu2mmfp2iov@alap3.anarazel.de Backpatch-through: 9.6 Branch ------ REL_13_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/194e535a07b3c36738b67f59aaa1291276929e9b Modified Files -------------- src/backend/access/transam/xlog.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix snapshot builds during promotion of hot standby node with 2P @ 2021-10-04 05:07 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 7+ messages in thread From: Michael Paquier @ 2021-10-04 05:07 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix snapshot builds during promotion of hot standby node with 2PC Some specific logic is done at the end of recovery when involving 2PC transactions: 1) Call RecoverPreparedTransactions(), to recover the state of 2PC transactions into memory (re-acquire locks, etc.). 2) ShutdownRecoveryTransactionEnvironment(), to move back to normal operations, mainly cleaning up recovery locks and KnownAssignedXids (including any 2PC transaction tracked previously). 3) Switch XLogCtl->SharedRecoveryState to RECOVERY_STATE_DONE, which is the tipping point for any process calling RecoveryInProgress() to check if the cluster is still in recovery or not. Any snapshot taken between steps 2) and 3) would be empty, causing any transaction relying on a snapshot at this point to potentially corrupt data as there could still be some 2PC transactions to track, with RecentXmin moving backwards on successive calls to GetSnapshotData() in the same transaction. As SharedRecoveryState is the point to take into account to know if it is safe to discard KnownAssignedXids, this commit moves step 2) after step 3), so as we can never finish with empty snapshots. This exists since the introduction of hot standby, so backpatch all the way down. The window with incorrect snapshots is extremely small, but I have seen it when running 023_pitr_prepared_xact.pl, as did buildfarm member fairywren. Thomas Munro also found it independently. Special thanks to Andres Freund for taking the time to analyze this issue. Reported-by: Thomas Munro, Michael Paquier Analyzed-by: Andres Freund Discussion: https://postgr.es/m/20210422203603.fdnh3fu2mmfp2iov@alap3.anarazel.de Backpatch-through: 9.6 Branch ------ REL_12_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/3c3f118d503ceebc01366ed1c09c967bac153405 Modified Files -------------- src/backend/access/transam/xlog.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix snapshot builds during promotion of hot standby node with 2P @ 2021-10-04 05:07 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 7+ messages in thread From: Michael Paquier @ 2021-10-04 05:07 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix snapshot builds during promotion of hot standby node with 2PC Some specific logic is done at the end of recovery when involving 2PC transactions: 1) Call RecoverPreparedTransactions(), to recover the state of 2PC transactions into memory (re-acquire locks, etc.). 2) ShutdownRecoveryTransactionEnvironment(), to move back to normal operations, mainly cleaning up recovery locks and KnownAssignedXids (including any 2PC transaction tracked previously). 3) Switch XLogCtl->SharedRecoveryState to RECOVERY_STATE_DONE, which is the tipping point for any process calling RecoveryInProgress() to check if the cluster is still in recovery or not. Any snapshot taken between steps 2) and 3) would be empty, causing any transaction relying on a snapshot at this point to potentially corrupt data as there could still be some 2PC transactions to track, with RecentXmin moving backwards on successive calls to GetSnapshotData() in the same transaction. As SharedRecoveryState is the point to take into account to know if it is safe to discard KnownAssignedXids, this commit moves step 2) after step 3), so as we can never finish with empty snapshots. This exists since the introduction of hot standby, so backpatch all the way down. The window with incorrect snapshots is extremely small, but I have seen it when running 023_pitr_prepared_xact.pl, as did buildfarm member fairywren. Thomas Munro also found it independently. Special thanks to Andres Freund for taking the time to analyze this issue. Reported-by: Thomas Munro, Michael Paquier Analyzed-by: Andres Freund Discussion: https://postgr.es/m/20210422203603.fdnh3fu2mmfp2iov@alap3.anarazel.de Backpatch-through: 9.6 Branch ------ REL_11_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/0a561d4d00166b2171579017e2cd8e57dcf83d6a Modified Files -------------- src/backend/access/transam/xlog.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2021-10-04 05:07 UTC | newest] Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2021-10-04 05:07 pgsql: Fix snapshot builds during promotion of hot standby node with 2P Michael Paquier <michael@paquier.xyz> 2021-10-04 05:07 pgsql: Fix snapshot builds during promotion of hot standby node with 2P Michael Paquier <michael@paquier.xyz> 2021-10-04 05:07 pgsql: Fix snapshot builds during promotion of hot standby node with 2P Michael Paquier <michael@paquier.xyz> 2021-10-04 05:07 pgsql: Fix snapshot builds during promotion of hot standby node with 2P Michael Paquier <michael@paquier.xyz> 2021-10-04 05:07 pgsql: Fix snapshot builds during promotion of hot standby node with 2P Michael Paquier <michael@paquier.xyz> 2021-10-04 05:07 pgsql: Fix snapshot builds during promotion of hot standby node with 2P Michael Paquier <michael@paquier.xyz> 2021-10-04 05:07 pgsql: Fix snapshot builds during promotion of hot standby node with 2P Michael Paquier <michael@paquier.xyz>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox