pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: Fix recovery of 2PC transaction during crash recovery 7+ messages / 1 participants [nested] [flat]
* pgsql: Fix recovery of 2PC transaction during crash recovery @ 2023-07-18 04:45 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 7+ messages in thread From: Michael Paquier @ 2023-07-18 04:45 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix recovery of 2PC transaction during crash recovery A crash in the middle of a checkpoint with some two-phase state data already flushed to disk by this checkpoint could cause a follow-up crash recovery to recover twice the same transaction, once from what has been found in pg_twophase/ at the beginning of recovery and a second time when replaying its corresponding record. This would lead to FATAL failures in the startup process during recovery, where the same transaction would have a state recovered twice instead of once: LOG: recovering prepared transaction 731 from shared memory LOG: recovering prepared transaction 731 from shared memory FATAL: lock ExclusiveLock on object 731/0/0 is already held This issue is fixed by skipping the addition of any 2PC state coming from a record whose equivalent 2PC state file has already been loaded in TwoPhaseState at the beginning of recovery by restoreTwoPhaseData(), which is OK as long as the system has not reached a consistent state. The timing to get a messed up recovery processing is very racy, and would very unlikely happen. The thread that has reported the issue has demonstrated the bug using injection points to force a PANIC in the middle of a checkpoint. Issue introduced in 728bd99, so backpatch all the way down. Reported-by: "suyu.cmj" <mengjuan.cmj@alibaba-inc.com> Author: "suyu.cmj" <mengjuan.cmj@alibaba-inc.com> Author: Michael Paquier Discussion: https://postgr.es/m/109e6994-b971-48cb-84f6-829646f18b4c.mengjuan.cmj@alibaba-inc.com Backpatch-through: 11 Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/cb0cca1880723b4c90c56cdcf3842489ef036800 Modified Files -------------- src/backend/access/transam/twophase.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix recovery of 2PC transaction during crash recovery @ 2023-07-18 04:45 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 7+ messages in thread From: Michael Paquier @ 2023-07-18 04:45 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix recovery of 2PC transaction during crash recovery A crash in the middle of a checkpoint with some two-phase state data already flushed to disk by this checkpoint could cause a follow-up crash recovery to recover twice the same transaction, once from what has been found in pg_twophase/ at the beginning of recovery and a second time when replaying its corresponding record. This would lead to FATAL failures in the startup process during recovery, where the same transaction would have a state recovered twice instead of once: LOG: recovering prepared transaction 731 from shared memory LOG: recovering prepared transaction 731 from shared memory FATAL: lock ExclusiveLock on object 731/0/0 is already held This issue is fixed by skipping the addition of any 2PC state coming from a record whose equivalent 2PC state file has already been loaded in TwoPhaseState at the beginning of recovery by restoreTwoPhaseData(), which is OK as long as the system has not reached a consistent state. The timing to get a messed up recovery processing is very racy, and would very unlikely happen. The thread that has reported the issue has demonstrated the bug using injection points to force a PANIC in the middle of a checkpoint. Issue introduced in 728bd99, so backpatch all the way down. Reported-by: "suyu.cmj" <mengjuan.cmj@alibaba-inc.com> Author: "suyu.cmj" <mengjuan.cmj@alibaba-inc.com> Author: Michael Paquier Discussion: https://postgr.es/m/109e6994-b971-48cb-84f6-829646f18b4c.mengjuan.cmj@alibaba-inc.com Backpatch-through: 11 Branch ------ REL_16_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/f88bc9f3883970327598271d26afdb525112874b Modified Files -------------- src/backend/access/transam/twophase.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix recovery of 2PC transaction during crash recovery @ 2023-07-18 04:45 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 7+ messages in thread From: Michael Paquier @ 2023-07-18 04:45 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix recovery of 2PC transaction during crash recovery A crash in the middle of a checkpoint with some two-phase state data already flushed to disk by this checkpoint could cause a follow-up crash recovery to recover twice the same transaction, once from what has been found in pg_twophase/ at the beginning of recovery and a second time when replaying its corresponding record. This would lead to FATAL failures in the startup process during recovery, where the same transaction would have a state recovered twice instead of once: LOG: recovering prepared transaction 731 from shared memory LOG: recovering prepared transaction 731 from shared memory FATAL: lock ExclusiveLock on object 731/0/0 is already held This issue is fixed by skipping the addition of any 2PC state coming from a record whose equivalent 2PC state file has already been loaded in TwoPhaseState at the beginning of recovery by restoreTwoPhaseData(), which is OK as long as the system has not reached a consistent state. The timing to get a messed up recovery processing is very racy, and would very unlikely happen. The thread that has reported the issue has demonstrated the bug using injection points to force a PANIC in the middle of a checkpoint. Issue introduced in 728bd99, so backpatch all the way down. Reported-by: "suyu.cmj" <mengjuan.cmj@alibaba-inc.com> Author: "suyu.cmj" <mengjuan.cmj@alibaba-inc.com> Author: Michael Paquier Discussion: https://postgr.es/m/109e6994-b971-48cb-84f6-829646f18b4c.mengjuan.cmj@alibaba-inc.com Backpatch-through: 11 Branch ------ REL_15_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/a878eff6b4b9600763db95ba9dd5725497847fa3 Modified Files -------------- src/backend/access/transam/twophase.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix recovery of 2PC transaction during crash recovery @ 2023-07-18 04:45 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 7+ messages in thread From: Michael Paquier @ 2023-07-18 04:45 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix recovery of 2PC transaction during crash recovery A crash in the middle of a checkpoint with some two-phase state data already flushed to disk by this checkpoint could cause a follow-up crash recovery to recover twice the same transaction, once from what has been found in pg_twophase/ at the beginning of recovery and a second time when replaying its corresponding record. This would lead to FATAL failures in the startup process during recovery, where the same transaction would have a state recovered twice instead of once: LOG: recovering prepared transaction 731 from shared memory LOG: recovering prepared transaction 731 from shared memory FATAL: lock ExclusiveLock on object 731/0/0 is already held This issue is fixed by skipping the addition of any 2PC state coming from a record whose equivalent 2PC state file has already been loaded in TwoPhaseState at the beginning of recovery by restoreTwoPhaseData(), which is OK as long as the system has not reached a consistent state. The timing to get a messed up recovery processing is very racy, and would very unlikely happen. The thread that has reported the issue has demonstrated the bug using injection points to force a PANIC in the middle of a checkpoint. Issue introduced in 728bd99, so backpatch all the way down. Reported-by: "suyu.cmj" <mengjuan.cmj@alibaba-inc.com> Author: "suyu.cmj" <mengjuan.cmj@alibaba-inc.com> Author: Michael Paquier Discussion: https://postgr.es/m/109e6994-b971-48cb-84f6-829646f18b4c.mengjuan.cmj@alibaba-inc.com Backpatch-through: 11 Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/442749100d34a213f71c73d98bc7b43478832e3a Modified Files -------------- src/backend/access/transam/twophase.c | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix recovery of 2PC transaction during crash recovery @ 2023-07-18 04:45 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 7+ messages in thread From: Michael Paquier @ 2023-07-18 04:45 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix recovery of 2PC transaction during crash recovery A crash in the middle of a checkpoint with some two-phase state data already flushed to disk by this checkpoint could cause a follow-up crash recovery to recover twice the same transaction, once from what has been found in pg_twophase/ at the beginning of recovery and a second time when replaying its corresponding record. This would lead to FATAL failures in the startup process during recovery, where the same transaction would have a state recovered twice instead of once: LOG: recovering prepared transaction 731 from shared memory LOG: recovering prepared transaction 731 from shared memory FATAL: lock ExclusiveLock on object 731/0/0 is already held This issue is fixed by skipping the addition of any 2PC state coming from a record whose equivalent 2PC state file has already been loaded in TwoPhaseState at the beginning of recovery by restoreTwoPhaseData(), which is OK as long as the system has not reached a consistent state. The timing to get a messed up recovery processing is very racy, and would very unlikely happen. The thread that has reported the issue has demonstrated the bug using injection points to force a PANIC in the middle of a checkpoint. Issue introduced in 728bd99, so backpatch all the way down. Reported-by: "suyu.cmj" <mengjuan.cmj@alibaba-inc.com> Author: "suyu.cmj" <mengjuan.cmj@alibaba-inc.com> Author: Michael Paquier Discussion: https://postgr.es/m/109e6994-b971-48cb-84f6-829646f18b4c.mengjuan.cmj@alibaba-inc.com Backpatch-through: 11 Branch ------ REL_13_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/db59108a2b6703bd359f01dc84665308c9d7ee73 Modified Files -------------- src/backend/access/transam/twophase.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix recovery of 2PC transaction during crash recovery @ 2023-07-18 04:45 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 7+ messages in thread From: Michael Paquier @ 2023-07-18 04:45 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix recovery of 2PC transaction during crash recovery A crash in the middle of a checkpoint with some two-phase state data already flushed to disk by this checkpoint could cause a follow-up crash recovery to recover twice the same transaction, once from what has been found in pg_twophase/ at the beginning of recovery and a second time when replaying its corresponding record. This would lead to FATAL failures in the startup process during recovery, where the same transaction would have a state recovered twice instead of once: LOG: recovering prepared transaction 731 from shared memory LOG: recovering prepared transaction 731 from shared memory FATAL: lock ExclusiveLock on object 731/0/0 is already held This issue is fixed by skipping the addition of any 2PC state coming from a record whose equivalent 2PC state file has already been loaded in TwoPhaseState at the beginning of recovery by restoreTwoPhaseData(), which is OK as long as the system has not reached a consistent state. The timing to get a messed up recovery processing is very racy, and would very unlikely happen. The thread that has reported the issue has demonstrated the bug using injection points to force a PANIC in the middle of a checkpoint. Issue introduced in 728bd99, so backpatch all the way down. Reported-by: "suyu.cmj" <mengjuan.cmj@alibaba-inc.com> Author: "suyu.cmj" <mengjuan.cmj@alibaba-inc.com> Author: Michael Paquier Discussion: https://postgr.es/m/109e6994-b971-48cb-84f6-829646f18b4c.mengjuan.cmj@alibaba-inc.com Backpatch-through: 11 Branch ------ REL_12_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/d2ee542a2bbec269115f7b1b4eb7ea59f879dc58 Modified Files -------------- src/backend/access/transam/twophase.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix recovery of 2PC transaction during crash recovery @ 2023-07-18 04:45 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 7+ messages in thread From: Michael Paquier @ 2023-07-18 04:45 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix recovery of 2PC transaction during crash recovery A crash in the middle of a checkpoint with some two-phase state data already flushed to disk by this checkpoint could cause a follow-up crash recovery to recover twice the same transaction, once from what has been found in pg_twophase/ at the beginning of recovery and a second time when replaying its corresponding record. This would lead to FATAL failures in the startup process during recovery, where the same transaction would have a state recovered twice instead of once: LOG: recovering prepared transaction 731 from shared memory LOG: recovering prepared transaction 731 from shared memory FATAL: lock ExclusiveLock on object 731/0/0 is already held This issue is fixed by skipping the addition of any 2PC state coming from a record whose equivalent 2PC state file has already been loaded in TwoPhaseState at the beginning of recovery by restoreTwoPhaseData(), which is OK as long as the system has not reached a consistent state. The timing to get a messed up recovery processing is very racy, and would very unlikely happen. The thread that has reported the issue has demonstrated the bug using injection points to force a PANIC in the middle of a checkpoint. Issue introduced in 728bd99, so backpatch all the way down. Reported-by: "suyu.cmj" <mengjuan.cmj@alibaba-inc.com> Author: "suyu.cmj" <mengjuan.cmj@alibaba-inc.com> Author: Michael Paquier Discussion: https://postgr.es/m/109e6994-b971-48cb-84f6-829646f18b4c.mengjuan.cmj@alibaba-inc.com Backpatch-through: 11 Branch ------ REL_11_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/bc0581f8fb8b52a497e7ea06454f276b99abb72b Modified Files -------------- src/backend/access/transam/twophase.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) ^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2023-07-18 04:45 UTC | newest] Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2023-07-18 04:45 pgsql: Fix recovery of 2PC transaction during crash recovery Michael Paquier <michael@paquier.xyz> 2023-07-18 04:45 pgsql: Fix recovery of 2PC transaction during crash recovery Michael Paquier <michael@paquier.xyz> 2023-07-18 04:45 pgsql: Fix recovery of 2PC transaction during crash recovery Michael Paquier <michael@paquier.xyz> 2023-07-18 04:45 pgsql: Fix recovery of 2PC transaction during crash recovery Michael Paquier <michael@paquier.xyz> 2023-07-18 04:45 pgsql: Fix recovery of 2PC transaction during crash recovery Michael Paquier <michael@paquier.xyz> 2023-07-18 04:45 pgsql: Fix recovery of 2PC transaction during crash recovery Michael Paquier <michael@paquier.xyz> 2023-07-18 04:45 pgsql: Fix recovery of 2PC transaction during crash recovery Michael Paquier <michael@paquier.xyz>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox