agora inbox for pgsql-committers@postgresql.orghelp / color / mirror / Atom feed
pgsql: Wait for transactions of an initial decoding snapshot to commit 7+ messages / 1 participants [nested] [flat]
* pgsql: Wait for transactions of an initial decoding snapshot to commit @ 2026-09-24 10:18 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 7+ messages in thread From: Álvaro Herrera @ 2026-09-24 10:18 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Wait for transactions of an initial decoding snapshot to commit SnapBuildInitialSnapshot() converts the snapshot builder's list of committed transactions into a regular MVCC snapshot, which is then used with HeapTupleSatisfiesMVCC(). However, a snapshot produced that way and used as an MVCC snapshot can potentially cause data corruption, if it captures a transaction after it writes its commit WAL record but before it has updated its CLOG entry: it will be incorrectly used to set hint bits as if that transaction had aborted. Fix by having SnapBuildInitialSnapshot() wait until such transactions have removed themselves from procarray, which guarantees correct visibility. Other uses of SnapBuildBuildSnapshot only use the snapshot as historical, so they don't need the waits, but add commentary in that function to alert possible future callers. While at this, backpatch commit 504fe10d1d26, "Fix misplaced comment in snapbuild.c." The comment it moved was nearby the place we modify. This bug was diagnosed by stress-testing of REPACK (CONCURRENTLY) in branch 19, so add a test case that tickles it using that feature in that branch and master. However, the bug exists in all branches, and in branch 18 we can add a test with regular logical decoding using injection points that tickles it. Unfortunately that test cannot be backpatched to earlier branches for lack of facilities. Reported-by: Mihail Nikalayeu <mihailnikalayeu@gmail.com> Author: Antonin Houska <ah@cybertec.at> Author: Rui Zhao <zhaorui126@gmail.com> Backpatch-through: 14 Discussion: https://postgr.es/m/85833.1768840165@localhost Discussion: https://postgr.es/m/CADzfLwU78as45To9a%3D-Qkr5jEg3tMxc5rUtdKy2MTv4r_SDGng%40mail.gmail.com Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/46024c573bcb10b323874651702581cc312f4425 Modified Files -------------- src/backend/access/transam/xact.c | 10 ++ src/backend/replication/logical/snapbuild.c | 39 ++++++ src/test/modules/injection_points/Makefile | 1 + .../expected/repack_commit_race.out | 64 +++++++++ .../modules/injection_points/injection_points.c | 11 +- src/test/modules/injection_points/meson.build | 1 + .../injection_points/specs/repack_commit_race.spec | 95 +++++++++++++ src/test/recovery/meson.build | 1 + src/test/recovery/t/057_snapshot_commit_race.pl | 148 +++++++++++++++++++++ 9 files changed, 367 insertions(+), 3 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Wait for transactions of an initial decoding snapshot to commit @ 2026-09-24 10:18 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 7+ messages in thread From: Álvaro Herrera @ 2026-09-24 10:18 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Wait for transactions of an initial decoding snapshot to commit SnapBuildInitialSnapshot() converts the snapshot builder's list of committed transactions into a regular MVCC snapshot, which is then used with HeapTupleSatisfiesMVCC(). However, a snapshot produced that way and used as an MVCC snapshot can potentially cause data corruption, if it captures a transaction after it writes its commit WAL record but before it has updated its CLOG entry: it will be incorrectly used to set hint bits as if that transaction had aborted. Fix by having SnapBuildInitialSnapshot() wait until such transactions have removed themselves from procarray, which guarantees correct visibility. Other uses of SnapBuildBuildSnapshot only use the snapshot as historical, so they don't need the waits, but add commentary in that function to alert possible future callers. While at this, backpatch commit 504fe10d1d26, "Fix misplaced comment in snapbuild.c." The comment it moved was nearby the place we modify. This bug was diagnosed by stress-testing of REPACK (CONCURRENTLY) in branch 19, so add a test case that tickles it using that feature in that branch and master. However, the bug exists in all branches, and in branch 18 we can add a test with regular logical decoding using injection points that tickles it. Unfortunately that test cannot be backpatched to earlier branches for lack of facilities. Reported-by: Mihail Nikalayeu <mihailnikalayeu@gmail.com> Author: Antonin Houska <ah@cybertec.at> Author: Rui Zhao <zhaorui126@gmail.com> Backpatch-through: 14 Discussion: https://postgr.es/m/85833.1768840165@localhost Discussion: https://postgr.es/m/CADzfLwU78as45To9a%3D-Qkr5jEg3tMxc5rUtdKy2MTv4r_SDGng%40mail.gmail.com Branch ------ REL_18_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/6210f00cac2fa62b24881b31130bdb2224f7b950 Modified Files -------------- src/backend/access/transam/xact.c | 10 ++ src/backend/replication/logical/snapbuild.c | 46 +++++++- src/test/recovery/meson.build | 1 + src/test/recovery/t/057_snapshot_commit_race.pl | 148 ++++++++++++++++++++++++ 4 files changed, 202 insertions(+), 3 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Wait for transactions of an initial decoding snapshot to commit @ 2026-09-24 10:18 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 7+ messages in thread From: Álvaro Herrera @ 2026-09-24 10:18 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Wait for transactions of an initial decoding snapshot to commit SnapBuildInitialSnapshot() converts the snapshot builder's list of committed transactions into a regular MVCC snapshot, which is then used with HeapTupleSatisfiesMVCC(). However, a snapshot produced that way and used as an MVCC snapshot can potentially cause data corruption, if it captures a transaction after it writes its commit WAL record but before it has updated its CLOG entry: it will be incorrectly used to set hint bits as if that transaction had aborted. Fix by having SnapBuildInitialSnapshot() wait until such transactions have removed themselves from procarray, which guarantees correct visibility. Other uses of SnapBuildBuildSnapshot only use the snapshot as historical, so they don't need the waits, but add commentary in that function to alert possible future callers. While at this, backpatch commit 504fe10d1d26, "Fix misplaced comment in snapbuild.c." The comment it moved was nearby the place we modify. This bug was diagnosed by stress-testing of REPACK (CONCURRENTLY) in branch 19, so add a test case that tickles it using that feature in that branch and master. However, the bug exists in all branches, and in branch 18 we can add a test with regular logical decoding using injection points that tickles it. Unfortunately that test cannot be backpatched to earlier branches for lack of facilities. Reported-by: Mihail Nikalayeu <mihailnikalayeu@gmail.com> Author: Antonin Houska <ah@cybertec.at> Author: Rui Zhao <zhaorui126@gmail.com> Backpatch-through: 14 Discussion: https://postgr.es/m/85833.1768840165@localhost Discussion: https://postgr.es/m/CADzfLwU78as45To9a%3D-Qkr5jEg3tMxc5rUtdKy2MTv4r_SDGng%40mail.gmail.com Branch ------ REL_17_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/b93df6eacb76fdb89a7bb4c0aac1b2e62814c242 Modified Files -------------- src/backend/replication/logical/snapbuild.c | 46 +++++++++++++++++++++++++++-- 1 file changed, 43 insertions(+), 3 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Wait for transactions of an initial decoding snapshot to commit @ 2026-09-24 10:18 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 7+ messages in thread From: Álvaro Herrera @ 2026-09-24 10:18 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Wait for transactions of an initial decoding snapshot to commit SnapBuildInitialSnapshot() converts the snapshot builder's list of committed transactions into a regular MVCC snapshot, which is then used with HeapTupleSatisfiesMVCC(). However, a snapshot produced that way and used as an MVCC snapshot can potentially cause data corruption, if it captures a transaction after it writes its commit WAL record but before it has updated its CLOG entry: it will be incorrectly used to set hint bits as if that transaction had aborted. Fix by having SnapBuildInitialSnapshot() wait until such transactions have removed themselves from procarray, which guarantees correct visibility. Other uses of SnapBuildBuildSnapshot only use the snapshot as historical, so they don't need the waits, but add commentary in that function to alert possible future callers. While at this, backpatch commit 504fe10d1d26, "Fix misplaced comment in snapbuild.c." The comment it moved was nearby the place we modify. This bug was diagnosed by stress-testing of REPACK (CONCURRENTLY) in branch 19, so add a test case that tickles it using that feature in that branch and master. However, the bug exists in all branches, and in branch 18 we can add a test with regular logical decoding using injection points that tickles it. Unfortunately that test cannot be backpatched to earlier branches for lack of facilities. Reported-by: Mihail Nikalayeu <mihailnikalayeu@gmail.com> Author: Antonin Houska <ah@cybertec.at> Author: Rui Zhao <zhaorui126@gmail.com> Backpatch-through: 14 Discussion: https://postgr.es/m/85833.1768840165@localhost Discussion: https://postgr.es/m/CADzfLwU78as45To9a%3D-Qkr5jEg3tMxc5rUtdKy2MTv4r_SDGng%40mail.gmail.com Branch ------ REL_16_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/a33ffaf7b8b3645fd8f93e891cc6ff6ed640e674 Modified Files -------------- src/backend/replication/logical/snapbuild.c | 46 +++++++++++++++++++++++++++-- 1 file changed, 43 insertions(+), 3 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Wait for transactions of an initial decoding snapshot to commit @ 2026-09-24 10:18 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 7+ messages in thread From: Álvaro Herrera @ 2026-09-24 10:18 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Wait for transactions of an initial decoding snapshot to commit SnapBuildInitialSnapshot() converts the snapshot builder's list of committed transactions into a regular MVCC snapshot, which is then used with HeapTupleSatisfiesMVCC(). However, a snapshot produced that way and used as an MVCC snapshot can potentially cause data corruption, if it captures a transaction after it writes its commit WAL record but before it has updated its CLOG entry: it will be incorrectly used to set hint bits as if that transaction had aborted. Fix by having SnapBuildInitialSnapshot() wait until such transactions have removed themselves from procarray, which guarantees correct visibility. Other uses of SnapBuildBuildSnapshot only use the snapshot as historical, so they don't need the waits, but add commentary in that function to alert possible future callers. While at this, backpatch commit 504fe10d1d26, "Fix misplaced comment in snapbuild.c." The comment it moved was nearby the place we modify. This bug was diagnosed by stress-testing of REPACK (CONCURRENTLY) in branch 19, so add a test case that tickles it using that feature in that branch and master. However, the bug exists in all branches, and in branch 18 we can add a test with regular logical decoding using injection points that tickles it. Unfortunately that test cannot be backpatched to earlier branches for lack of facilities. Reported-by: Mihail Nikalayeu <mihailnikalayeu@gmail.com> Author: Antonin Houska <ah@cybertec.at> Author: Rui Zhao <zhaorui126@gmail.com> Backpatch-through: 14 Discussion: https://postgr.es/m/85833.1768840165@localhost Discussion: https://postgr.es/m/CADzfLwU78as45To9a%3D-Qkr5jEg3tMxc5rUtdKy2MTv4r_SDGng%40mail.gmail.com Branch ------ REL_15_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/5ff6a3d3f8f8b5712ba57d4c4cf352aaeea3aab8 Modified Files -------------- src/backend/replication/logical/snapbuild.c | 50 +++++++++++++++++++++++++++-- 1 file changed, 47 insertions(+), 3 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Wait for transactions of an initial decoding snapshot to commit @ 2026-09-24 10:18 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 7+ messages in thread From: Álvaro Herrera @ 2026-09-24 10:18 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Wait for transactions of an initial decoding snapshot to commit SnapBuildInitialSnapshot() converts the snapshot builder's list of committed transactions into a regular MVCC snapshot, which is then used with HeapTupleSatisfiesMVCC(). However, a snapshot produced that way and used as an MVCC snapshot can potentially cause data corruption, if it captures a transaction after it writes its commit WAL record but before it has updated its CLOG entry: it will be incorrectly used to set hint bits as if that transaction had aborted. Fix by having SnapBuildInitialSnapshot() wait until such transactions have removed themselves from procarray, which guarantees correct visibility. Other uses of SnapBuildBuildSnapshot only use the snapshot as historical, so they don't need the waits, but add commentary in that function to alert possible future callers. While at this, backpatch commit 504fe10d1d26, "Fix misplaced comment in snapbuild.c." The comment it moved was nearby the place we modify. This bug was diagnosed by stress-testing of REPACK (CONCURRENTLY) in branch 19, so add a test case that tickles it using that feature in that branch and master. However, the bug exists in all branches, and in branch 18 we can add a test with regular logical decoding using injection points that tickles it. Unfortunately that test cannot be backpatched to earlier branches for lack of facilities. Reported-by: Mihail Nikalayeu <mihailnikalayeu@gmail.com> Author: Antonin Houska <ah@cybertec.at> Author: Rui Zhao <zhaorui126@gmail.com> Backpatch-through: 14 Discussion: https://postgr.es/m/85833.1768840165@localhost Discussion: https://postgr.es/m/CADzfLwU78as45To9a%3D-Qkr5jEg3tMxc5rUtdKy2MTv4r_SDGng%40mail.gmail.com Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/070e21268b2cde728dbd4aae529ac591f26afb27 Modified Files -------------- src/backend/replication/logical/snapbuild.c | 50 +++++++++++++++++++++++++++-- 1 file changed, 47 insertions(+), 3 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Wait for transactions of an initial decoding snapshot to commit @ 2026-09-24 10:25 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 7+ messages in thread From: Álvaro Herrera @ 2026-09-24 10:25 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Wait for transactions of an initial decoding snapshot to commit SnapBuildInitialSnapshot() converts the snapshot builder's list of committed transactions into a regular MVCC snapshot, which is then used with HeapTupleSatisfiesMVCC(). However, a snapshot produced that way and used as an MVCC snapshot can potentially cause data corruption, if it captures a transaction after it writes its commit WAL record but before it has updated its CLOG entry: it will be incorrectly used to set hint bits as if that transaction had aborted. Fix by having SnapBuildInitialSnapshot() wait until such transactions have removed themselves from procarray, which guarantees correct visibility. Other uses of SnapBuildBuildSnapshot only use the snapshot as historical, so they don't need the waits, but add commentary in that function to alert possible future callers. While at this, backpatch commit 504fe10d1d26, "Fix misplaced comment in snapbuild.c." The comment it moved was nearby the place we modify. This bug was diagnosed by stress-testing of REPACK (CONCURRENTLY) in branch 19, so add a test case that tickles it using that feature in that branch and master. However, the bug exists in all branches, and in branch 18 we can add a test with regular logical decoding using injection points that tickles it. Unfortunately that test cannot be backpatched to earlier branches for lack of facilities. Reported-by: Mihail Nikalayeu <mihailnikalayeu@gmail.com> Author: Antonin Houska <ah@cybertec.at> Author: Rui Zhao <zhaorui126@gmail.com> Backpatch-through: 14 Discussion: https://postgr.es/m/85833.1768840165@localhost Discussion: https://postgr.es/m/CADzfLwU78as45To9a%3D-Qkr5jEg3tMxc5rUtdKy2MTv4r_SDGng%40mail.gmail.com Branch ------ REL_19_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/4a4bf490dc50600462979d1e8ba7255a5975dbc1 Modified Files -------------- src/backend/access/transam/xact.c | 10 ++ src/backend/replication/logical/snapbuild.c | 46 ++++++- src/test/modules/injection_points/Makefile | 1 + .../expected/repack_commit_race.out | 64 +++++++++ .../modules/injection_points/injection_points.c | 11 +- src/test/modules/injection_points/meson.build | 1 + .../injection_points/specs/repack_commit_race.spec | 95 +++++++++++++ src/test/recovery/meson.build | 1 + src/test/recovery/t/057_snapshot_commit_race.pl | 148 +++++++++++++++++++++ 9 files changed, 371 insertions(+), 6 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2026-09-24 10:25 UTC | newest] Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2026-09-24 10:18 pgsql: Wait for transactions of an initial decoding snapshot to commit Álvaro Herrera <alvherre@kurilemu.de> 2026-09-24 10:18 pgsql: Wait for transactions of an initial decoding snapshot to commit Álvaro Herrera <alvherre@kurilemu.de> 2026-09-24 10:18 pgsql: Wait for transactions of an initial decoding snapshot to commit Álvaro Herrera <alvherre@kurilemu.de> 2026-09-24 10:18 pgsql: Wait for transactions of an initial decoding snapshot to commit Álvaro Herrera <alvherre@kurilemu.de> 2026-09-24 10:18 pgsql: Wait for transactions of an initial decoding snapshot to commit Álvaro Herrera <alvherre@kurilemu.de> 2026-09-24 10:18 pgsql: Wait for transactions of an initial decoding snapshot to commit Álvaro Herrera <alvherre@kurilemu.de> 2026-09-24 10:25 pgsql: Wait for transactions of an initial decoding snapshot to commit Álvaro Herrera <alvherre@kurilemu.de>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox