pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: Fix xmin advancement during fast_forward decoding. 6+ messages / 1 participants [nested] [flat]
* pgsql: Fix xmin advancement during fast_forward decoding. @ 2025-04-28 06:35 Amit Kapila <akapila@postgresql.org> 0 siblings, 0 replies; 6+ messages in thread From: Amit Kapila @ 2025-04-28 06:35 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix xmin advancement during fast_forward decoding. During logical decoding, we advance catalog_xmin of logical too early in fast_forward mode, resulting in required catalog data being removed by vacuum. This mode is normally used to advance the slot without processing the changes, but we still can't let the slot's xmin to advance to an incorrect value. Commit f49a80c481 fixed a similar issue where the logical slot's catalog_xmin was getting advanced prematurely during non-fast-forward mode. During xl_running_xacts processing, instead of directly advancing the slot's xmin to the oldest running xid in the record, it allowed the xmin to be held back for snapshots that can be used for not-yet-replayed transactions, as those might consider older txns as running too. However, it missed the fact that the same problem can happen during fast_forward mode decoding, as we won't build a base snapshot in that mode, and the future call to get_changes from the same slot can miss seeing the required catalog changes leading to incorrect reslts. This commit allows building the base snapshot even in fast_forward mode to prevent the early advancement of xmin. Reported-by: Amit Kapila <amit.kapila16@gmail.com> Author: Zhijie Hou <houzj.fnst@fujitsu.com> Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com> Reviewed-by: shveta malik <shveta.malik@gmail.com> Reviewed-by: Amit Kapila <amit.kapila16@gmail.com> Backpatch-through: 13 Discussion: https://postgr.es/m/CAA4eK1LqWncUOqKijiafe+Ypt1gQAQRjctKLMY953J79xDBgAg@mail.gmail.com Discussion: https://postgr.es/m/OS0PR01MB57163087F86621D44D9A72BF94BB2@OS0PR01MB5716.jpnprd01.prod.outlook.com Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/aaf9e95e8764093ab55baf27c35705f406f592f9 Modified Files -------------- contrib/test_decoding/expected/oldest_xmin.out | 41 ++++++++++++++++++++++++++ contrib/test_decoding/specs/oldest_xmin.spec | 5 ++++ src/backend/replication/logical/decode.c | 37 +++++++++++++++-------- 3 files changed, 71 insertions(+), 12 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix xmin advancement during fast_forward decoding. @ 2025-04-28 06:35 Amit Kapila <akapila@postgresql.org> 0 siblings, 0 replies; 6+ messages in thread From: Amit Kapila @ 2025-04-28 06:35 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix xmin advancement during fast_forward decoding. During logical decoding, we advance catalog_xmin of logical too early in fast_forward mode, resulting in required catalog data being removed by vacuum. This mode is normally used to advance the slot without processing the changes, but we still can't let the slot's xmin to advance to an incorrect value. Commit f49a80c481 fixed a similar issue where the logical slot's catalog_xmin was getting advanced prematurely during non-fast-forward mode. During xl_running_xacts processing, instead of directly advancing the slot's xmin to the oldest running xid in the record, it allowed the xmin to be held back for snapshots that can be used for not-yet-replayed transactions, as those might consider older txns as running too. However, it missed the fact that the same problem can happen during fast_forward mode decoding, as we won't build a base snapshot in that mode, and the future call to get_changes from the same slot can miss seeing the required catalog changes leading to incorrect reslts. This commit allows building the base snapshot even in fast_forward mode to prevent the early advancement of xmin. Reported-by: Amit Kapila <amit.kapila16@gmail.com> Author: Zhijie Hou <houzj.fnst@fujitsu.com> Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com> Reviewed-by: shveta malik <shveta.malik@gmail.com> Reviewed-by: Amit Kapila <amit.kapila16@gmail.com> Backpatch-through: 13 Discussion: https://postgr.es/m/CAA4eK1LqWncUOqKijiafe+Ypt1gQAQRjctKLMY953J79xDBgAg@mail.gmail.com Discussion: https://postgr.es/m/OS0PR01MB57163087F86621D44D9A72BF94BB2@OS0PR01MB5716.jpnprd01.prod.outlook.com Branch ------ REL_17_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/36148b22ee09f504e049e7da2495d32ca1c265bd Modified Files -------------- contrib/test_decoding/expected/oldest_xmin.out | 41 ++++++++++++++++++++++++++ contrib/test_decoding/specs/oldest_xmin.spec | 5 ++++ src/backend/replication/logical/decode.c | 37 +++++++++++++++-------- 3 files changed, 71 insertions(+), 12 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix xmin advancement during fast_forward decoding. @ 2025-04-28 06:35 Amit Kapila <akapila@postgresql.org> 0 siblings, 0 replies; 6+ messages in thread From: Amit Kapila @ 2025-04-28 06:35 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix xmin advancement during fast_forward decoding. During logical decoding, we advance catalog_xmin of logical too early in fast_forward mode, resulting in required catalog data being removed by vacuum. This mode is normally used to advance the slot without processing the changes, but we still can't let the slot's xmin to advance to an incorrect value. Commit f49a80c481 fixed a similar issue where the logical slot's catalog_xmin was getting advanced prematurely during non-fast-forward mode. During xl_running_xacts processing, instead of directly advancing the slot's xmin to the oldest running xid in the record, it allowed the xmin to be held back for snapshots that can be used for not-yet-replayed transactions, as those might consider older txns as running too. However, it missed the fact that the same problem can happen during fast_forward mode decoding, as we won't build a base snapshot in that mode, and the future call to get_changes from the same slot can miss seeing the required catalog changes leading to incorrect reslts. This commit allows building the base snapshot even in fast_forward mode to prevent the early advancement of xmin. Reported-by: Amit Kapila <amit.kapila16@gmail.com> Author: Zhijie Hou <houzj.fnst@fujitsu.com> Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com> Reviewed-by: shveta malik <shveta.malik@gmail.com> Reviewed-by: Amit Kapila <amit.kapila16@gmail.com> Backpatch-through: 13 Discussion: https://postgr.es/m/CAA4eK1LqWncUOqKijiafe+Ypt1gQAQRjctKLMY953J79xDBgAg@mail.gmail.com Discussion: https://postgr.es/m/OS0PR01MB57163087F86621D44D9A72BF94BB2@OS0PR01MB5716.jpnprd01.prod.outlook.com Branch ------ REL_16_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/21a7caeeb948e3bdcf36d7aca785b7fb953c0e3a Modified Files -------------- contrib/test_decoding/expected/oldest_xmin.out | 41 ++++++++++++++++++++++++++ contrib/test_decoding/specs/oldest_xmin.spec | 5 ++++ src/backend/replication/logical/decode.c | 38 ++++++++++++++++-------- 3 files changed, 71 insertions(+), 13 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix xmin advancement during fast_forward decoding. @ 2025-04-28 06:35 Amit Kapila <akapila@postgresql.org> 0 siblings, 0 replies; 6+ messages in thread From: Amit Kapila @ 2025-04-28 06:35 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix xmin advancement during fast_forward decoding. During logical decoding, we advance catalog_xmin of logical too early in fast_forward mode, resulting in required catalog data being removed by vacuum. This mode is normally used to advance the slot without processing the changes, but we still can't let the slot's xmin to advance to an incorrect value. Commit f49a80c481 fixed a similar issue where the logical slot's catalog_xmin was getting advanced prematurely during non-fast-forward mode. During xl_running_xacts processing, instead of directly advancing the slot's xmin to the oldest running xid in the record, it allowed the xmin to be held back for snapshots that can be used for not-yet-replayed transactions, as those might consider older txns as running too. However, it missed the fact that the same problem can happen during fast_forward mode decoding, as we won't build a base snapshot in that mode, and the future call to get_changes from the same slot can miss seeing the required catalog changes leading to incorrect reslts. This commit allows building the base snapshot even in fast_forward mode to prevent the early advancement of xmin. Reported-by: Amit Kapila <amit.kapila16@gmail.com> Author: Zhijie Hou <houzj.fnst@fujitsu.com> Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com> Reviewed-by: shveta malik <shveta.malik@gmail.com> Reviewed-by: Amit Kapila <amit.kapila16@gmail.com> Backpatch-through: 13 Discussion: https://postgr.es/m/CAA4eK1LqWncUOqKijiafe+Ypt1gQAQRjctKLMY953J79xDBgAg@mail.gmail.com Discussion: https://postgr.es/m/OS0PR01MB57163087F86621D44D9A72BF94BB2@OS0PR01MB5716.jpnprd01.prod.outlook.com Branch ------ REL_15_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/f6429bd7db5e33aa673cc1ea2392346c86727806 Modified Files -------------- contrib/test_decoding/expected/oldest_xmin.out | 41 ++++++++++++++++++++++++++ contrib/test_decoding/specs/oldest_xmin.spec | 5 ++++ src/backend/replication/logical/decode.c | 38 ++++++++++++++++-------- 3 files changed, 71 insertions(+), 13 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix xmin advancement during fast_forward decoding. @ 2025-04-28 06:35 Amit Kapila <akapila@postgresql.org> 0 siblings, 0 replies; 6+ messages in thread From: Amit Kapila @ 2025-04-28 06:35 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix xmin advancement during fast_forward decoding. During logical decoding, we advance catalog_xmin of logical too early in fast_forward mode, resulting in required catalog data being removed by vacuum. This mode is normally used to advance the slot without processing the changes, but we still can't let the slot's xmin to advance to an incorrect value. Commit f49a80c481 fixed a similar issue where the logical slot's catalog_xmin was getting advanced prematurely during non-fast-forward mode. During xl_running_xacts processing, instead of directly advancing the slot's xmin to the oldest running xid in the record, it allowed the xmin to be held back for snapshots that can be used for not-yet-replayed transactions, as those might consider older txns as running too. However, it missed the fact that the same problem can happen during fast_forward mode decoding, as we won't build a base snapshot in that mode, and the future call to get_changes from the same slot can miss seeing the required catalog changes leading to incorrect reslts. This commit allows building the base snapshot even in fast_forward mode to prevent the early advancement of xmin. Reported-by: Amit Kapila <amit.kapila16@gmail.com> Author: Zhijie Hou <houzj.fnst@fujitsu.com> Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com> Reviewed-by: shveta malik <shveta.malik@gmail.com> Reviewed-by: Amit Kapila <amit.kapila16@gmail.com> Backpatch-through: 13 Discussion: https://postgr.es/m/CAA4eK1LqWncUOqKijiafe+Ypt1gQAQRjctKLMY953J79xDBgAg@mail.gmail.com Discussion: https://postgr.es/m/OS0PR01MB57163087F86621D44D9A72BF94BB2@OS0PR01MB5716.jpnprd01.prod.outlook.com Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/1f63b3626a52e596398c7a04a42081a773b12d76 Modified Files -------------- contrib/test_decoding/expected/oldest_xmin.out | 41 ++++++++++++++++++++++++++ contrib/test_decoding/specs/oldest_xmin.spec | 5 ++++ src/backend/replication/logical/decode.c | 38 ++++++++++++++++-------- 3 files changed, 71 insertions(+), 13 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix xmin advancement during fast_forward decoding. @ 2025-04-28 06:35 Amit Kapila <akapila@postgresql.org> 0 siblings, 0 replies; 6+ messages in thread From: Amit Kapila @ 2025-04-28 06:35 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix xmin advancement during fast_forward decoding. During logical decoding, we advance catalog_xmin of logical too early in fast_forward mode, resulting in required catalog data being removed by vacuum. This mode is normally used to advance the slot without processing the changes, but we still can't let the slot's xmin to advance to an incorrect value. Commit f49a80c481 fixed a similar issue where the logical slot's catalog_xmin was getting advanced prematurely during non-fast-forward mode. During xl_running_xacts processing, instead of directly advancing the slot's xmin to the oldest running xid in the record, it allowed the xmin to be held back for snapshots that can be used for not-yet-replayed transactions, as those might consider older txns as running too. However, it missed the fact that the same problem can happen during fast_forward mode decoding, as we won't build a base snapshot in that mode, and the future call to get_changes from the same slot can miss seeing the required catalog changes leading to incorrect reslts. This commit allows building the base snapshot even in fast_forward mode to prevent the early advancement of xmin. Reported-by: Amit Kapila <amit.kapila16@gmail.com> Author: Zhijie Hou <houzj.fnst@fujitsu.com> Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com> Reviewed-by: shveta malik <shveta.malik@gmail.com> Reviewed-by: Amit Kapila <amit.kapila16@gmail.com> Backpatch-through: 13 Discussion: https://postgr.es/m/CAA4eK1LqWncUOqKijiafe+Ypt1gQAQRjctKLMY953J79xDBgAg@mail.gmail.com Discussion: https://postgr.es/m/OS0PR01MB57163087F86621D44D9A72BF94BB2@OS0PR01MB5716.jpnprd01.prod.outlook.com Branch ------ REL_13_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/d65485b02b1f476cc14eb6f7a1cb29cd8fd0009f Modified Files -------------- contrib/test_decoding/expected/oldest_xmin.out | 41 ++++++++++++++++++++++++++ contrib/test_decoding/specs/oldest_xmin.spec | 5 ++++ src/backend/replication/logical/decode.c | 38 ++++++++++++++++-------- 3 files changed, 71 insertions(+), 13 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2025-04-28 06:35 UTC | newest] Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2025-04-28 06:35 pgsql: Fix xmin advancement during fast_forward decoding. Amit Kapila <akapila@postgresql.org> 2025-04-28 06:35 pgsql: Fix xmin advancement during fast_forward decoding. Amit Kapila <akapila@postgresql.org> 2025-04-28 06:35 pgsql: Fix xmin advancement during fast_forward decoding. Amit Kapila <akapila@postgresql.org> 2025-04-28 06:35 pgsql: Fix xmin advancement during fast_forward decoding. Amit Kapila <akapila@postgresql.org> 2025-04-28 06:35 pgsql: Fix xmin advancement during fast_forward decoding. Amit Kapila <akapila@postgresql.org> 2025-04-28 06:35 pgsql: Fix xmin advancement during fast_forward decoding. Amit Kapila <akapila@postgresql.org>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox