pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: Don't advance origin during apply failure. 7+ messages / 1 participants [nested] [flat]
* pgsql: Don't advance origin during apply failure. @ 2024-08-21 04:05 Amit Kapila <akapila@postgresql.org> 0 siblings, 0 replies; 7+ messages in thread From: Amit Kapila @ 2024-08-21 04:05 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Don't advance origin during apply failure. We advance origin progress during abort on successful streaming and application of ROLLBACK in parallel streaming mode. But the origin shouldn't be advanced during an error or unsuccessful apply due to shutdown. Otherwise, it will result in a transaction loss as such a transaction won't be sent again by the server. Reported-by: Hou Zhijie Author: Hayato Kuroda and Shveta Malik Reviewed-by: Amit Kapila Backpatch-through: 16 Discussion: https://postgr.es/m/TYAPR01MB5692FAC23BE40C69DA8ED4AFF5B92@TYAPR01MB5692.jpnprd01.prod.outlook.com Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/3f28b2fcac33fb352d261fac298cfe68c3899d32 Modified Files -------------- src/backend/replication/logical/worker.c | 35 ++++++++++++++++++++++++++++++++ src/backend/utils/error/elog.c | 17 ++++++++++++++++ src/include/utils/elog.h | 1 + src/test/subscription/t/021_twophase.pl | 14 ++++++++++++- 4 files changed, 66 insertions(+), 1 deletion(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Don't advance origin during apply failure. @ 2024-08-21 04:05 Amit Kapila <akapila@postgresql.org> 0 siblings, 0 replies; 7+ messages in thread From: Amit Kapila @ 2024-08-21 04:05 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Don't advance origin during apply failure. We advance origin progress during abort on successful streaming and application of ROLLBACK in parallel streaming mode. But the origin shouldn't be advanced during an error or unsuccessful apply due to shutdown. Otherwise, it will result in a transaction loss as such a transaction won't be sent again by the server. Reported-by: Hou Zhijie Author: Hayato Kuroda and Shveta Malik Reviewed-by: Amit Kapila Backpatch-through: 16 Discussion: https://postgr.es/m/TYAPR01MB5692FAC23BE40C69DA8ED4AFF5B92@TYAPR01MB5692.jpnprd01.prod.outlook.com Branch ------ REL_16_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/b39c5272c1d22d80a736647081b60a55a2d9d51e Modified Files -------------- src/backend/replication/logical/worker.c | 35 ++++++++++++++++++++++++++++++++ src/backend/utils/error/elog.c | 17 ++++++++++++++++ src/include/utils/elog.h | 1 + src/test/subscription/t/021_twophase.pl | 14 ++++++++++++- 4 files changed, 66 insertions(+), 1 deletion(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Don't advance origin during apply failure. @ 2024-08-21 04:05 Amit Kapila <akapila@postgresql.org> 0 siblings, 0 replies; 7+ messages in thread From: Amit Kapila @ 2024-08-21 04:05 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Don't advance origin during apply failure. We advance origin progress during abort on successful streaming and application of ROLLBACK in parallel streaming mode. But the origin shouldn't be advanced during an error or unsuccessful apply due to shutdown. Otherwise, it will result in a transaction loss as such a transaction won't be sent again by the server. Reported-by: Hou Zhijie Author: Hayato Kuroda and Shveta Malik Reviewed-by: Amit Kapila Backpatch-through: 16 Discussion: https://postgr.es/m/TYAPR01MB5692FAC23BE40C69DA8ED4AFF5B92@TYAPR01MB5692.jpnprd01.prod.outlook.com Branch ------ REL_17_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/915aafe82a7c31e9f7452e8cedf6371c318388bd Modified Files -------------- src/backend/replication/logical/worker.c | 35 ++++++++++++++++++++++++++++++++ src/backend/utils/error/elog.c | 17 ++++++++++++++++ src/include/utils/elog.h | 1 + src/test/subscription/t/021_twophase.pl | 14 ++++++++++++- 4 files changed, 66 insertions(+), 1 deletion(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Don't advance origin during apply failure. @ 2025-12-24 04:47 Amit Kapila <akapila@postgresql.org> 0 siblings, 0 replies; 7+ messages in thread From: Amit Kapila @ 2025-12-24 04:47 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Don't advance origin during apply failure. The logical replication parallel apply worker could incorrectly advance the origin progress during an error or failed apply. This behavior risks transaction loss because such transactions will not be resent by the server. Commit 3f28b2fcac addressed a similar issue for both the apply worker and the table sync worker by registering a before_shmem_exit callback to reset origin information. This prevents the worker from advancing the origin during transaction abortion on shutdown. This patch applies the same fix to the parallel apply worker, ensuring consistent behavior across all worker types. As with 3f28b2fcac, we are backpatching through version 16, since parallel apply mode was introduced there and the issue only occurs when changes are applied before the transaction end record (COMMIT or ABORT) is received. Author: Hou Zhijie <houzj.fnst@fujitsu.com> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Reviewed-by: Amit Kapila <amit.kapila16@gmail.com> Backpatch-through: 16 Discussion: https://postgr.es/m/TY4PR01MB169078771FB31B395AB496A6B94B4A@TY4PR01MB16907.jpnprd01.prod.outlook.com Discussion: https://postgr.es/m/TYAPR01MB5692FAC23BE40C69DA8ED4AFF5B92@TYAPR01MB5692.jpnprd01.prod.outlook.com Branch ------ REL_18_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/2f7ffe124a9ba0ddc477fa5643da5e59cc1e4db0 Modified Files -------------- src/backend/replication/logical/worker.c | 30 +++++++++-------- src/test/subscription/t/023_twophase_stream.pl | 45 ++++++++++++++++++++++++++ 2 files changed, 62 insertions(+), 13 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Don't advance origin during apply failure. @ 2025-12-24 04:47 Amit Kapila <akapila@postgresql.org> 0 siblings, 0 replies; 7+ messages in thread From: Amit Kapila @ 2025-12-24 04:47 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Don't advance origin during apply failure. The logical replication parallel apply worker could incorrectly advance the origin progress during an error or failed apply. This behavior risks transaction loss because such transactions will not be resent by the server. Commit 3f28b2fcac addressed a similar issue for both the apply worker and the table sync worker by registering a before_shmem_exit callback to reset origin information. This prevents the worker from advancing the origin during transaction abortion on shutdown. This patch applies the same fix to the parallel apply worker, ensuring consistent behavior across all worker types. As with 3f28b2fcac, we are backpatching through version 16, since parallel apply mode was introduced there and the issue only occurs when changes are applied before the transaction end record (COMMIT or ABORT) is received. Author: Hou Zhijie <houzj.fnst@fujitsu.com> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Reviewed-by: Amit Kapila <amit.kapila16@gmail.com> Backpatch-through: 16 Discussion: https://postgr.es/m/TY4PR01MB169078771FB31B395AB496A6B94B4A@TY4PR01MB16907.jpnprd01.prod.outlook.com Discussion: https://postgr.es/m/TYAPR01MB5692FAC23BE40C69DA8ED4AFF5B92@TYAPR01MB5692.jpnprd01.prod.outlook.com Branch ------ REL_16_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/63a65adf4d8ea85ad9870fdce9351f73be00e26d Modified Files -------------- src/backend/replication/logical/worker.c | 30 +++++++++-------- src/test/subscription/t/023_twophase_stream.pl | 45 ++++++++++++++++++++++++++ 2 files changed, 62 insertions(+), 13 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Don't advance origin during apply failure. @ 2025-12-24 04:47 Amit Kapila <akapila@postgresql.org> 0 siblings, 0 replies; 7+ messages in thread From: Amit Kapila @ 2025-12-24 04:47 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Don't advance origin during apply failure. The logical replication parallel apply worker could incorrectly advance the origin progress during an error or failed apply. This behavior risks transaction loss because such transactions will not be resent by the server. Commit 3f28b2fcac addressed a similar issue for both the apply worker and the table sync worker by registering a before_shmem_exit callback to reset origin information. This prevents the worker from advancing the origin during transaction abortion on shutdown. This patch applies the same fix to the parallel apply worker, ensuring consistent behavior across all worker types. As with 3f28b2fcac, we are backpatching through version 16, since parallel apply mode was introduced there and the issue only occurs when changes are applied before the transaction end record (COMMIT or ABORT) is received. Author: Hou Zhijie <houzj.fnst@fujitsu.com> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Reviewed-by: Amit Kapila <amit.kapila16@gmail.com> Backpatch-through: 16 Discussion: https://postgr.es/m/TY4PR01MB169078771FB31B395AB496A6B94B4A@TY4PR01MB16907.jpnprd01.prod.outlook.com Discussion: https://postgr.es/m/TYAPR01MB5692FAC23BE40C69DA8ED4AFF5B92@TYAPR01MB5692.jpnprd01.prod.outlook.com Branch ------ REL_17_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/0ed8f1afb15f5913458b0363c0eb818e758f97bd Modified Files -------------- src/backend/replication/logical/worker.c | 30 +++++++++-------- src/test/subscription/t/023_twophase_stream.pl | 45 ++++++++++++++++++++++++++ 2 files changed, 62 insertions(+), 13 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Don't advance origin during apply failure. @ 2025-12-24 04:47 Amit Kapila <akapila@postgresql.org> 0 siblings, 0 replies; 7+ messages in thread From: Amit Kapila @ 2025-12-24 04:47 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Don't advance origin during apply failure. The logical replication parallel apply worker could incorrectly advance the origin progress during an error or failed apply. This behavior risks transaction loss because such transactions will not be resent by the server. Commit 3f28b2fcac addressed a similar issue for both the apply worker and the table sync worker by registering a before_shmem_exit callback to reset origin information. This prevents the worker from advancing the origin during transaction abortion on shutdown. This patch applies the same fix to the parallel apply worker, ensuring consistent behavior across all worker types. As with 3f28b2fcac, we are backpatching through version 16, since parallel apply mode was introduced there and the issue only occurs when changes are applied before the transaction end record (COMMIT or ABORT) is received. Author: Hou Zhijie <houzj.fnst@fujitsu.com> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Reviewed-by: Amit Kapila <amit.kapila16@gmail.com> Backpatch-through: 16 Discussion: https://postgr.es/m/TY4PR01MB169078771FB31B395AB496A6B94B4A@TY4PR01MB16907.jpnprd01.prod.outlook.com Discussion: https://postgr.es/m/TYAPR01MB5692FAC23BE40C69DA8ED4AFF5B92@TYAPR01MB5692.jpnprd01.prod.outlook.com Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/1528b0d899d65e0b01120f7d06c23256d390e4fe Modified Files -------------- src/backend/replication/logical/worker.c | 30 +++++++++-------- src/test/subscription/t/023_twophase_stream.pl | 45 ++++++++++++++++++++++++++ 2 files changed, 62 insertions(+), 13 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2025-12-24 04:47 UTC | newest] Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2024-08-21 04:05 pgsql: Don't advance origin during apply failure. Amit Kapila <akapila@postgresql.org> 2024-08-21 04:05 pgsql: Don't advance origin during apply failure. Amit Kapila <akapila@postgresql.org> 2024-08-21 04:05 pgsql: Don't advance origin during apply failure. Amit Kapila <akapila@postgresql.org> 2025-12-24 04:47 pgsql: Don't advance origin during apply failure. Amit Kapila <akapila@postgresql.org> 2025-12-24 04:47 pgsql: Don't advance origin during apply failure. Amit Kapila <akapila@postgresql.org> 2025-12-24 04:47 pgsql: Don't advance origin during apply failure. Amit Kapila <akapila@postgresql.org> 2025-12-24 04:47 pgsql: Don't advance origin during apply failure. Amit Kapila <akapila@postgresql.org>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox