pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: Fix SPI's handling of errors during transaction commit. 5+ messages / 1 participants [nested] [flat]
* pgsql: Fix SPI's handling of errors during transaction commit. @ 2022-02-28 17:45 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 5+ messages in thread From: Tom Lane @ 2022-02-28 17:45 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix SPI's handling of errors during transaction commit. SPI_commit previously left it up to the caller to recover from any error occurring during commit. Since that's complicated and requires use of low-level xact.c facilities, it's not too surprising that no caller got it right. Let's move the responsibility for cleanup into spi.c. Doing that requires redefining SPI_commit as starting a new transaction, so that it becomes equivalent to SPI_commit_and_chain except that you get default transaction characteristics instead of preserving the prior transaction's characteristics. We can make this pretty transparent API-wise by redefining SPI_start_transaction() as a no-op. Callers that expect to do something in between might be surprised, but available evidence is that no callers do so. Having made that API redefinition, we can fix this mess by having SPI_commit[_and_chain] trap errors and start a new, clean transaction before re-throwing the error. Likewise for SPI_rollback[_and_chain]. Some cleanup is also needed in AtEOXact_SPI, which was nowhere near smart enough to deal with SPI contexts nested inside a committing context. While plperl and pltcl need no changes beyond removing their now-useless SPI_start_transaction() calls, plpython needs some more work because it hadn't gotten the memo about catching commit/rollback errors in the first place. Such an error resulted in longjmp'ing out of the Python interpreter, which leaks Python stack entries at present and is reported to crash Python 3.11 altogether. Add the missing logic to catch such errors and convert them into Python exceptions. We are probably going to have to back-patch this once Python 3.11 ships, but it's a sufficiently basic change that I'm a bit nervous about doing so immediately. Let's let it bake awhile in HEAD first. Peter Eisentraut and Tom Lane Discussion: https://postgr.es/m/3375ffd8-d71c-2565-e348-a597d6e739e3@enterprisedb.com Discussion: https://postgr.es/m/17416-ed8fe5d7213d6c25@postgresql.org Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/2e517818f4af4abe93bf56442469944544f10d4b Modified Files -------------- doc/src/sgml/spi.sgml | 51 +++-- src/backend/executor/spi.c | 221 +++++++++++++++------- src/backend/tcop/postgres.c | 2 - src/backend/utils/mmgr/portalmem.c | 2 +- src/include/executor/spi.h | 1 - src/pl/plperl/expected/plperl_transaction.out | 48 +++++ src/pl/plperl/plperl.c | 2 - src/pl/plperl/sql/plperl_transaction.sql | 32 ++++ src/pl/plpgsql/src/pl_exec.c | 6 - src/pl/plpython/expected/plpython_transaction.out | 67 ++++++- src/pl/plpython/plpy_plpymodule.c | 30 --- src/pl/plpython/plpy_spi.c | 94 +++++++++ src/pl/plpython/plpy_spi.h | 3 + src/pl/plpython/sql/plpython_transaction.sql | 30 +++ src/pl/tcl/expected/pltcl_transaction.out | 49 +++++ src/pl/tcl/pltcl.c | 2 - src/pl/tcl/sql/pltcl_transaction.sql | 37 ++++ 17 files changed, 535 insertions(+), 142 deletions(-) ^ permalink raw reply [nested|flat] 5+ messages in thread
* pgsql: Fix SPI's handling of errors during transaction commit. @ 2022-06-22 16:12 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 5+ messages in thread From: Tom Lane @ 2022-06-22 16:12 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix SPI's handling of errors during transaction commit. SPI_commit previously left it up to the caller to recover from any error occurring during commit. Since that's complicated and requires use of low-level xact.c facilities, it's not too surprising that no caller got it right. Let's move the responsibility for cleanup into spi.c. Doing that requires redefining SPI_commit as starting a new transaction, so that it becomes equivalent to SPI_commit_and_chain except that you get default transaction characteristics instead of preserving the prior transaction's characteristics. We can make this pretty transparent API-wise by redefining SPI_start_transaction() as a no-op. Callers that expect to do something in between might be surprised, but available evidence is that no callers do so. Having made that API redefinition, we can fix this mess by having SPI_commit[_and_chain] trap errors and start a new, clean transaction before re-throwing the error. Likewise for SPI_rollback[_and_chain]. Some cleanup is also needed in AtEOXact_SPI, which was nowhere near smart enough to deal with SPI contexts nested inside a committing context. While plperl and pltcl need no changes beyond removing their now-useless SPI_start_transaction() calls, plpython needs some more work because it hadn't gotten the memo about catching commit/rollback errors in the first place. Such an error resulted in longjmp'ing out of the Python interpreter, which leaks Python stack entries at present and is reported to crash Python 3.11 altogether. Add the missing logic to catch such errors and convert them into Python exceptions. This is a back-patch of commit 2e517818f. That's now aged long enough to reduce the concerns about whether it will break something, and we do need to ensure that supported branches will work with Python 3.11. Peter Eisentraut and Tom Lane Discussion: https://postgr.es/m/3375ffd8-d71c-2565-e348-a597d6e739e3@enterprisedb.com Discussion: https://postgr.es/m/17416-ed8fe5d7213d6c25@postgresql.org Branch ------ REL_13_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/cfc86f987349372dbbfc0391f9f519c0a7b27b84 Modified Files -------------- doc/src/sgml/spi.sgml | 51 +++-- src/backend/executor/spi.c | 221 +++++++++++++++------- src/backend/tcop/postgres.c | 2 - src/backend/utils/mmgr/portalmem.c | 2 +- src/include/executor/spi.h | 1 - src/pl/plperl/expected/plperl_transaction.out | 48 +++++ src/pl/plperl/plperl.c | 2 - src/pl/plperl/sql/plperl_transaction.sql | 32 ++++ src/pl/plpgsql/src/pl_exec.c | 6 - src/pl/plpython/expected/plpython_transaction.out | 67 ++++++- src/pl/plpython/plpy_plpymodule.c | 30 --- src/pl/plpython/plpy_spi.c | 94 +++++++++ src/pl/plpython/plpy_spi.h | 3 + src/pl/plpython/sql/plpython_transaction.sql | 30 +++ src/pl/tcl/expected/pltcl_transaction.out | 49 +++++ src/pl/tcl/pltcl.c | 2 - src/pl/tcl/sql/pltcl_transaction.sql | 37 ++++ 17 files changed, 535 insertions(+), 142 deletions(-) ^ permalink raw reply [nested|flat] 5+ messages in thread
* pgsql: Fix SPI's handling of errors during transaction commit. @ 2022-06-22 16:12 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 5+ messages in thread From: Tom Lane @ 2022-06-22 16:12 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix SPI's handling of errors during transaction commit. SPI_commit previously left it up to the caller to recover from any error occurring during commit. Since that's complicated and requires use of low-level xact.c facilities, it's not too surprising that no caller got it right. Let's move the responsibility for cleanup into spi.c. Doing that requires redefining SPI_commit as starting a new transaction, so that it becomes equivalent to SPI_commit_and_chain except that you get default transaction characteristics instead of preserving the prior transaction's characteristics. We can make this pretty transparent API-wise by redefining SPI_start_transaction() as a no-op. Callers that expect to do something in between might be surprised, but available evidence is that no callers do so. Having made that API redefinition, we can fix this mess by having SPI_commit[_and_chain] trap errors and start a new, clean transaction before re-throwing the error. Likewise for SPI_rollback[_and_chain]. Some cleanup is also needed in AtEOXact_SPI, which was nowhere near smart enough to deal with SPI contexts nested inside a committing context. While plperl and pltcl need no changes beyond removing their now-useless SPI_start_transaction() calls, plpython needs some more work because it hadn't gotten the memo about catching commit/rollback errors in the first place. Such an error resulted in longjmp'ing out of the Python interpreter, which leaks Python stack entries at present and is reported to crash Python 3.11 altogether. Add the missing logic to catch such errors and convert them into Python exceptions. This is a back-patch of commit 2e517818f. That's now aged long enough to reduce the concerns about whether it will break something, and we do need to ensure that supported branches will work with Python 3.11. Peter Eisentraut and Tom Lane Discussion: https://postgr.es/m/3375ffd8-d71c-2565-e348-a597d6e739e3@enterprisedb.com Discussion: https://postgr.es/m/17416-ed8fe5d7213d6c25@postgresql.org Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/604651880c71c5106a72529b9ce29eaad0cfab27 Modified Files -------------- doc/src/sgml/spi.sgml | 51 +++-- src/backend/executor/spi.c | 221 +++++++++++++++------- src/backend/tcop/postgres.c | 2 - src/backend/utils/mmgr/portalmem.c | 2 +- src/include/executor/spi.h | 1 - src/pl/plperl/expected/plperl_transaction.out | 48 +++++ src/pl/plperl/plperl.c | 2 - src/pl/plperl/sql/plperl_transaction.sql | 32 ++++ src/pl/plpgsql/src/pl_exec.c | 6 - src/pl/plpython/expected/plpython_transaction.out | 67 ++++++- src/pl/plpython/plpy_plpymodule.c | 30 --- src/pl/plpython/plpy_spi.c | 94 +++++++++ src/pl/plpython/plpy_spi.h | 3 + src/pl/plpython/sql/plpython_transaction.sql | 30 +++ src/pl/tcl/expected/pltcl_transaction.out | 49 +++++ src/pl/tcl/pltcl.c | 2 - src/pl/tcl/sql/pltcl_transaction.sql | 37 ++++ 17 files changed, 535 insertions(+), 142 deletions(-) ^ permalink raw reply [nested|flat] 5+ messages in thread
* pgsql: Fix SPI's handling of errors during transaction commit. @ 2022-06-22 16:12 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 5+ messages in thread From: Tom Lane @ 2022-06-22 16:12 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix SPI's handling of errors during transaction commit. SPI_commit previously left it up to the caller to recover from any error occurring during commit. Since that's complicated and requires use of low-level xact.c facilities, it's not too surprising that no caller got it right. Let's move the responsibility for cleanup into spi.c. Doing that requires redefining SPI_commit as starting a new transaction, so that it becomes equivalent to SPI_commit_and_chain except that you get default transaction characteristics instead of preserving the prior transaction's characteristics. We can make this pretty transparent API-wise by redefining SPI_start_transaction() as a no-op. Callers that expect to do something in between might be surprised, but available evidence is that no callers do so. Having made that API redefinition, we can fix this mess by having SPI_commit[_and_chain] trap errors and start a new, clean transaction before re-throwing the error. Likewise for SPI_rollback[_and_chain]. Some cleanup is also needed in AtEOXact_SPI, which was nowhere near smart enough to deal with SPI contexts nested inside a committing context. While plperl and pltcl need no changes beyond removing their now-useless SPI_start_transaction() calls, plpython needs some more work because it hadn't gotten the memo about catching commit/rollback errors in the first place. Such an error resulted in longjmp'ing out of the Python interpreter, which leaks Python stack entries at present and is reported to crash Python 3.11 altogether. Add the missing logic to catch such errors and convert them into Python exceptions. This is a back-patch of commit 2e517818f. That's now aged long enough to reduce the concerns about whether it will break something, and we do need to ensure that supported branches will work with Python 3.11. Peter Eisentraut and Tom Lane Discussion: https://postgr.es/m/3375ffd8-d71c-2565-e348-a597d6e739e3@enterprisedb.com Discussion: https://postgr.es/m/17416-ed8fe5d7213d6c25@postgresql.org Branch ------ REL_12_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/293f5c5f496cd8ce87c65b393613da675fc0bb8d Modified Files -------------- doc/src/sgml/spi.sgml | 51 +++-- src/backend/executor/spi.c | 221 +++++++++++++++------- src/backend/tcop/postgres.c | 2 - src/backend/utils/mmgr/portalmem.c | 2 +- src/include/executor/spi.h | 1 - src/pl/plperl/expected/plperl_transaction.out | 48 +++++ src/pl/plperl/plperl.c | 2 - src/pl/plperl/sql/plperl_transaction.sql | 32 ++++ src/pl/plpgsql/src/pl_exec.c | 6 - src/pl/plpython/expected/plpython_transaction.out | 67 ++++++- src/pl/plpython/plpy_plpymodule.c | 30 --- src/pl/plpython/plpy_spi.c | 94 +++++++++ src/pl/plpython/plpy_spi.h | 3 + src/pl/plpython/sql/plpython_transaction.sql | 30 +++ src/pl/tcl/expected/pltcl_transaction.out | 49 +++++ src/pl/tcl/pltcl.c | 2 - src/pl/tcl/sql/pltcl_transaction.sql | 37 ++++ 17 files changed, 535 insertions(+), 142 deletions(-) ^ permalink raw reply [nested|flat] 5+ messages in thread
* pgsql: Fix SPI's handling of errors during transaction commit. @ 2022-06-22 16:12 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 5+ messages in thread From: Tom Lane @ 2022-06-22 16:12 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix SPI's handling of errors during transaction commit. SPI_commit previously left it up to the caller to recover from any error occurring during commit. Since that's complicated and requires use of low-level xact.c facilities, it's not too surprising that no caller got it right. Let's move the responsibility for cleanup into spi.c. Doing that requires redefining SPI_commit as starting a new transaction, so that it becomes equivalent to SPI_commit_and_chain except that you get default transaction characteristics instead of preserving the prior transaction's characteristics. We can make this pretty transparent API-wise by redefining SPI_start_transaction() as a no-op. Callers that expect to do something in between might be surprised, but available evidence is that no callers do so. Having made that API redefinition, we can fix this mess by having SPI_commit[_and_chain] trap errors and start a new, clean transaction before re-throwing the error. Likewise for SPI_rollback[_and_chain]. Some cleanup is also needed in AtEOXact_SPI, which was nowhere near smart enough to deal with SPI contexts nested inside a committing context. While plperl and pltcl need no changes beyond removing their now-useless SPI_start_transaction() calls, plpython needs some more work because it hadn't gotten the memo about catching commit/rollback errors in the first place. Such an error resulted in longjmp'ing out of the Python interpreter, which leaks Python stack entries at present and is reported to crash Python 3.11 altogether. Add the missing logic to catch such errors and convert them into Python exceptions. This is a back-patch of commit 2e517818f. That's now aged long enough to reduce the concerns about whether it will break something, and we do need to ensure that supported branches will work with Python 3.11. Peter Eisentraut and Tom Lane Discussion: https://postgr.es/m/3375ffd8-d71c-2565-e348-a597d6e739e3@enterprisedb.com Discussion: https://postgr.es/m/17416-ed8fe5d7213d6c25@postgresql.org Branch ------ REL_11_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/2f6b8c287b881fd33168d8c15c56f71163ff3ba6 Modified Files -------------- doc/src/sgml/spi.sgml | 43 ++--- src/backend/executor/spi.c | 204 ++++++++++++++++------ src/backend/tcop/postgres.c | 2 - src/backend/utils/mmgr/portalmem.c | 2 +- src/include/executor/spi.h | 1 - src/pl/plperl/expected/plperl_transaction.out | 48 +++++ src/pl/plperl/plperl.c | 2 - src/pl/plperl/sql/plperl_transaction.sql | 32 ++++ src/pl/plpgsql/src/pl_exec.c | 2 - src/pl/plpython/expected/plpython_transaction.out | 67 ++++++- src/pl/plpython/plpy_plpymodule.c | 30 ---- src/pl/plpython/plpy_spi.c | 94 ++++++++++ src/pl/plpython/plpy_spi.h | 3 + src/pl/plpython/sql/plpython_transaction.sql | 30 ++++ src/pl/tcl/expected/pltcl_transaction.out | 49 ++++++ src/pl/tcl/pltcl.c | 2 - src/pl/tcl/sql/pltcl_transaction.sql | 37 ++++ 17 files changed, 524 insertions(+), 124 deletions(-) ^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2022-06-22 16:12 UTC | newest] Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2022-02-28 17:45 pgsql: Fix SPI's handling of errors during transaction commit. Tom Lane <tgl@sss.pgh.pa.us> 2022-06-22 16:12 pgsql: Fix SPI's handling of errors during transaction commit. Tom Lane <tgl@sss.pgh.pa.us> 2022-06-22 16:12 pgsql: Fix SPI's handling of errors during transaction commit. Tom Lane <tgl@sss.pgh.pa.us> 2022-06-22 16:12 pgsql: Fix SPI's handling of errors during transaction commit. Tom Lane <tgl@sss.pgh.pa.us> 2022-06-22 16:12 pgsql: Fix SPI's handling of errors during transaction commit. Tom Lane <tgl@sss.pgh.pa.us>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox