pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: Handle better implicit transaction state of pipeline mode 6+ messages / 1 participants [nested] [flat]
* pgsql: Handle better implicit transaction state of pipeline mode @ 2024-11-27 00:34 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 6+ messages in thread From: Michael Paquier @ 2024-11-27 00:34 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Handle better implicit transaction state of pipeline mode When using a pipeline, a transaction starts from the first command and is committed with a Sync message or when the pipeline ends. Functions like IsInTransactionBlock() or PreventInTransactionBlock() were already able to understand a pipeline as being in a transaction block, but it was not the case of CheckTransactionBlock(). This function is called for example to generate a WARNING for SET LOCAL, complaining that it is used outside of a transaction block. The current state of the code caused multiple problems, like: - SET LOCAL executed at any stage of a pipeline issued a WARNING, even if the command was at least second in line where the pipeline is in a transaction state. - LOCK TABLE failed when invoked at any step of a pipeline, even if it should be able to work within a transaction block. The pipeline protocol assumes that the first command of a pipeline is not part of a transaction block, and that any follow-up commands is considered as within a transaction block. This commit changes the backend so as an implicit transaction block is started each time the first Execute message of a pipeline has finished processing, with this implicit transaction block ended once a sync is processed. The checks based on XACT_FLAGS_PIPELINING in the routines checking if we are in a transaction block are not necessary: it is enough to rely on the existing ones. Some tests are added to pgbench, that can be backpatched down to v17 when \syncpipeline is involved and down to v14 where \startpipeline and \endpipeline are available. This is unfortunately limited regarding the error patterns that can be checked, but it provides coverage for various pipeline combinations to check if these succeed or fail. These tests are able to capture the case of SET LOCAL's WARNING. The author has proposed a different feature to improve the coverage by adding similar meta-commands to psql where error messages could be checked, something more useful for the cases where commands cannot be used in transaction blocks, like REINDEX CONCURRENTLY or VACUUM. This is considered as future work for v18~. Author: Anthonin Bonnefoy Reviewed-by: Jelte Fennema-Nio, Michael Paquier Discussion: https://postgr.es/m/CAO6_XqrWO8uNBQrSu5r6jh+vTGi5Oiyk4y8yXDORdE2jbzw8xw@mail.gmail.com Backpatch-through: 13 Branch ------ REL_13_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/5c3067282ec12f1e89c5f2ae8a3a5968520716e8 Modified Files -------------- doc/src/sgml/protocol.sgml | 21 +++++++++++---------- src/backend/access/transam/xact.c | 13 ------------- src/backend/tcop/postgres.c | 18 ++++++++++++++++++ 3 files changed, 29 insertions(+), 23 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Handle better implicit transaction state of pipeline mode @ 2024-11-27 00:34 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 6+ messages in thread From: Michael Paquier @ 2024-11-27 00:34 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Handle better implicit transaction state of pipeline mode When using a pipeline, a transaction starts from the first command and is committed with a Sync message or when the pipeline ends. Functions like IsInTransactionBlock() or PreventInTransactionBlock() were already able to understand a pipeline as being in a transaction block, but it was not the case of CheckTransactionBlock(). This function is called for example to generate a WARNING for SET LOCAL, complaining that it is used outside of a transaction block. The current state of the code caused multiple problems, like: - SET LOCAL executed at any stage of a pipeline issued a WARNING, even if the command was at least second in line where the pipeline is in a transaction state. - LOCK TABLE failed when invoked at any step of a pipeline, even if it should be able to work within a transaction block. The pipeline protocol assumes that the first command of a pipeline is not part of a transaction block, and that any follow-up commands is considered as within a transaction block. This commit changes the backend so as an implicit transaction block is started each time the first Execute message of a pipeline has finished processing, with this implicit transaction block ended once a sync is processed. The checks based on XACT_FLAGS_PIPELINING in the routines checking if we are in a transaction block are not necessary: it is enough to rely on the existing ones. Some tests are added to pgbench, that can be backpatched down to v17 when \syncpipeline is involved and down to v14 where \startpipeline and \endpipeline are available. This is unfortunately limited regarding the error patterns that can be checked, but it provides coverage for various pipeline combinations to check if these succeed or fail. These tests are able to capture the case of SET LOCAL's WARNING. The author has proposed a different feature to improve the coverage by adding similar meta-commands to psql where error messages could be checked, something more useful for the cases where commands cannot be used in transaction blocks, like REINDEX CONCURRENTLY or VACUUM. This is considered as future work for v18~. Author: Anthonin Bonnefoy Reviewed-by: Jelte Fennema-Nio, Michael Paquier Discussion: https://postgr.es/m/CAO6_XqrWO8uNBQrSu5r6jh+vTGi5Oiyk4y8yXDORdE2jbzw8xw@mail.gmail.com Backpatch-through: 13 Branch ------ REL_17_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/d77f91214fb70879f7013dfd10a7bcac2a8159cc Modified Files -------------- doc/src/sgml/protocol.sgml | 21 ++-- src/backend/access/transam/xact.c | 13 -- src/backend/tcop/postgres.c | 18 +++ src/bin/pgbench/t/001_pgbench_with_server.pl | 174 +++++++++++++++++++++++++++ 4 files changed, 203 insertions(+), 23 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Handle better implicit transaction state of pipeline mode @ 2024-11-27 00:34 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 6+ messages in thread From: Michael Paquier @ 2024-11-27 00:34 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Handle better implicit transaction state of pipeline mode When using a pipeline, a transaction starts from the first command and is committed with a Sync message or when the pipeline ends. Functions like IsInTransactionBlock() or PreventInTransactionBlock() were already able to understand a pipeline as being in a transaction block, but it was not the case of CheckTransactionBlock(). This function is called for example to generate a WARNING for SET LOCAL, complaining that it is used outside of a transaction block. The current state of the code caused multiple problems, like: - SET LOCAL executed at any stage of a pipeline issued a WARNING, even if the command was at least second in line where the pipeline is in a transaction state. - LOCK TABLE failed when invoked at any step of a pipeline, even if it should be able to work within a transaction block. The pipeline protocol assumes that the first command of a pipeline is not part of a transaction block, and that any follow-up commands is considered as within a transaction block. This commit changes the backend so as an implicit transaction block is started each time the first Execute message of a pipeline has finished processing, with this implicit transaction block ended once a sync is processed. The checks based on XACT_FLAGS_PIPELINING in the routines checking if we are in a transaction block are not necessary: it is enough to rely on the existing ones. Some tests are added to pgbench, that can be backpatched down to v17 when \syncpipeline is involved and down to v14 where \startpipeline and \endpipeline are available. This is unfortunately limited regarding the error patterns that can be checked, but it provides coverage for various pipeline combinations to check if these succeed or fail. These tests are able to capture the case of SET LOCAL's WARNING. The author has proposed a different feature to improve the coverage by adding similar meta-commands to psql where error messages could be checked, something more useful for the cases where commands cannot be used in transaction blocks, like REINDEX CONCURRENTLY or VACUUM. This is considered as future work for v18~. Author: Anthonin Bonnefoy Reviewed-by: Jelte Fennema-Nio, Michael Paquier Discussion: https://postgr.es/m/CAO6_XqrWO8uNBQrSu5r6jh+vTGi5Oiyk4y8yXDORdE2jbzw8xw@mail.gmail.com Backpatch-through: 13 Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/2366211949a0122ccadd56a34085478ebcfb2f5e Modified Files -------------- doc/src/sgml/protocol.sgml | 21 ++-- src/backend/access/transam/xact.c | 13 --- src/backend/tcop/postgres.c | 18 ++++ src/bin/pgbench/t/001_pgbench_with_server.pl | 155 +++++++++++++++++++++++++++ 4 files changed, 184 insertions(+), 23 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Handle better implicit transaction state of pipeline mode @ 2024-11-27 00:34 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 6+ messages in thread From: Michael Paquier @ 2024-11-27 00:34 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Handle better implicit transaction state of pipeline mode When using a pipeline, a transaction starts from the first command and is committed with a Sync message or when the pipeline ends. Functions like IsInTransactionBlock() or PreventInTransactionBlock() were already able to understand a pipeline as being in a transaction block, but it was not the case of CheckTransactionBlock(). This function is called for example to generate a WARNING for SET LOCAL, complaining that it is used outside of a transaction block. The current state of the code caused multiple problems, like: - SET LOCAL executed at any stage of a pipeline issued a WARNING, even if the command was at least second in line where the pipeline is in a transaction state. - LOCK TABLE failed when invoked at any step of a pipeline, even if it should be able to work within a transaction block. The pipeline protocol assumes that the first command of a pipeline is not part of a transaction block, and that any follow-up commands is considered as within a transaction block. This commit changes the backend so as an implicit transaction block is started each time the first Execute message of a pipeline has finished processing, with this implicit transaction block ended once a sync is processed. The checks based on XACT_FLAGS_PIPELINING in the routines checking if we are in a transaction block are not necessary: it is enough to rely on the existing ones. Some tests are added to pgbench, that can be backpatched down to v17 when \syncpipeline is involved and down to v14 where \startpipeline and \endpipeline are available. This is unfortunately limited regarding the error patterns that can be checked, but it provides coverage for various pipeline combinations to check if these succeed or fail. These tests are able to capture the case of SET LOCAL's WARNING. The author has proposed a different feature to improve the coverage by adding similar meta-commands to psql where error messages could be checked, something more useful for the cases where commands cannot be used in transaction blocks, like REINDEX CONCURRENTLY or VACUUM. This is considered as future work for v18~. Author: Anthonin Bonnefoy Reviewed-by: Jelte Fennema-Nio, Michael Paquier Discussion: https://postgr.es/m/CAO6_XqrWO8uNBQrSu5r6jh+vTGi5Oiyk4y8yXDORdE2jbzw8xw@mail.gmail.com Backpatch-through: 13 Branch ------ REL_15_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/88a16b3dbb16dc127894ad56c57c53d309ae816a Modified Files -------------- doc/src/sgml/protocol.sgml | 21 ++-- src/backend/access/transam/xact.c | 13 --- src/backend/tcop/postgres.c | 18 ++++ src/bin/pgbench/t/001_pgbench_with_server.pl | 155 +++++++++++++++++++++++++++ 4 files changed, 184 insertions(+), 23 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Handle better implicit transaction state of pipeline mode @ 2024-11-27 00:34 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 6+ messages in thread From: Michael Paquier @ 2024-11-27 00:34 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Handle better implicit transaction state of pipeline mode When using a pipeline, a transaction starts from the first command and is committed with a Sync message or when the pipeline ends. Functions like IsInTransactionBlock() or PreventInTransactionBlock() were already able to understand a pipeline as being in a transaction block, but it was not the case of CheckTransactionBlock(). This function is called for example to generate a WARNING for SET LOCAL, complaining that it is used outside of a transaction block. The current state of the code caused multiple problems, like: - SET LOCAL executed at any stage of a pipeline issued a WARNING, even if the command was at least second in line where the pipeline is in a transaction state. - LOCK TABLE failed when invoked at any step of a pipeline, even if it should be able to work within a transaction block. The pipeline protocol assumes that the first command of a pipeline is not part of a transaction block, and that any follow-up commands is considered as within a transaction block. This commit changes the backend so as an implicit transaction block is started each time the first Execute message of a pipeline has finished processing, with this implicit transaction block ended once a sync is processed. The checks based on XACT_FLAGS_PIPELINING in the routines checking if we are in a transaction block are not necessary: it is enough to rely on the existing ones. Some tests are added to pgbench, that can be backpatched down to v17 when \syncpipeline is involved and down to v14 where \startpipeline and \endpipeline are available. This is unfortunately limited regarding the error patterns that can be checked, but it provides coverage for various pipeline combinations to check if these succeed or fail. These tests are able to capture the case of SET LOCAL's WARNING. The author has proposed a different feature to improve the coverage by adding similar meta-commands to psql where error messages could be checked, something more useful for the cases where commands cannot be used in transaction blocks, like REINDEX CONCURRENTLY or VACUUM. This is considered as future work for v18~. Author: Anthonin Bonnefoy Reviewed-by: Jelte Fennema-Nio, Michael Paquier Discussion: https://postgr.es/m/CAO6_XqrWO8uNBQrSu5r6jh+vTGi5Oiyk4y8yXDORdE2jbzw8xw@mail.gmail.com Backpatch-through: 13 Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/d0eb4297cc5bbfb02b64fdaaf5510415e92e5312 Modified Files -------------- doc/src/sgml/protocol.sgml | 21 ++-- src/backend/access/transam/xact.c | 13 -- src/backend/tcop/postgres.c | 18 +++ src/bin/pgbench/t/001_pgbench_with_server.pl | 174 +++++++++++++++++++++++++++ 4 files changed, 203 insertions(+), 23 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Handle better implicit transaction state of pipeline mode @ 2024-11-27 00:34 Michael Paquier <michael@paquier.xyz> 0 siblings, 0 replies; 6+ messages in thread From: Michael Paquier @ 2024-11-27 00:34 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Handle better implicit transaction state of pipeline mode When using a pipeline, a transaction starts from the first command and is committed with a Sync message or when the pipeline ends. Functions like IsInTransactionBlock() or PreventInTransactionBlock() were already able to understand a pipeline as being in a transaction block, but it was not the case of CheckTransactionBlock(). This function is called for example to generate a WARNING for SET LOCAL, complaining that it is used outside of a transaction block. The current state of the code caused multiple problems, like: - SET LOCAL executed at any stage of a pipeline issued a WARNING, even if the command was at least second in line where the pipeline is in a transaction state. - LOCK TABLE failed when invoked at any step of a pipeline, even if it should be able to work within a transaction block. The pipeline protocol assumes that the first command of a pipeline is not part of a transaction block, and that any follow-up commands is considered as within a transaction block. This commit changes the backend so as an implicit transaction block is started each time the first Execute message of a pipeline has finished processing, with this implicit transaction block ended once a sync is processed. The checks based on XACT_FLAGS_PIPELINING in the routines checking if we are in a transaction block are not necessary: it is enough to rely on the existing ones. Some tests are added to pgbench, that can be backpatched down to v17 when \syncpipeline is involved and down to v14 where \startpipeline and \endpipeline are available. This is unfortunately limited regarding the error patterns that can be checked, but it provides coverage for various pipeline combinations to check if these succeed or fail. These tests are able to capture the case of SET LOCAL's WARNING. The author has proposed a different feature to improve the coverage by adding similar meta-commands to psql where error messages could be checked, something more useful for the cases where commands cannot be used in transaction blocks, like REINDEX CONCURRENTLY or VACUUM. This is considered as future work for v18~. Author: Anthonin Bonnefoy Reviewed-by: Jelte Fennema-Nio, Michael Paquier Discussion: https://postgr.es/m/CAO6_XqrWO8uNBQrSu5r6jh+vTGi5Oiyk4y8yXDORdE2jbzw8xw@mail.gmail.com Backpatch-through: 13 Branch ------ REL_16_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/a8abd36e88f50531177d5b52026194af52d45421 Modified Files -------------- doc/src/sgml/protocol.sgml | 21 ++-- src/backend/access/transam/xact.c | 13 --- src/backend/tcop/postgres.c | 18 ++++ src/bin/pgbench/t/001_pgbench_with_server.pl | 155 +++++++++++++++++++++++++++ 4 files changed, 184 insertions(+), 23 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2024-11-27 00:34 UTC | newest] Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2024-11-27 00:34 pgsql: Handle better implicit transaction state of pipeline mode Michael Paquier <michael@paquier.xyz> 2024-11-27 00:34 pgsql: Handle better implicit transaction state of pipeline mode Michael Paquier <michael@paquier.xyz> 2024-11-27 00:34 pgsql: Handle better implicit transaction state of pipeline mode Michael Paquier <michael@paquier.xyz> 2024-11-27 00:34 pgsql: Handle better implicit transaction state of pipeline mode Michael Paquier <michael@paquier.xyz> 2024-11-27 00:34 pgsql: Handle better implicit transaction state of pipeline mode Michael Paquier <michael@paquier.xyz> 2024-11-27 00:34 pgsql: Handle better implicit transaction state of pipeline mode Michael Paquier <michael@paquier.xyz>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox