pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: tests: BackgroundPsql: Fix potential for lost errors on windows 6+ messages / 1 participants [nested] [flat]
* pgsql: tests: BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-19 17:45 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 6+ messages in thread From: Andres Freund @ 2025-02-19 17:45 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org tests: BackgroundPsql: Fix potential for lost errors on windows This addresses various corner cases in BackgroundPsql: - On windows stdout and stderr may arrive out of order, leading to errors not being reported, or attributed to the wrong statement. To fix, emit the "query-separation banner" on both stdout and stderr and wait for both. - Very occasionally the "query-separation banner" would not get removed, because we waited until the banner arrived, but then replaced the banner plus newline. To fix, wait for banner and newline. - For interactive psql replacing $banner\n is not sufficient, interactive psql outputs \r\n. - For interactive psql, where commands are echoed to stdout, the \echo command, rather than its output, would be matched. This would sometimes lead to output from the prior query, or wait_connect(), being returned in the next command. This also affected wait_connect(), leading to sometimes sending queries to psql before the connection actually was established. While debugging these issues I also found that it's hard to know whether a query separation banner was attributed to the right query. Make that easier by counting the queries each BackgroundPsql instance has emitted and include the number in the banner. Also emit psql stdout/stderr in query() and wait_connect() as Test::More notes, without that it's rather hard to debug some issues in CI and buildfarm. As this can cause issues not just to-be-added tests, but also existing ones, backpatch the fix to all supported versions. Reviewed-by: Daniel Gustafsson <daniel@yesql.se> Reviewed-by: Noah Misch <noah@leadboat.com> Discussion: https://postgr.es/m/wmovm6xcbwh7twdtymxuboaoarbvwj2haasd3sikzlb3dkgz76@n45rzycluzft Backpatch-through: 13 Branch ------ REL_15_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/70b650d1855331ebfc54c172092a2a1575911060 Modified Files -------------- src/test/perl/PostgreSQL/Test/BackgroundPsql.pm | 75 ++++++++++++++++++++----- 1 file changed, 60 insertions(+), 15 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: tests: BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-19 17:45 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 6+ messages in thread From: Andres Freund @ 2025-02-19 17:45 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org tests: BackgroundPsql: Fix potential for lost errors on windows This addresses various corner cases in BackgroundPsql: - On windows stdout and stderr may arrive out of order, leading to errors not being reported, or attributed to the wrong statement. To fix, emit the "query-separation banner" on both stdout and stderr and wait for both. - Very occasionally the "query-separation banner" would not get removed, because we waited until the banner arrived, but then replaced the banner plus newline. To fix, wait for banner and newline. - For interactive psql replacing $banner\n is not sufficient, interactive psql outputs \r\n. - For interactive psql, where commands are echoed to stdout, the \echo command, rather than its output, would be matched. This would sometimes lead to output from the prior query, or wait_connect(), being returned in the next command. This also affected wait_connect(), leading to sometimes sending queries to psql before the connection actually was established. While debugging these issues I also found that it's hard to know whether a query separation banner was attributed to the right query. Make that easier by counting the queries each BackgroundPsql instance has emitted and include the number in the banner. Also emit psql stdout/stderr in query() and wait_connect() as Test::More notes, without that it's rather hard to debug some issues in CI and buildfarm. As this can cause issues not just to-be-added tests, but also existing ones, backpatch the fix to all supported versions. Reviewed-by: Daniel Gustafsson <daniel@yesql.se> Reviewed-by: Noah Misch <noah@leadboat.com> Discussion: https://postgr.es/m/wmovm6xcbwh7twdtymxuboaoarbvwj2haasd3sikzlb3dkgz76@n45rzycluzft Backpatch-through: 13 Branch ------ REL_13_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/bb2bf22761e1d1c7bec40b3ae244ebdb10e23ed9 Modified Files -------------- src/test/perl/PostgreSQL/Test/BackgroundPsql.pm | 75 ++++++++++++++++++++----- 1 file changed, 60 insertions(+), 15 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: tests: BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-19 17:45 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 6+ messages in thread From: Andres Freund @ 2025-02-19 17:45 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org tests: BackgroundPsql: Fix potential for lost errors on windows This addresses various corner cases in BackgroundPsql: - On windows stdout and stderr may arrive out of order, leading to errors not being reported, or attributed to the wrong statement. To fix, emit the "query-separation banner" on both stdout and stderr and wait for both. - Very occasionally the "query-separation banner" would not get removed, because we waited until the banner arrived, but then replaced the banner plus newline. To fix, wait for banner and newline. - For interactive psql replacing $banner\n is not sufficient, interactive psql outputs \r\n. - For interactive psql, where commands are echoed to stdout, the \echo command, rather than its output, would be matched. This would sometimes lead to output from the prior query, or wait_connect(), being returned in the next command. This also affected wait_connect(), leading to sometimes sending queries to psql before the connection actually was established. While debugging these issues I also found that it's hard to know whether a query separation banner was attributed to the right query. Make that easier by counting the queries each BackgroundPsql instance has emitted and include the number in the banner. Also emit psql stdout/stderr in query() and wait_connect() as Test::More notes, without that it's rather hard to debug some issues in CI and buildfarm. As this can cause issues not just to-be-added tests, but also existing ones, backpatch the fix to all supported versions. Reviewed-by: Daniel Gustafsson <daniel@yesql.se> Reviewed-by: Noah Misch <noah@leadboat.com> Discussion: https://postgr.es/m/wmovm6xcbwh7twdtymxuboaoarbvwj2haasd3sikzlb3dkgz76@n45rzycluzft Backpatch-through: 13 Branch ------ REL_17_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/f61769adaa6214a01d087ab71ece8d170cda072f Modified Files -------------- src/test/perl/PostgreSQL/Test/BackgroundPsql.pm | 75 ++++++++++++++++++++----- 1 file changed, 60 insertions(+), 15 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: tests: BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-19 17:45 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 6+ messages in thread From: Andres Freund @ 2025-02-19 17:45 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org tests: BackgroundPsql: Fix potential for lost errors on windows This addresses various corner cases in BackgroundPsql: - On windows stdout and stderr may arrive out of order, leading to errors not being reported, or attributed to the wrong statement. To fix, emit the "query-separation banner" on both stdout and stderr and wait for both. - Very occasionally the "query-separation banner" would not get removed, because we waited until the banner arrived, but then replaced the banner plus newline. To fix, wait for banner and newline. - For interactive psql replacing $banner\n is not sufficient, interactive psql outputs \r\n. - For interactive psql, where commands are echoed to stdout, the \echo command, rather than its output, would be matched. This would sometimes lead to output from the prior query, or wait_connect(), being returned in the next command. This also affected wait_connect(), leading to sometimes sending queries to psql before the connection actually was established. While debugging these issues I also found that it's hard to know whether a query separation banner was attributed to the right query. Make that easier by counting the queries each BackgroundPsql instance has emitted and include the number in the banner. Also emit psql stdout/stderr in query() and wait_connect() as Test::More notes, without that it's rather hard to debug some issues in CI and buildfarm. As this can cause issues not just to-be-added tests, but also existing ones, backpatch the fix to all supported versions. Reviewed-by: Daniel Gustafsson <daniel@yesql.se> Reviewed-by: Noah Misch <noah@leadboat.com> Discussion: https://postgr.es/m/wmovm6xcbwh7twdtymxuboaoarbvwj2haasd3sikzlb3dkgz76@n45rzycluzft Backpatch-through: 13 Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/c21ebcca7965f29105276682ba236c2868f53d12 Modified Files -------------- src/test/perl/PostgreSQL/Test/BackgroundPsql.pm | 75 ++++++++++++++++++++----- 1 file changed, 60 insertions(+), 15 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: tests: BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-19 17:45 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 6+ messages in thread From: Andres Freund @ 2025-02-19 17:45 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org tests: BackgroundPsql: Fix potential for lost errors on windows This addresses various corner cases in BackgroundPsql: - On windows stdout and stderr may arrive out of order, leading to errors not being reported, or attributed to the wrong statement. To fix, emit the "query-separation banner" on both stdout and stderr and wait for both. - Very occasionally the "query-separation banner" would not get removed, because we waited until the banner arrived, but then replaced the banner plus newline. To fix, wait for banner and newline. - For interactive psql replacing $banner\n is not sufficient, interactive psql outputs \r\n. - For interactive psql, where commands are echoed to stdout, the \echo command, rather than its output, would be matched. This would sometimes lead to output from the prior query, or wait_connect(), being returned in the next command. This also affected wait_connect(), leading to sometimes sending queries to psql before the connection actually was established. While debugging these issues I also found that it's hard to know whether a query separation banner was attributed to the right query. Make that easier by counting the queries each BackgroundPsql instance has emitted and include the number in the banner. Also emit psql stdout/stderr in query() and wait_connect() as Test::More notes, without that it's rather hard to debug some issues in CI and buildfarm. As this can cause issues not just to-be-added tests, but also existing ones, backpatch the fix to all supported versions. Reviewed-by: Daniel Gustafsson <daniel@yesql.se> Reviewed-by: Noah Misch <noah@leadboat.com> Discussion: https://postgr.es/m/wmovm6xcbwh7twdtymxuboaoarbvwj2haasd3sikzlb3dkgz76@n45rzycluzft Backpatch-through: 13 Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/8b886a4e34886bf2d1dd4f5e7d95f4ddadc3afa8 Modified Files -------------- src/test/perl/PostgreSQL/Test/BackgroundPsql.pm | 75 ++++++++++++++++++++----- 1 file changed, 60 insertions(+), 15 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: tests: BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-19 17:45 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 6+ messages in thread From: Andres Freund @ 2025-02-19 17:45 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org tests: BackgroundPsql: Fix potential for lost errors on windows This addresses various corner cases in BackgroundPsql: - On windows stdout and stderr may arrive out of order, leading to errors not being reported, or attributed to the wrong statement. To fix, emit the "query-separation banner" on both stdout and stderr and wait for both. - Very occasionally the "query-separation banner" would not get removed, because we waited until the banner arrived, but then replaced the banner plus newline. To fix, wait for banner and newline. - For interactive psql replacing $banner\n is not sufficient, interactive psql outputs \r\n. - For interactive psql, where commands are echoed to stdout, the \echo command, rather than its output, would be matched. This would sometimes lead to output from the prior query, or wait_connect(), being returned in the next command. This also affected wait_connect(), leading to sometimes sending queries to psql before the connection actually was established. While debugging these issues I also found that it's hard to know whether a query separation banner was attributed to the right query. Make that easier by counting the queries each BackgroundPsql instance has emitted and include the number in the banner. Also emit psql stdout/stderr in query() and wait_connect() as Test::More notes, without that it's rather hard to debug some issues in CI and buildfarm. As this can cause issues not just to-be-added tests, but also existing ones, backpatch the fix to all supported versions. Reviewed-by: Daniel Gustafsson <daniel@yesql.se> Reviewed-by: Noah Misch <noah@leadboat.com> Discussion: https://postgr.es/m/wmovm6xcbwh7twdtymxuboaoarbvwj2haasd3sikzlb3dkgz76@n45rzycluzft Backpatch-through: 13 Branch ------ REL_16_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/4c9397c374d9cf567837682fe2672df421209921 Modified Files -------------- src/test/perl/PostgreSQL/Test/BackgroundPsql.pm | 75 ++++++++++++++++++++----- 1 file changed, 60 insertions(+), 15 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2025-02-19 17:45 UTC | newest] Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2025-02-19 17:45 pgsql: tests: BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-19 17:45 pgsql: tests: BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-19 17:45 pgsql: tests: BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-19 17:45 pgsql: tests: BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-19 17:45 pgsql: tests: BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-19 17:45 pgsql: tests: BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox