agora inbox for pgsql-hackers@postgresql.orghelp / color / mirror / Atom feed
[PATCH v1] BackgroundPsql: Fix potential for lost errors on windows 477+ messages / 2 participants [nested] [flat]
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows @ 2025-02-13 15:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 477+ messages in thread From: Andres Freund @ 2025-02-13 15:09 UTC (permalink / raw) Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch: --- .../perl/PostgreSQL/Test/BackgroundPsql.pm | 55 +++++++++++++++---- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm index dbfd82e4fac..7e76845307d 100644 --- a/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm +++ b/src/test/perl/PostgreSQL/Test/BackgroundPsql.pm @@ -89,7 +89,8 @@ sub new 'stdin' => '', 'stdout' => '', 'stderr' => '', - 'query_timer_restart' => undef + 'query_timer_restart' => undef, + 'query_cnt' => 1, }; my $run; @@ -219,27 +220,57 @@ sub query my ($self, $query) = @_; my $ret; my $output; + my $query_cnt = $self->{query_cnt}++; + local $Test::Builder::Level = $Test::Builder::Level + 1; - note "issuing query via background psql: $query"; + note "issuing query $query_cnt via background psql: $query"; $self->{timeout}->start() if (defined($self->{query_timer_restart})); # Feed the query to psql's stdin, followed by \n (so psql processes the # line), by a ; (so that psql issues the query, if it doesn't include a ; - # itself), and a separator echoed with \echo, that we can wait on. - my $banner = "background_psql: QUERY_SEPARATOR"; - $self->{stdin} .= "$query\n;\n\\echo $banner\n"; - - pump_until($self->{run}, $self->{timeout}, \$self->{stdout}, qr/$banner/); + # itself), and a separator echoed both with \echo and \warn, that we can + # wait on. + # + # To avoid somehow confusing the separator from separately issued queries, + # and to make it easier to debug, we include a per-psql query counter in + # the separator. + # + # We need both \echo (printing to stdout) and \warn (printing to stderr), + # because on windows we can get data on stdout before seeing data on + # stderr (or vice versa), even if psql printed them in the opposite + # order. We therefore wait on both. + # + # We need to match for the newline, because we try to remove it below, and + # it's possible to consume just the input *without* the newline. In + # interactive psql we emit \r\n, so we need to allow for that. Also need + # to be careful that we don't e.g. match the echoed \echo command, rather + # than its output. + my $banner = "background_psql: QUERY_SEPARATOR $query_cnt"; + my $banner_match = qr/(^|\n)$banner\r?\n/; + $self->{stdin} .= "$query\n;\n\\echo $banner\n\\warn $banner\n"; + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stdout}, qr/$banner_match/); + pump_until( + $self->{run}, $self->{timeout}, + \$self->{stderr}, qr/$banner_match/); die "psql query timed out" if $self->{timeout}->is_expired; + + note "query returned:\n", + explain { + stdout => $self->{stdout}, + stderr => $self->{stderr}, + }; + + # Remove banner from stdout and stderr, our caller doesn't care. The + # first newline is optional, as there would not be one if consuming an + # empty query result. $output = $self->{stdout}; - - # Remove banner again, our caller doesn't care. The first newline is - # optional, as there would not be one if consuming an empty query - # result. - $output =~ s/\n?$banner\n$//s; + $output =~ s/$banner_match//; + $self->{stderr} =~ s/$banner_match//; # clear out output for the next query $self->{stdout} = ''; -- 2.48.1.76.g4e746b1a31.dirty --sno7qfndt3qie53q-- ^ permalink raw reply [nested|flat] 477+ messages in thread
* [PATCH v51 01/10] Make index_concurrently_create_copy more general @ 2026-03-24 18:02 Álvaro Herrera <alvherre@kurilemu.de> 0 siblings, 0 replies; 477+ messages in thread From: Álvaro Herrera @ 2026-03-24 18:02 UTC (permalink / raw) Add a 'boolean concurrent' option, and make it work for both cases. Also rename it to index_create_copy. This allows it to be reused for other purposes -- specifically, for REPACK CONCURRENTLY. With the CONCURRENTLY option, REPACK cannot simply swap the heap file and rebuild its indexes. Instead, it needs to build a separate set of indexes (including system catalog entries) *before* the actual swap, to reduce the time AccessExclusiveLock needs to be held for. This approach is different from what CREATE INDEX CONCURRENTLY does. Per a suggestion from Mihail Nikalayeu. Author: Antonin Houska <ah@cybertec.at> Discussion: https://postgr.es/m/41104.1754922120@localhost --- src/backend/catalog/index.c | 41 ++++++++++++++++++++++---------- src/backend/commands/indexcmds.c | 9 +++---- src/include/catalog/index.h | 7 +++--- 3 files changed, 36 insertions(+), 21 deletions(-) diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index 1ccfa687f05..e418d67e8e4 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -1289,17 +1289,17 @@ index_create(Relation heapRelation, } /* - * index_concurrently_create_copy + * index_create_copy * - * Create concurrently an index based on the definition of the one provided by - * caller. The index is inserted into catalogs and needs to be built later - * on. This is called during concurrent reindex processing. + * Create an index based on the definition of the one provided by caller. The + * index is inserted into catalogs. If 'concurrently' is TRUE, it needs to be + * built later on; otherwise it's built immediately. * * "tablespaceOid" is the tablespace to use for this index. */ Oid -index_concurrently_create_copy(Relation heapRelation, Oid oldIndexId, - Oid tablespaceOid, const char *newName) +index_create_copy(Relation heapRelation, bool concurrently, + Oid oldIndexId, Oid tablespaceOid, const char *newName) { Relation indexRelation; IndexInfo *oldInfo, @@ -1318,6 +1318,7 @@ index_concurrently_create_copy(Relation heapRelation, Oid oldIndexId, List *indexColNames = NIL; List *indexExprs = NIL; List *indexPreds = NIL; + int flags = 0; indexRelation = index_open(oldIndexId, RowExclusiveLock); @@ -1328,7 +1329,7 @@ index_concurrently_create_copy(Relation heapRelation, Oid oldIndexId, * Concurrent build of an index with exclusion constraints is not * supported. */ - if (oldInfo->ii_ExclusionOps != NULL) + if (oldInfo->ii_ExclusionOps != NULL && concurrently) ereport(ERROR, (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("concurrent index creation for exclusion constraints is not supported"))); @@ -1384,9 +1385,7 @@ index_concurrently_create_copy(Relation heapRelation, Oid oldIndexId, } /* - * Build the index information for the new index. Note that rebuild of - * indexes with exclusion constraints is not supported, hence there is no - * need to fill all the ii_Exclusion* fields. + * Build the index information for the new index. */ newInfo = makeIndexInfo(oldInfo->ii_NumIndexAttrs, oldInfo->ii_NumIndexKeyAttrs, @@ -1395,11 +1394,24 @@ index_concurrently_create_copy(Relation heapRelation, Oid oldIndexId, indexPreds, oldInfo->ii_Unique, oldInfo->ii_NullsNotDistinct, - false, /* not ready for inserts */ - true, + !concurrently, /* isready */ + concurrently, /* concurrent */ indexRelation->rd_indam->amsummarizing, oldInfo->ii_WithoutOverlaps); + /* fetch exclusion constraint info if any */ + if (indexRelation->rd_index->indisexclusion) + { + /* + * XXX Beware: we're making newInfo point to oldInfo-owned memory. It + * would be more orthodox to palloc+memcpy, but we don't need that + * here at present. + */ + newInfo->ii_ExclusionOps = oldInfo->ii_ExclusionOps; + newInfo->ii_ExclusionProcs = oldInfo->ii_ExclusionProcs; + newInfo->ii_ExclusionStrats = oldInfo->ii_ExclusionStrats; + } + /* * Extract the list of column names and the column numbers for the new * index information. All this information will be used for the index @@ -1436,6 +1448,9 @@ index_concurrently_create_copy(Relation heapRelation, Oid oldIndexId, stattargets[i].isnull = isnull; } + if (concurrently) + flags = INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT; + /* * Now create the new index. * @@ -1459,7 +1474,7 @@ index_concurrently_create_copy(Relation heapRelation, Oid oldIndexId, indcoloptions->values, stattargets, reloptionsDatum, - INDEX_CREATE_SKIP_BUILD | INDEX_CREATE_CONCURRENT, + flags, 0, true, /* allow table to be a system catalog? */ false, /* is_internal? */ diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c index 373e8234794..cba379810c7 100644 --- a/src/backend/commands/indexcmds.c +++ b/src/backend/commands/indexcmds.c @@ -3989,10 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein tablespaceid = indexRel->rd_rel->reltablespace; /* Create new index definition based on given index */ - newIndexId = index_concurrently_create_copy(heapRel, - idx->indexId, - tablespaceid, - concurrentName); + newIndexId = index_create_copy(heapRel, + true, + idx->indexId, + tablespaceid, + concurrentName); /* * Now open the relation of the new index, a session-level lock is diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h index a38e95bc0eb..ed9e4c37d27 100644 --- a/src/include/catalog/index.h +++ b/src/include/catalog/index.h @@ -101,10 +101,9 @@ extern Oid index_create(Relation heapRelation, #define INDEX_CONSTR_CREATE_REMOVE_OLD_DEPS (1 << 4) #define INDEX_CONSTR_CREATE_WITHOUT_OVERLAPS (1 << 5) -extern Oid index_concurrently_create_copy(Relation heapRelation, - Oid oldIndexId, - Oid tablespaceOid, - const char *newName); +extern Oid index_create_copy(Relation heapRelation, bool concurrently, + Oid oldIndexId, Oid tablespaceOid, + const char *newName); extern void index_concurrently_build(Oid heapRelationId, Oid indexRelationId); -- 2.47.3 --r2slln3zpmilwu22 Content-Type: text/x-diff; charset=utf-8 Content-Disposition: attachment; filename="v51-0002-Rename-cluster.c-h-repack.c-h.patch" ^ permalink raw reply [nested|flat] 477+ messages in thread
end of thread, other threads:[~2026-03-24 18:02 UTC | newest] Thread overview: 477+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2025-02-13 15:09 [PATCH v1] BackgroundPsql: Fix potential for lost errors on windows Andres Freund <andres@anarazel.de> 2026-03-24 18:02 [PATCH v51 01/10] Make index_concurrently_create_copy more general Álvaro Herrera <alvherre@kurilemu.de>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox