agora inbox for [email protected]  

help / color / mirror / Atom feed

[PATCH 8/8] vcregress: run alltaptests in parallel
11+ messages / 2 participants
[nested] [flat]

* [PATCH 8/8] vcregress: run alltaptests in parallel
@ 2022-01-18 23:48  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 11+ messages in thread

From: Justin Pryzby @ 2022-01-18 23:48 UTC (permalink / raw)

The test changes are needed to avoid these failures:

https://github.com/justinpryzby/postgres/runs/5174636590
[15:59:59.408] Bailout called.  Further testing stopped:  could not create test directory "c:/cirrus/tmp_check/t_C:\cirrus\src\bin\pgbench\t\002_pgbench_no_server_stuff": Invalid argument
[15:59:59.408] FAILED--Further testing stopped: could not create test directory "c:/cirrus/tmp_check/t_C:\cirrus\src\bin\pgbench\t\002_pgbench_no_server_stuff": Invalid argument

https://github.com/justinpryzby/postgres/runs/5174788205
[16:37:09.891] #   Failed test 'reading traces/disallowed_in_pipeline.trace: could not open "traces/disallowed_in_pipeline.trace": The system cannot find the path specified at C:\cirrus\src\test\modules\libpq_pipeline\t\001_libpq_pipeline.pl line 70.

https://github.com/justinpryzby/postgres/runs/5174877506
pg_regress: could not open file "../regress/parallel_schedule" for reading: No such file or directory

XXX: avoid breaking src/bin/psql/t/010_tab_completion.pl

Should pass the srcdir to all taptests, or go back to writing the results to
tmp_check ?

See also:
f4ce6c4d3a30ec3a12c7f64b90a6fc82887ddd7b
795862c280c5949bafcd8c44543d887fd32b590a
db973ffb3ca43e65a0bf15175a35184a53bf977d
---
 src/bin/pgbench/t/002_pgbench_no_server.pl       |  4 ++--
 .../libpq_pipeline/t/001_libpq_pipeline.pl       |  5 ++++-
 src/test/recovery/t/027_stream_regress.pl        |  4 ++--
 src/tools/msvc/vcregress.pl                      | 16 +++++-----------
 4 files changed, 13 insertions(+), 16 deletions(-)

diff --git a/src/bin/pgbench/t/002_pgbench_no_server.pl b/src/bin/pgbench/t/002_pgbench_no_server.pl
index acad19edd0c..84e36dcd3fe 100644
--- a/src/bin/pgbench/t/002_pgbench_no_server.pl
+++ b/src/bin/pgbench/t/002_pgbench_no_server.pl
@@ -8,12 +8,12 @@
 use strict;
 use warnings;
 
+use File::Basename;
 use PostgreSQL::Test::Utils;
 use Test::More;
 
 # create a directory for scripts
-my $testname = $0;
-$testname =~ s,.*/,,;
+my $testname = basename($0);
 $testname =~ s/\.pl$//;
 
 my $testdir = "$PostgreSQL::Test::Utils::tmp_check/t_${testname}_stuff";
diff --git a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
index 0c164dcaba5..facfec5cad4 100644
--- a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
+++ b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
@@ -49,7 +49,10 @@ for my $testname (@tests)
 		my $expected;
 		my $result;
 
-		$expected = slurp_file_eval("traces/$testname.trace");
+		# Hack to allow TESTDIR=. during parallel tap tests
+		my $inputdir = "$ENV{'TESTDIR'}/src/test/modules/libpq_pipeline";
+		$inputdir = "$ENV{'TESTDIR'}" if ! -e $inputdir;
+		$expected = slurp_file_eval("$inputdir/traces/$testname.trace");
 		next unless $expected ne "";
 		$result = slurp_file_eval($traceout);
 		next unless $result ne "";
diff --git a/src/test/recovery/t/027_stream_regress.pl b/src/test/recovery/t/027_stream_regress.pl
index c0aae707ea1..4b84e3e78f5 100644
--- a/src/test/recovery/t/027_stream_regress.pl
+++ b/src/test/recovery/t/027_stream_regress.pl
@@ -58,9 +58,9 @@ system_or_bail($ENV{PG_REGRESS} . " $extra_opts " .
 			   "--bindir= " .
 			   "--host=" . $node_primary->host . " " .
 			   "--port=" . $node_primary->port . " " .
-			   "--schedule=../regress/parallel_schedule " .
+			   "--schedule=\"$dlpath/parallel_schedule\" " .
 			   "--max-concurrent-tests=20 " .
-			   "--inputdir=../regress " .
+			   "--inputdir=\"$dlpath\" " .
 			   "--outputdir=\"$outputdir\"");
 
 # Clobber all sequences with their next value, so that we don't have
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index 4572db8eca4..b178713b911 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -299,8 +299,6 @@ sub alltaptests
 {
 	InstallTemp();
 
-	my $mstat = 0;
-
 	# Find out all the existing TAP tests by looking for t/ directories
 	# in the tree.
 	my @tap_dirs = ();
@@ -314,15 +312,11 @@ sub alltaptests
 		},
 		@top_dir);
 
-	$ENV{REGRESS_OUTPUTDIR} = "$topdir/src/test/recovery/tmp_check";
-	# Process each test
-	foreach my $test_path (@tap_dirs)
-	{
-		my $dir = dirname($test_path);
-		my $status = tap_check($dir);
-		$mstat ||= $status;
-	}
-	exit $mstat if $mstat;
+	# Run all the tap tests in a single prove instance for good performance
+	$ENV{REGRESS_OUTPUTDIR} = "$topdir/src/test/recovery/tmp_check"; # For recovery check
+	$ENV{PROVE_TESTS} = "@tap_dirs";
+	my $status = tap_check('PROVE_FLAGS=--ext=.pl', "$topdir");
+	exit $status if $status;
 	return;
 }
 
-- 
2.17.1


--4e5ZDkbgLEOfWmLx--





^ permalink  raw  reply  [nested|flat] 11+ messages in thread

* [PATCH 10/21] vcregress: run alltaptests in parallel
@ 2022-02-19 19:41  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 11+ messages in thread

From: Justin Pryzby @ 2022-02-19 19:41 UTC (permalink / raw)

ci-os-only: windows

The test changes are needed to avoid these failures:

https://github.com/justinpryzby/postgres/runs/5174636590
[15:59:59.408] Bailout called.  Further testing stopped:  could not create test directory "c:/cirrus/tmp_check/t_C:\cirrus\src\bin\pgbench\t\002_pgbench_no_server_stuff": Invalid argument
[15:59:59.408] FAILED--Further testing stopped: could not create test directory "c:/cirrus/tmp_check/t_C:\cirrus\src\bin\pgbench\t\002_pgbench_no_server_stuff": Invalid argument

https://github.com/justinpryzby/postgres/runs/5174788205
[16:37:09.891] #   Failed test 'reading traces/disallowed_in_pipeline.trace: could not open "traces/disallowed_in_pipeline.trace": The system cannot find the path specified at C:\cirrus\src\test\modules\libpq_pipeline\t\001_libpq_pipeline.pl line 70.

https://github.com/justinpryzby/postgres/runs/5174877506
pg_regress: could not open file "../regress/parallel_schedule" for reading: No such file or directory

XXX: avoid breaking src/bin/psql/t/010_tab_completion.pl:

See also:
f4ce6c4d3a30ec3a12c7f64b90a6fc82887ddd7b
795862c280c5949bafcd8c44543d887fd32b590a
db973ffb3ca43e65a0bf15175a35184a53bf977d
---
 src/bin/pg_upgrade/t/002_pg_upgrade.pl             |  2 +-
 src/bin/pgbench/t/002_pgbench_no_server.pl         |  4 ++--
 .../modules/libpq_pipeline/t/001_libpq_pipeline.pl |  3 ++-
 src/test/modules/test_misc/t/003_check_guc.pl      |  2 +-
 src/test/recovery/t/027_stream_regress.pl          |  5 +++--
 src/tools/msvc/vcregress.pl                        | 14 ++++----------
 6 files changed, 13 insertions(+), 17 deletions(-)

diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index 2f9b13bf0ae..757f4c76f7d 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -64,7 +64,7 @@ $oldnode->init(extra => [ '--wal-segsize', '1', '--allow-group-access' ]);
 $oldnode->start;
 
 # The default location of the source code is the root of this directory.
-my $srcdir = abs_path("../../..");
+my $srcdir = abs_path("$ENV{TESTDIR}/../../..");
 
 # Set up the data of the old instance with a dump or pg_regress.
 if (defined($ENV{olddump}))
diff --git a/src/bin/pgbench/t/002_pgbench_no_server.pl b/src/bin/pgbench/t/002_pgbench_no_server.pl
index 50bde7dd0fc..bbbb49bb23d 100644
--- a/src/bin/pgbench/t/002_pgbench_no_server.pl
+++ b/src/bin/pgbench/t/002_pgbench_no_server.pl
@@ -8,12 +8,12 @@
 use strict;
 use warnings;
 
+use File::Basename;
 use PostgreSQL::Test::Utils;
 use Test::More;
 
 # create a directory for scripts
-my $testname = $0;
-$testname =~ s,.*/,,;
+my $testname = basename($0);
 $testname =~ s/\.pl$//;
 
 my $testdir = "$PostgreSQL::Test::Utils::tmp_check/t_${testname}_stuff";
diff --git a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
index 0821329c8d3..7e6752562fd 100644
--- a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
+++ b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
@@ -50,7 +50,8 @@ for my $testname (@tests)
 		my $expected;
 		my $result;
 
-		$expected = slurp_file_eval("traces/$testname.trace");
+		my $inputdir = "$ENV{'TESTDIR'}";
+		$expected = slurp_file_eval("$inputdir/traces/$testname.trace");
 		next unless $expected ne "";
 		$result = slurp_file_eval($traceout);
 		next unless $result ne "";
diff --git a/src/test/modules/test_misc/t/003_check_guc.pl b/src/test/modules/test_misc/t/003_check_guc.pl
index 60459ef759e..167b6ea44b7 100644
--- a/src/test/modules/test_misc/t/003_check_guc.pl
+++ b/src/test/modules/test_misc/t/003_check_guc.pl
@@ -35,7 +35,7 @@ my @not_in_sample_array = split("\n", lc($not_in_sample));
 
 # TAP tests are executed in the directory of the test, in the source tree,
 # even for VPATH builds, so rely on that to find postgresql.conf.sample.
-my $rootdir     = "../../../..";
+my $rootdir     = "$ENV{TESTDIR}/../../../..";
 my $sample_file = "$rootdir/src/backend/utils/misc/postgresql.conf.sample";
 
 # List of all the GUCs found in the sample file.
diff --git a/src/test/recovery/t/027_stream_regress.pl b/src/test/recovery/t/027_stream_regress.pl
index 7982ac08d0a..c062877bec6 100644
--- a/src/test/recovery/t/027_stream_regress.pl
+++ b/src/test/recovery/t/027_stream_regress.pl
@@ -70,10 +70,11 @@ my $rc =
 	  . $node_primary->host . " "
 	  . "--port="
 	  . $node_primary->port . " "
-	  . "--schedule=../regress/parallel_schedule "
+	  . "--schedule=\"$dlpath/parallel_schedule\" "
 	  . "--max-concurrent-tests=20 "
-	  . "--inputdir=../regress "
+	  . "--inputdir=\"$dlpath\" "
 	  . "--outputdir=\"$outputdir\"");
+
 if ($rc != 0)
 {
 	# Dump out the regression diffs file, if there is one
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index 5dcc3419ad7..49168e298ac 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -328,8 +328,6 @@ sub alltaptests
 {
 	InstallTemp();
 
-	my $mstat = 0;
-
 	# Find out all the existing TAP tests by looking for t/ directories
 	# in the tree.
 	my @tap_dirs = ();
@@ -343,14 +341,10 @@ sub alltaptests
 		},
 		@top_dir);
 
-	# Process each test
-	foreach my $test_path (@tap_dirs)
-	{
-		my $dir = dirname($test_path);
-		my $status = tap_check($dir);
-		$mstat ||= $status;
-	}
-	exit $mstat if $mstat;
+	# Run all the tap tests in a single prove instance for better performance
+	$ENV{PROVE_TESTS} = "@tap_dirs";
+	my $status = tap_check('PROVE_FLAGS=--ext=.pl', "$topdir");
+	exit $status if $status;
 	return;
 }
 
-- 
2.17.1


--mln0rGgUGuXEqmuI
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="0011-cirrus-make-DebugInformationFormat-OldStyle-for-CI-b.patch"



^ permalink  raw  reply  [nested|flat] 11+ messages in thread

* [PATCH 16/25] vcregress: run alltaptests in parallel
@ 2022-02-19 19:41  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 11+ messages in thread

From: Justin Pryzby @ 2022-02-19 19:41 UTC (permalink / raw)

XX-os-only: windows

The test changes are needed to avoid these failures:

https://github.com/justinpryzby/postgres/runs/5174636590
[15:59:59.408] Bailout called.  Further testing stopped:  could not create test directory "c:/cirrus/tmp_check/t_C:\cirrus\src\bin\pgbench\t\002_pgbench_no_server_stuff": Invalid argument
[15:59:59.408] FAILED--Further testing stopped: could not create test directory "c:/cirrus/tmp_check/t_C:\cirrus\src\bin\pgbench\t\002_pgbench_no_server_stuff": Invalid argument

https://github.com/justinpryzby/postgres/runs/5174788205
[16:37:09.891] #   Failed test 'reading traces/disallowed_in_pipeline.trace: could not open "traces/disallowed_in_pipeline.trace": The system cannot find the path specified at C:\cirrus\src\test\modules\libpq_pipeline\t\001_libpq_pipeline.pl line 70.

https://github.com/justinpryzby/postgres/runs/5174877506
pg_regress: could not open file "../regress/parallel_schedule" for reading: No such file or directory

XXX: avoid breaking src/bin/psql/t/010_tab_completion.pl:

See also:
f4ce6c4d3a30ec3a12c7f64b90a6fc82887ddd7b
795862c280c5949bafcd8c44543d887fd32b590a
db973ffb3ca43e65a0bf15175a35184a53bf977d
---
 src/bin/pgbench/t/002_pgbench_no_server.pl |  4 ++--
 src/tools/msvc/vcregress.pl                | 14 ++++----------
 2 files changed, 6 insertions(+), 12 deletions(-)

diff --git a/src/bin/pgbench/t/002_pgbench_no_server.pl b/src/bin/pgbench/t/002_pgbench_no_server.pl
index 50bde7dd0fc..bbbb49bb23d 100644
--- a/src/bin/pgbench/t/002_pgbench_no_server.pl
+++ b/src/bin/pgbench/t/002_pgbench_no_server.pl
@@ -8,12 +8,12 @@
 use strict;
 use warnings;
 
+use File::Basename;
 use PostgreSQL::Test::Utils;
 use Test::More;
 
 # create a directory for scripts
-my $testname = $0;
-$testname =~ s,.*/,,;
+my $testname = basename($0);
 $testname =~ s/\.pl$//;
 
 my $testdir = "$PostgreSQL::Test::Utils::tmp_check/t_${testname}_stuff";
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index 5dcc3419ad7..49168e298ac 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -328,8 +328,6 @@ sub alltaptests
 {
 	InstallTemp();
 
-	my $mstat = 0;
-
 	# Find out all the existing TAP tests by looking for t/ directories
 	# in the tree.
 	my @tap_dirs = ();
@@ -343,14 +341,10 @@ sub alltaptests
 		},
 		@top_dir);
 
-	# Process each test
-	foreach my $test_path (@tap_dirs)
-	{
-		my $dir = dirname($test_path);
-		my $status = tap_check($dir);
-		$mstat ||= $status;
-	}
-	exit $mstat if $mstat;
+	# Run all the tap tests in a single prove instance for better performance
+	$ENV{PROVE_TESTS} = "@tap_dirs";
+	my $status = tap_check('PROVE_FLAGS=--ext=.pl', "$topdir");
+	exit $status if $status;
 	return;
 }
 
-- 
2.17.1


--IS0zKkzwUGydFO0o
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="0017-another-way-to-install-uri_regress-testclient.patch"



^ permalink  raw  reply  [nested|flat] 11+ messages in thread

* [PATCH 10/19] vcregress: run alltaptests in parallel
@ 2022-02-19 19:41  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 11+ messages in thread

From: Justin Pryzby @ 2022-02-19 19:41 UTC (permalink / raw)

ci-os-only: windows

The test changes are needed to avoid these failures:

https://github.com/justinpryzby/postgres/runs/5174636590
[15:59:59.408] Bailout called.  Further testing stopped:  could not create test directory "c:/cirrus/tmp_check/t_C:\cirrus\src\bin\pgbench\t\002_pgbench_no_server_stuff": Invalid argument
[15:59:59.408] FAILED--Further testing stopped: could not create test directory "c:/cirrus/tmp_check/t_C:\cirrus\src\bin\pgbench\t\002_pgbench_no_server_stuff": Invalid argument

https://github.com/justinpryzby/postgres/runs/5174788205
[16:37:09.891] #   Failed test 'reading traces/disallowed_in_pipeline.trace: could not open "traces/disallowed_in_pipeline.trace": The system cannot find the path specified at C:\cirrus\src\test\modules\libpq_pipeline\t\001_libpq_pipeline.pl line 70.

https://github.com/justinpryzby/postgres/runs/5174877506
pg_regress: could not open file "../regress/parallel_schedule" for reading: No such file or directory

XXX: avoid breaking src/bin/psql/t/010_tab_completion.pl:

See also:
f4ce6c4d3a30ec3a12c7f64b90a6fc82887ddd7b
795862c280c5949bafcd8c44543d887fd32b590a
db973ffb3ca43e65a0bf15175a35184a53bf977d
---
 src/bin/pg_upgrade/t/002_pg_upgrade.pl             |  2 +-
 src/bin/pgbench/t/002_pgbench_no_server.pl         |  4 ++--
 .../modules/libpq_pipeline/t/001_libpq_pipeline.pl |  3 ++-
 src/test/modules/test_misc/t/003_check_guc.pl      |  2 +-
 src/test/recovery/t/027_stream_regress.pl          |  5 +++--
 src/tools/msvc/vcregress.pl                        | 14 ++++----------
 6 files changed, 13 insertions(+), 17 deletions(-)

diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index 75ac768a96e..3f8fc986e72 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -60,7 +60,7 @@ $oldnode->init(extra => [ '--wal-segsize', '1', '--allow-group-access' ]);
 $oldnode->start;
 
 # The default location of the source code is the root of this directory.
-my $srcdir = abs_path("../../..");
+my $srcdir = abs_path("$ENV{TESTDIR}/../../..");
 
 # Set up the data of the old instance with a dump or pg_regress.
 if (defined($ENV{olddump}))
diff --git a/src/bin/pgbench/t/002_pgbench_no_server.pl b/src/bin/pgbench/t/002_pgbench_no_server.pl
index 50bde7dd0fc..bbbb49bb23d 100644
--- a/src/bin/pgbench/t/002_pgbench_no_server.pl
+++ b/src/bin/pgbench/t/002_pgbench_no_server.pl
@@ -8,12 +8,12 @@
 use strict;
 use warnings;
 
+use File::Basename;
 use PostgreSQL::Test::Utils;
 use Test::More;
 
 # create a directory for scripts
-my $testname = $0;
-$testname =~ s,.*/,,;
+my $testname = basename($0);
 $testname =~ s/\.pl$//;
 
 my $testdir = "$PostgreSQL::Test::Utils::tmp_check/t_${testname}_stuff";
diff --git a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
index 4cb1170438a..e79980469b4 100644
--- a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
+++ b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
@@ -49,7 +49,8 @@ for my $testname (@tests)
 		my $expected;
 		my $result;
 
-		$expected = slurp_file_eval("traces/$testname.trace");
+		my $inputdir = "$ENV{'TESTDIR'}/test";
+		$expected = slurp_file_eval("$inputdir/$testname.trace");
 		next unless $expected ne "";
 		$result = slurp_file_eval($traceout);
 		next unless $result ne "";
diff --git a/src/test/modules/test_misc/t/003_check_guc.pl b/src/test/modules/test_misc/t/003_check_guc.pl
index 60459ef759e..167b6ea44b7 100644
--- a/src/test/modules/test_misc/t/003_check_guc.pl
+++ b/src/test/modules/test_misc/t/003_check_guc.pl
@@ -35,7 +35,7 @@ my @not_in_sample_array = split("\n", lc($not_in_sample));
 
 # TAP tests are executed in the directory of the test, in the source tree,
 # even for VPATH builds, so rely on that to find postgresql.conf.sample.
-my $rootdir     = "../../../..";
+my $rootdir     = "$ENV{TESTDIR}/../../../..";
 my $sample_file = "$rootdir/src/backend/utils/misc/postgresql.conf.sample";
 
 # List of all the GUCs found in the sample file.
diff --git a/src/test/recovery/t/027_stream_regress.pl b/src/test/recovery/t/027_stream_regress.pl
index fdb4ea0bf50..c1ec234d607 100644
--- a/src/test/recovery/t/027_stream_regress.pl
+++ b/src/test/recovery/t/027_stream_regress.pl
@@ -70,10 +70,11 @@ my $rc =
 	  . $node_primary->host . " "
 	  . "--port="
 	  . $node_primary->port . " "
-	  . "--schedule=../regress/parallel_schedule "
+	  . "--schedule=\"$dlpath/parallel_schedule\" "
 	  . "--max-concurrent-tests=20 "
-	  . "--inputdir=../regress "
+	  . "--inputdir=\"$dlpath\" "
 	  . "--outputdir=\"$outputdir\"");
+
 if ($rc != 0)
 {
 	# Dump out the regression diffs file, if there is one
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index 1b1387acac0..870b848919c 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -328,8 +328,6 @@ sub alltaptests
 {
 	InstallTemp();
 
-	my $mstat = 0;
-
 	# Find out all the existing TAP tests by looking for t/ directories
 	# in the tree.
 	my @tap_dirs = ();
@@ -343,14 +341,10 @@ sub alltaptests
 		},
 		@top_dir);
 
-	# Process each test
-	foreach my $test_path (@tap_dirs)
-	{
-		my $dir = dirname($test_path);
-		my $status = tap_check($dir);
-		$mstat ||= $status;
-	}
-	exit $mstat if $mstat;
+	# Run all the tap tests in a single prove instance for better performance
+	$ENV{PROVE_TESTS} = "@tap_dirs";
+	my $status = tap_check('PROVE_FLAGS=--ext=.pl', "$topdir");
+	exit $status if $status;
 	return;
 }
 
-- 
2.17.1


--Sw7tCqrGA+HQ0/zt
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="0011-cirrus-make-DebugInformationFormat-OldStyle-for-CI-b.patch"



^ permalink  raw  reply  [nested|flat] 11+ messages in thread

* [PATCH 10/21] vcregress: run alltaptests in parallel
@ 2022-02-19 19:41  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 11+ messages in thread

From: Justin Pryzby @ 2022-02-19 19:41 UTC (permalink / raw)

ci-os-only: windows

The test changes are needed to avoid these failures:

https://github.com/justinpryzby/postgres/runs/5174636590
[15:59:59.408] Bailout called.  Further testing stopped:  could not create test directory "c:/cirrus/tmp_check/t_C:\cirrus\src\bin\pgbench\t\002_pgbench_no_server_stuff": Invalid argument
[15:59:59.408] FAILED--Further testing stopped: could not create test directory "c:/cirrus/tmp_check/t_C:\cirrus\src\bin\pgbench\t\002_pgbench_no_server_stuff": Invalid argument

https://github.com/justinpryzby/postgres/runs/5174788205
[16:37:09.891] #   Failed test 'reading traces/disallowed_in_pipeline.trace: could not open "traces/disallowed_in_pipeline.trace": The system cannot find the path specified at C:\cirrus\src\test\modules\libpq_pipeline\t\001_libpq_pipeline.pl line 70.

https://github.com/justinpryzby/postgres/runs/5174877506
pg_regress: could not open file "../regress/parallel_schedule" for reading: No such file or directory

XXX: avoid breaking src/bin/psql/t/010_tab_completion.pl:

See also:
f4ce6c4d3a30ec3a12c7f64b90a6fc82887ddd7b
795862c280c5949bafcd8c44543d887fd32b590a
db973ffb3ca43e65a0bf15175a35184a53bf977d
---
 src/bin/pg_upgrade/t/002_pg_upgrade.pl             |  2 +-
 src/bin/pgbench/t/002_pgbench_no_server.pl         |  4 ++--
 .../modules/libpq_pipeline/t/001_libpq_pipeline.pl |  3 ++-
 src/test/modules/test_misc/t/003_check_guc.pl      |  2 +-
 src/test/recovery/t/027_stream_regress.pl          |  5 +++--
 src/tools/msvc/vcregress.pl                        | 14 ++++----------
 6 files changed, 13 insertions(+), 17 deletions(-)

diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index 3f11540e189..90bfcfed026 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -64,7 +64,7 @@ $oldnode->init(extra => [ '--wal-segsize', '1', '--allow-group-access' ]);
 $oldnode->start;
 
 # The default location of the source code is the root of this directory.
-my $srcdir = abs_path("../../..");
+my $srcdir = abs_path("$ENV{TESTDIR}/../../..");
 
 # Set up the data of the old instance with a dump or pg_regress.
 if (defined($ENV{olddump}))
diff --git a/src/bin/pgbench/t/002_pgbench_no_server.pl b/src/bin/pgbench/t/002_pgbench_no_server.pl
index 50bde7dd0fc..bbbb49bb23d 100644
--- a/src/bin/pgbench/t/002_pgbench_no_server.pl
+++ b/src/bin/pgbench/t/002_pgbench_no_server.pl
@@ -8,12 +8,12 @@
 use strict;
 use warnings;
 
+use File::Basename;
 use PostgreSQL::Test::Utils;
 use Test::More;
 
 # create a directory for scripts
-my $testname = $0;
-$testname =~ s,.*/,,;
+my $testname = basename($0);
 $testname =~ s/\.pl$//;
 
 my $testdir = "$PostgreSQL::Test::Utils::tmp_check/t_${testname}_stuff";
diff --git a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
index 4cb1170438a..e5c1c0bfa89 100644
--- a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
+++ b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
@@ -49,7 +49,8 @@ for my $testname (@tests)
 		my $expected;
 		my $result;
 
-		$expected = slurp_file_eval("traces/$testname.trace");
+		my $inputdir = "$ENV{'TESTDIR'}";
+		$expected = slurp_file_eval("$inputdir/traces/$testname.trace");
 		next unless $expected ne "";
 		$result = slurp_file_eval($traceout);
 		next unless $result ne "";
diff --git a/src/test/modules/test_misc/t/003_check_guc.pl b/src/test/modules/test_misc/t/003_check_guc.pl
index 60459ef759e..167b6ea44b7 100644
--- a/src/test/modules/test_misc/t/003_check_guc.pl
+++ b/src/test/modules/test_misc/t/003_check_guc.pl
@@ -35,7 +35,7 @@ my @not_in_sample_array = split("\n", lc($not_in_sample));
 
 # TAP tests are executed in the directory of the test, in the source tree,
 # even for VPATH builds, so rely on that to find postgresql.conf.sample.
-my $rootdir     = "../../../..";
+my $rootdir     = "$ENV{TESTDIR}/../../../..";
 my $sample_file = "$rootdir/src/backend/utils/misc/postgresql.conf.sample";
 
 # List of all the GUCs found in the sample file.
diff --git a/src/test/recovery/t/027_stream_regress.pl b/src/test/recovery/t/027_stream_regress.pl
index fdb4ea0bf50..c1ec234d607 100644
--- a/src/test/recovery/t/027_stream_regress.pl
+++ b/src/test/recovery/t/027_stream_regress.pl
@@ -70,10 +70,11 @@ my $rc =
 	  . $node_primary->host . " "
 	  . "--port="
 	  . $node_primary->port . " "
-	  . "--schedule=../regress/parallel_schedule "
+	  . "--schedule=\"$dlpath/parallel_schedule\" "
 	  . "--max-concurrent-tests=20 "
-	  . "--inputdir=../regress "
+	  . "--inputdir=\"$dlpath\" "
 	  . "--outputdir=\"$outputdir\"");
+
 if ($rc != 0)
 {
 	# Dump out the regression diffs file, if there is one
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index 5dcc3419ad7..49168e298ac 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -328,8 +328,6 @@ sub alltaptests
 {
 	InstallTemp();
 
-	my $mstat = 0;
-
 	# Find out all the existing TAP tests by looking for t/ directories
 	# in the tree.
 	my @tap_dirs = ();
@@ -343,14 +341,10 @@ sub alltaptests
 		},
 		@top_dir);
 
-	# Process each test
-	foreach my $test_path (@tap_dirs)
-	{
-		my $dir = dirname($test_path);
-		my $status = tap_check($dir);
-		$mstat ||= $status;
-	}
-	exit $mstat if $mstat;
+	# Run all the tap tests in a single prove instance for better performance
+	$ENV{PROVE_TESTS} = "@tap_dirs";
+	my $status = tap_check('PROVE_FLAGS=--ext=.pl', "$topdir");
+	exit $status if $status;
 	return;
 }
 
-- 
2.17.1


--STPqjqpCrtky8aYs
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="0011-cirrus-make-DebugInformationFormat-OldStyle-for-CI-b.patch"



^ permalink  raw  reply  [nested|flat] 11+ messages in thread

* [PATCH 6/7] vcregress: run alltaptests in parallel
@ 2022-02-19 19:41  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 11+ messages in thread

From: Justin Pryzby @ 2022-02-19 19:41 UTC (permalink / raw)

ci-os-only: windows

The test changes are needed to avoid these failures:

https://github.com/justinpryzby/postgres/runs/5174636590
[15:59:59.408] Bailout called.  Further testing stopped:  could not create test directory "c:/cirrus/tmp_check/t_C:\cirrus\src\bin\pgbench\t\002_pgbench_no_server_stuff": Invalid argument
[15:59:59.408] FAILED--Further testing stopped: could not create test directory "c:/cirrus/tmp_check/t_C:\cirrus\src\bin\pgbench\t\002_pgbench_no_server_stuff": Invalid argument

https://github.com/justinpryzby/postgres/runs/5174788205
[16:37:09.891] #   Failed test 'reading traces/disallowed_in_pipeline.trace: could not open "traces/disallowed_in_pipeline.trace": The system cannot find the path specified at C:\cirrus\src\test\modules\libpq_pipeline\t\001_libpq_pipeline.pl line 70.

https://github.com/justinpryzby/postgres/runs/5174877506
pg_regress: could not open file "../regress/parallel_schedule" for reading: No such file or directory

XXX: avoid breaking src/bin/psql/t/010_tab_completion.pl:
Should pass the srcdir to all taptests, or go back to writing the results to
tmp_check ?

See also:
f4ce6c4d3a30ec3a12c7f64b90a6fc82887ddd7b
795862c280c5949bafcd8c44543d887fd32b590a
db973ffb3ca43e65a0bf15175a35184a53bf977d
---
 src/bin/pgbench/t/002_pgbench_no_server.pl         |  4 ++--
 .../modules/libpq_pipeline/t/001_libpq_pipeline.pl |  3 ++-
 src/test/modules/test_misc/t/003_check_guc.pl      |  2 +-
 src/test/recovery/t/027_stream_regress.pl          |  4 ++--
 src/tools/msvc/vcregress.pl                        | 14 ++++----------
 5 files changed, 11 insertions(+), 16 deletions(-)

diff --git a/src/bin/pgbench/t/002_pgbench_no_server.pl b/src/bin/pgbench/t/002_pgbench_no_server.pl
index acad19edd0..84e36dcd3f 100644
--- a/src/bin/pgbench/t/002_pgbench_no_server.pl
+++ b/src/bin/pgbench/t/002_pgbench_no_server.pl
@@ -8,12 +8,12 @@
 use strict;
 use warnings;
 
+use File::Basename;
 use PostgreSQL::Test::Utils;
 use Test::More;
 
 # create a directory for scripts
-my $testname = $0;
-$testname =~ s,.*/,,;
+my $testname = basename($0);
 $testname =~ s/\.pl$//;
 
 my $testdir = "$PostgreSQL::Test::Utils::tmp_check/t_${testname}_stuff";
diff --git a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
index 0c164dcaba..2a0eca7744 100644
--- a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
+++ b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
@@ -49,7 +49,8 @@ for my $testname (@tests)
 		my $expected;
 		my $result;
 
-		$expected = slurp_file_eval("traces/$testname.trace");
+		my $inputdir = "$ENV{'TESTDIR'}/tmp_check";
+		$expected = slurp_file_eval("$inputdir/traces/$testname.trace");
 		next unless $expected ne "";
 		$result = slurp_file_eval($traceout);
 		next unless $result ne "";
diff --git a/src/test/modules/test_misc/t/003_check_guc.pl b/src/test/modules/test_misc/t/003_check_guc.pl
index 60459ef759..167b6ea44b 100644
--- a/src/test/modules/test_misc/t/003_check_guc.pl
+++ b/src/test/modules/test_misc/t/003_check_guc.pl
@@ -35,7 +35,7 @@ my @not_in_sample_array = split("\n", lc($not_in_sample));
 
 # TAP tests are executed in the directory of the test, in the source tree,
 # even for VPATH builds, so rely on that to find postgresql.conf.sample.
-my $rootdir     = "../../../..";
+my $rootdir     = "$ENV{TESTDIR}/../../../..";
 my $sample_file = "$rootdir/src/backend/utils/misc/postgresql.conf.sample";
 
 # List of all the GUCs found in the sample file.
diff --git a/src/test/recovery/t/027_stream_regress.pl b/src/test/recovery/t/027_stream_regress.pl
index c40951b7ba..f7679306f5 100644
--- a/src/test/recovery/t/027_stream_regress.pl
+++ b/src/test/recovery/t/027_stream_regress.pl
@@ -58,9 +58,9 @@ system_or_bail($ENV{PG_REGRESS} . " $extra_opts " .
 			   "--bindir= " .
 			   "--host=" . $node_primary->host . " " .
 			   "--port=" . $node_primary->port . " " .
-			   "--schedule=../regress/parallel_schedule " .
+			   "--schedule=\"$dlpath/parallel_schedule\" " .
 			   "--max-concurrent-tests=20 " .
-			   "--inputdir=../regress " .
+			   "--inputdir=\"$dlpath\" " .
 			   "--outputdir=\"$outputdir\"");
 
 # Clobber all sequences with their next value, so that we don't have
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index e27c1d15e5..cc8d690c29 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -295,8 +295,6 @@ sub alltaptests
 {
 	InstallTemp();
 
-	my $mstat = 0;
-
 	# Find out all the existing TAP tests by looking for t/ directories
 	# in the tree.
 	my @tap_dirs = ();
@@ -310,14 +308,10 @@ sub alltaptests
 		},
 		@top_dir);
 
-	# Process each test
-	foreach my $test_path (@tap_dirs)
-	{
-		my $dir = dirname($test_path);
-		my $status = tap_check($dir);
-		$mstat ||= $status;
-	}
-	exit $mstat if $mstat;
+	# Run all the tap tests in a single prove instance for good performance
+	$ENV{PROVE_TESTS} = "@tap_dirs";
+	my $status = tap_check('PROVE_FLAGS=--ext=.pl', "$topdir");
+	exit $status if $status;
 	return;
 }
 
-- 
2.17.1


--ibvzjYYg+QDzMCy1
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0007-tmp-run-tap-tests-first.patch"



^ permalink  raw  reply  [nested|flat] 11+ messages in thread

* [PATCH 14/23] vcregress: run alltaptests in parallel
@ 2022-02-19 19:41  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 11+ messages in thread

From: Justin Pryzby @ 2022-02-19 19:41 UTC (permalink / raw)

XX-os-only: windows

The test changes are needed to avoid these failures:

https://github.com/justinpryzby/postgres/runs/5174636590
[15:59:59.408] Bailout called.  Further testing stopped:  could not create test directory "c:/cirrus/tmp_check/t_C:\cirrus\src\bin\pgbench\t\002_pgbench_no_server_stuff": Invalid argument
[15:59:59.408] FAILED--Further testing stopped: could not create test directory "c:/cirrus/tmp_check/t_C:\cirrus\src\bin\pgbench\t\002_pgbench_no_server_stuff": Invalid argument

https://github.com/justinpryzby/postgres/runs/5174788205
[16:37:09.891] #   Failed test 'reading traces/disallowed_in_pipeline.trace: could not open "traces/disallowed_in_pipeline.trace": The system cannot find the path specified at C:\cirrus\src\test\modules\libpq_pipeline\t\001_libpq_pipeline.pl line 70.

https://github.com/justinpryzby/postgres/runs/5174877506
pg_regress: could not open file "../regress/parallel_schedule" for reading: No such file or directory

XXX: avoid breaking src/bin/psql/t/010_tab_completion.pl:

See also:
f4ce6c4d3a30ec3a12c7f64b90a6fc82887ddd7b
795862c280c5949bafcd8c44543d887fd32b590a
db973ffb3ca43e65a0bf15175a35184a53bf977d
---
 src/bin/pgbench/t/002_pgbench_no_server.pl |  4 ++--
 src/tools/msvc/vcregress.pl                | 14 ++++----------
 2 files changed, 6 insertions(+), 12 deletions(-)

diff --git a/src/bin/pgbench/t/002_pgbench_no_server.pl b/src/bin/pgbench/t/002_pgbench_no_server.pl
index 50bde7dd0fc..bbbb49bb23d 100644
--- a/src/bin/pgbench/t/002_pgbench_no_server.pl
+++ b/src/bin/pgbench/t/002_pgbench_no_server.pl
@@ -8,12 +8,12 @@
 use strict;
 use warnings;
 
+use File::Basename;
 use PostgreSQL::Test::Utils;
 use Test::More;
 
 # create a directory for scripts
-my $testname = $0;
-$testname =~ s,.*/,,;
+my $testname = basename($0);
 $testname =~ s/\.pl$//;
 
 my $testdir = "$PostgreSQL::Test::Utils::tmp_check/t_${testname}_stuff";
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index 5dcc3419ad7..49168e298ac 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -328,8 +328,6 @@ sub alltaptests
 {
 	InstallTemp();
 
-	my $mstat = 0;
-
 	# Find out all the existing TAP tests by looking for t/ directories
 	# in the tree.
 	my @tap_dirs = ();
@@ -343,14 +341,10 @@ sub alltaptests
 		},
 		@top_dir);
 
-	# Process each test
-	foreach my $test_path (@tap_dirs)
-	{
-		my $dir = dirname($test_path);
-		my $status = tap_check($dir);
-		$mstat ||= $status;
-	}
-	exit $mstat if $mstat;
+	# Run all the tap tests in a single prove instance for better performance
+	$ENV{PROVE_TESTS} = "@tap_dirs";
+	my $status = tap_check('PROVE_FLAGS=--ext=.pl', "$topdir");
+	exit $status if $status;
 	return;
 }
 
-- 
2.17.1


--tKkaNMvYmhQvRCRK
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="0015-another-way-to-install-uri_regress-testclient.patch"



^ permalink  raw  reply  [nested|flat] 11+ messages in thread

* [PATCH 10/19] vcregress: run alltaptests in parallel
@ 2022-02-19 19:41  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 11+ messages in thread

From: Justin Pryzby @ 2022-02-19 19:41 UTC (permalink / raw)

ci-os-only: windows

The test changes are needed to avoid these failures:

https://github.com/justinpryzby/postgres/runs/5174636590
[15:59:59.408] Bailout called.  Further testing stopped:  could not create test directory "c:/cirrus/tmp_check/t_C:\cirrus\src\bin\pgbench\t\002_pgbench_no_server_stuff": Invalid argument
[15:59:59.408] FAILED--Further testing stopped: could not create test directory "c:/cirrus/tmp_check/t_C:\cirrus\src\bin\pgbench\t\002_pgbench_no_server_stuff": Invalid argument

https://github.com/justinpryzby/postgres/runs/5174788205
[16:37:09.891] #   Failed test 'reading traces/disallowed_in_pipeline.trace: could not open "traces/disallowed_in_pipeline.trace": The system cannot find the path specified at C:\cirrus\src\test\modules\libpq_pipeline\t\001_libpq_pipeline.pl line 70.

https://github.com/justinpryzby/postgres/runs/5174877506
pg_regress: could not open file "../regress/parallel_schedule" for reading: No such file or directory

XXX: avoid breaking src/bin/psql/t/010_tab_completion.pl:

See also:
f4ce6c4d3a30ec3a12c7f64b90a6fc82887ddd7b
795862c280c5949bafcd8c44543d887fd32b590a
db973ffb3ca43e65a0bf15175a35184a53bf977d
---
 src/bin/pg_upgrade/t/002_pg_upgrade.pl             |  2 +-
 src/bin/pgbench/t/002_pgbench_no_server.pl         |  4 ++--
 .../modules/libpq_pipeline/t/001_libpq_pipeline.pl |  3 ++-
 src/test/modules/test_misc/t/003_check_guc.pl      |  2 +-
 src/test/recovery/t/027_stream_regress.pl          |  5 +++--
 src/tools/msvc/vcregress.pl                        | 14 ++++----------
 6 files changed, 13 insertions(+), 17 deletions(-)

diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index 75ac768a96e..3f8fc986e72 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -60,7 +60,7 @@ $oldnode->init(extra => [ '--wal-segsize', '1', '--allow-group-access' ]);
 $oldnode->start;
 
 # The default location of the source code is the root of this directory.
-my $srcdir = abs_path("../../..");
+my $srcdir = abs_path("$ENV{TESTDIR}/../../..");
 
 # Set up the data of the old instance with a dump or pg_regress.
 if (defined($ENV{olddump}))
diff --git a/src/bin/pgbench/t/002_pgbench_no_server.pl b/src/bin/pgbench/t/002_pgbench_no_server.pl
index 50bde7dd0fc..bbbb49bb23d 100644
--- a/src/bin/pgbench/t/002_pgbench_no_server.pl
+++ b/src/bin/pgbench/t/002_pgbench_no_server.pl
@@ -8,12 +8,12 @@
 use strict;
 use warnings;
 
+use File::Basename;
 use PostgreSQL::Test::Utils;
 use Test::More;
 
 # create a directory for scripts
-my $testname = $0;
-$testname =~ s,.*/,,;
+my $testname = basename($0);
 $testname =~ s/\.pl$//;
 
 my $testdir = "$PostgreSQL::Test::Utils::tmp_check/t_${testname}_stuff";
diff --git a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
index 4cb1170438a..e79980469b4 100644
--- a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
+++ b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
@@ -49,7 +49,8 @@ for my $testname (@tests)
 		my $expected;
 		my $result;
 
-		$expected = slurp_file_eval("traces/$testname.trace");
+		my $inputdir = "$ENV{'TESTDIR'}/test";
+		$expected = slurp_file_eval("$inputdir/$testname.trace");
 		next unless $expected ne "";
 		$result = slurp_file_eval($traceout);
 		next unless $result ne "";
diff --git a/src/test/modules/test_misc/t/003_check_guc.pl b/src/test/modules/test_misc/t/003_check_guc.pl
index 60459ef759e..167b6ea44b7 100644
--- a/src/test/modules/test_misc/t/003_check_guc.pl
+++ b/src/test/modules/test_misc/t/003_check_guc.pl
@@ -35,7 +35,7 @@ my @not_in_sample_array = split("\n", lc($not_in_sample));
 
 # TAP tests are executed in the directory of the test, in the source tree,
 # even for VPATH builds, so rely on that to find postgresql.conf.sample.
-my $rootdir     = "../../../..";
+my $rootdir     = "$ENV{TESTDIR}/../../../..";
 my $sample_file = "$rootdir/src/backend/utils/misc/postgresql.conf.sample";
 
 # List of all the GUCs found in the sample file.
diff --git a/src/test/recovery/t/027_stream_regress.pl b/src/test/recovery/t/027_stream_regress.pl
index fdb4ea0bf50..c1ec234d607 100644
--- a/src/test/recovery/t/027_stream_regress.pl
+++ b/src/test/recovery/t/027_stream_regress.pl
@@ -70,10 +70,11 @@ my $rc =
 	  . $node_primary->host . " "
 	  . "--port="
 	  . $node_primary->port . " "
-	  . "--schedule=../regress/parallel_schedule "
+	  . "--schedule=\"$dlpath/parallel_schedule\" "
 	  . "--max-concurrent-tests=20 "
-	  . "--inputdir=../regress "
+	  . "--inputdir=\"$dlpath\" "
 	  . "--outputdir=\"$outputdir\"");
+
 if ($rc != 0)
 {
 	# Dump out the regression diffs file, if there is one
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index 1b1387acac0..870b848919c 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -328,8 +328,6 @@ sub alltaptests
 {
 	InstallTemp();
 
-	my $mstat = 0;
-
 	# Find out all the existing TAP tests by looking for t/ directories
 	# in the tree.
 	my @tap_dirs = ();
@@ -343,14 +341,10 @@ sub alltaptests
 		},
 		@top_dir);
 
-	# Process each test
-	foreach my $test_path (@tap_dirs)
-	{
-		my $dir = dirname($test_path);
-		my $status = tap_check($dir);
-		$mstat ||= $status;
-	}
-	exit $mstat if $mstat;
+	# Run all the tap tests in a single prove instance for better performance
+	$ENV{PROVE_TESTS} = "@tap_dirs";
+	my $status = tap_check('PROVE_FLAGS=--ext=.pl', "$topdir");
+	exit $status if $status;
 	return;
 }
 
-- 
2.17.1


--Sw7tCqrGA+HQ0/zt
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="0011-cirrus-make-DebugInformationFormat-OldStyle-for-CI-b.patch"



^ permalink  raw  reply  [nested|flat] 11+ messages in thread

* [PATCH 10/19] vcregress: run alltaptests in parallel
@ 2022-02-19 19:41  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 11+ messages in thread

From: Justin Pryzby @ 2022-02-19 19:41 UTC (permalink / raw)

ci-os-only: windows

The test changes are needed to avoid these failures:

https://github.com/justinpryzby/postgres/runs/5174636590
[15:59:59.408] Bailout called.  Further testing stopped:  could not create test directory "c:/cirrus/tmp_check/t_C:\cirrus\src\bin\pgbench\t\002_pgbench_no_server_stuff": Invalid argument
[15:59:59.408] FAILED--Further testing stopped: could not create test directory "c:/cirrus/tmp_check/t_C:\cirrus\src\bin\pgbench\t\002_pgbench_no_server_stuff": Invalid argument

https://github.com/justinpryzby/postgres/runs/5174788205
[16:37:09.891] #   Failed test 'reading traces/disallowed_in_pipeline.trace: could not open "traces/disallowed_in_pipeline.trace": The system cannot find the path specified at C:\cirrus\src\test\modules\libpq_pipeline\t\001_libpq_pipeline.pl line 70.

https://github.com/justinpryzby/postgres/runs/5174877506
pg_regress: could not open file "../regress/parallel_schedule" for reading: No such file or directory

XXX: avoid breaking src/bin/psql/t/010_tab_completion.pl:

See also:
f4ce6c4d3a30ec3a12c7f64b90a6fc82887ddd7b
795862c280c5949bafcd8c44543d887fd32b590a
db973ffb3ca43e65a0bf15175a35184a53bf977d
---
 src/bin/pg_upgrade/t/002_pg_upgrade.pl             |  2 +-
 src/bin/pgbench/t/002_pgbench_no_server.pl         |  4 ++--
 .../modules/libpq_pipeline/t/001_libpq_pipeline.pl |  3 ++-
 src/test/modules/test_misc/t/003_check_guc.pl      |  2 +-
 src/test/recovery/t/027_stream_regress.pl          |  5 +++--
 src/tools/msvc/vcregress.pl                        | 14 ++++----------
 6 files changed, 13 insertions(+), 17 deletions(-)

diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index 75ac768a96e..3f8fc986e72 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -60,7 +60,7 @@ $oldnode->init(extra => [ '--wal-segsize', '1', '--allow-group-access' ]);
 $oldnode->start;
 
 # The default location of the source code is the root of this directory.
-my $srcdir = abs_path("../../..");
+my $srcdir = abs_path("$ENV{TESTDIR}/../../..");
 
 # Set up the data of the old instance with a dump or pg_regress.
 if (defined($ENV{olddump}))
diff --git a/src/bin/pgbench/t/002_pgbench_no_server.pl b/src/bin/pgbench/t/002_pgbench_no_server.pl
index 50bde7dd0fc..bbbb49bb23d 100644
--- a/src/bin/pgbench/t/002_pgbench_no_server.pl
+++ b/src/bin/pgbench/t/002_pgbench_no_server.pl
@@ -8,12 +8,12 @@
 use strict;
 use warnings;
 
+use File::Basename;
 use PostgreSQL::Test::Utils;
 use Test::More;
 
 # create a directory for scripts
-my $testname = $0;
-$testname =~ s,.*/,,;
+my $testname = basename($0);
 $testname =~ s/\.pl$//;
 
 my $testdir = "$PostgreSQL::Test::Utils::tmp_check/t_${testname}_stuff";
diff --git a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
index 4cb1170438a..e79980469b4 100644
--- a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
+++ b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
@@ -49,7 +49,8 @@ for my $testname (@tests)
 		my $expected;
 		my $result;
 
-		$expected = slurp_file_eval("traces/$testname.trace");
+		my $inputdir = "$ENV{'TESTDIR'}/test";
+		$expected = slurp_file_eval("$inputdir/$testname.trace");
 		next unless $expected ne "";
 		$result = slurp_file_eval($traceout);
 		next unless $result ne "";
diff --git a/src/test/modules/test_misc/t/003_check_guc.pl b/src/test/modules/test_misc/t/003_check_guc.pl
index 60459ef759e..167b6ea44b7 100644
--- a/src/test/modules/test_misc/t/003_check_guc.pl
+++ b/src/test/modules/test_misc/t/003_check_guc.pl
@@ -35,7 +35,7 @@ my @not_in_sample_array = split("\n", lc($not_in_sample));
 
 # TAP tests are executed in the directory of the test, in the source tree,
 # even for VPATH builds, so rely on that to find postgresql.conf.sample.
-my $rootdir     = "../../../..";
+my $rootdir     = "$ENV{TESTDIR}/../../../..";
 my $sample_file = "$rootdir/src/backend/utils/misc/postgresql.conf.sample";
 
 # List of all the GUCs found in the sample file.
diff --git a/src/test/recovery/t/027_stream_regress.pl b/src/test/recovery/t/027_stream_regress.pl
index fdb4ea0bf50..c1ec234d607 100644
--- a/src/test/recovery/t/027_stream_regress.pl
+++ b/src/test/recovery/t/027_stream_regress.pl
@@ -70,10 +70,11 @@ my $rc =
 	  . $node_primary->host . " "
 	  . "--port="
 	  . $node_primary->port . " "
-	  . "--schedule=../regress/parallel_schedule "
+	  . "--schedule=\"$dlpath/parallel_schedule\" "
 	  . "--max-concurrent-tests=20 "
-	  . "--inputdir=../regress "
+	  . "--inputdir=\"$dlpath\" "
 	  . "--outputdir=\"$outputdir\"");
+
 if ($rc != 0)
 {
 	# Dump out the regression diffs file, if there is one
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index 1b1387acac0..870b848919c 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -328,8 +328,6 @@ sub alltaptests
 {
 	InstallTemp();
 
-	my $mstat = 0;
-
 	# Find out all the existing TAP tests by looking for t/ directories
 	# in the tree.
 	my @tap_dirs = ();
@@ -343,14 +341,10 @@ sub alltaptests
 		},
 		@top_dir);
 
-	# Process each test
-	foreach my $test_path (@tap_dirs)
-	{
-		my $dir = dirname($test_path);
-		my $status = tap_check($dir);
-		$mstat ||= $status;
-	}
-	exit $mstat if $mstat;
+	# Run all the tap tests in a single prove instance for better performance
+	$ENV{PROVE_TESTS} = "@tap_dirs";
+	my $status = tap_check('PROVE_FLAGS=--ext=.pl', "$topdir");
+	exit $status if $status;
 	return;
 }
 
-- 
2.17.1


--Sw7tCqrGA+HQ0/zt
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="0011-cirrus-make-DebugInformationFormat-OldStyle-for-CI-b.patch"



^ permalink  raw  reply  [nested|flat] 11+ messages in thread

* [PATCH 14/23] vcregress: run alltaptests in parallel
@ 2022-02-19 19:41  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 11+ messages in thread

From: Justin Pryzby @ 2022-02-19 19:41 UTC (permalink / raw)

XX-os-only: windows

The test changes are needed to avoid these failures:

https://github.com/justinpryzby/postgres/runs/5174636590
[15:59:59.408] Bailout called.  Further testing stopped:  could not create test directory "c:/cirrus/tmp_check/t_C:\cirrus\src\bin\pgbench\t\002_pgbench_no_server_stuff": Invalid argument
[15:59:59.408] FAILED--Further testing stopped: could not create test directory "c:/cirrus/tmp_check/t_C:\cirrus\src\bin\pgbench\t\002_pgbench_no_server_stuff": Invalid argument

https://github.com/justinpryzby/postgres/runs/5174788205
[16:37:09.891] #   Failed test 'reading traces/disallowed_in_pipeline.trace: could not open "traces/disallowed_in_pipeline.trace": The system cannot find the path specified at C:\cirrus\src\test\modules\libpq_pipeline\t\001_libpq_pipeline.pl line 70.

https://github.com/justinpryzby/postgres/runs/5174877506
pg_regress: could not open file "../regress/parallel_schedule" for reading: No such file or directory

XXX: avoid breaking src/bin/psql/t/010_tab_completion.pl:

See also:
f4ce6c4d3a30ec3a12c7f64b90a6fc82887ddd7b
795862c280c5949bafcd8c44543d887fd32b590a
db973ffb3ca43e65a0bf15175a35184a53bf977d
---
 src/bin/pgbench/t/002_pgbench_no_server.pl |  4 ++--
 src/tools/msvc/vcregress.pl                | 14 ++++----------
 2 files changed, 6 insertions(+), 12 deletions(-)

diff --git a/src/bin/pgbench/t/002_pgbench_no_server.pl b/src/bin/pgbench/t/002_pgbench_no_server.pl
index 50bde7dd0fc..bbbb49bb23d 100644
--- a/src/bin/pgbench/t/002_pgbench_no_server.pl
+++ b/src/bin/pgbench/t/002_pgbench_no_server.pl
@@ -8,12 +8,12 @@
 use strict;
 use warnings;
 
+use File::Basename;
 use PostgreSQL::Test::Utils;
 use Test::More;
 
 # create a directory for scripts
-my $testname = $0;
-$testname =~ s,.*/,,;
+my $testname = basename($0);
 $testname =~ s/\.pl$//;
 
 my $testdir = "$PostgreSQL::Test::Utils::tmp_check/t_${testname}_stuff";
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index 5dcc3419ad7..49168e298ac 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -328,8 +328,6 @@ sub alltaptests
 {
 	InstallTemp();
 
-	my $mstat = 0;
-
 	# Find out all the existing TAP tests by looking for t/ directories
 	# in the tree.
 	my @tap_dirs = ();
@@ -343,14 +341,10 @@ sub alltaptests
 		},
 		@top_dir);
 
-	# Process each test
-	foreach my $test_path (@tap_dirs)
-	{
-		my $dir = dirname($test_path);
-		my $status = tap_check($dir);
-		$mstat ||= $status;
-	}
-	exit $mstat if $mstat;
+	# Run all the tap tests in a single prove instance for better performance
+	$ENV{PROVE_TESTS} = "@tap_dirs";
+	my $status = tap_check('PROVE_FLAGS=--ext=.pl', "$topdir");
+	exit $status if $status;
 	return;
 }
 
-- 
2.17.1


--tKkaNMvYmhQvRCRK
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="0015-another-way-to-install-uri_regress-testclient.patch"



^ permalink  raw  reply  [nested|flat] 11+ messages in thread

* [PATCH v2] Refactor grammar to create opt_utility_option_list
@ 2025-07-23 16:13  Álvaro Herrera <[email protected]>
  0 siblings, 0 replies; 11+ messages in thread

From: Álvaro Herrera @ 2025-07-23 16:13 UTC (permalink / raw)

This changes the grammar for REINDEX, CHECKPOINT and CLUSTER; they still
accept the same options as before, but the grammar is written
differently for convenience of future development.
---
 src/backend/parser/gram.y | 65 ++++++++++++++++++++++-----------------
 1 file changed, 36 insertions(+), 29 deletions(-)

diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 73345bb3c70..8a6236167db 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -318,6 +318,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
 %type <list>		opt_qualified_name
 %type <boolean>		opt_concurrently
 %type <dbehavior>	opt_drop_behavior
+%type <list>		opt_utility_option_list
 
 %type <node>	alter_column_default opclass_item opclass_drop alter_using
 %type <ival>	add_drop opt_asc_desc opt_nulls_order
@@ -556,7 +557,6 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
 %type <list>	generic_option_list alter_generic_option_list
 
 %type <ival>	reindex_target_relation reindex_target_all
-%type <list>	opt_reindex_option_list
 
 %type <node>	copy_generic_opt_arg copy_generic_opt_arg_list_item
 %type <defelt>	copy_generic_opt_elem
@@ -1141,6 +1141,11 @@ opt_drop_behavior:
 			| /* EMPTY */					{ $$ = DROP_RESTRICT; /* default */ }
 		;
 
+opt_utility_option_list:
+			'(' utility_option_list ')'		{ $$ = $2; }
+			| /* EMPTY */					{ $$ = NULL; }
+		;
+
 /*****************************************************************************
  *
  * CALL statement
@@ -2028,18 +2033,12 @@ constraints_set_mode:
  * Checkpoint statement
  */
 CheckPointStmt:
-			CHECKPOINT
+			CHECKPOINT opt_utility_option_list
 				{
 					CheckPointStmt *n = makeNode(CheckPointStmt);
 
 					$$ = (Node *) n;
-				}
-			| CHECKPOINT '(' utility_option_list ')'
-				{
-					CheckPointStmt *n = makeNode(CheckPointStmt);
-
-					$$ = (Node *) n;
-					n->options = $3;
+					n->options = $2;
 				}
 		;
 
@@ -9354,7 +9353,7 @@ DropTransformStmt: DROP TRANSFORM opt_if_exists FOR Typename LANGUAGE name opt_d
  *****************************************************************************/
 
 ReindexStmt:
-			REINDEX opt_reindex_option_list reindex_target_relation opt_concurrently qualified_name
+			REINDEX opt_utility_option_list reindex_target_relation opt_concurrently qualified_name
 				{
 					ReindexStmt *n = makeNode(ReindexStmt);
 
@@ -9367,7 +9366,7 @@ ReindexStmt:
 											makeDefElem("concurrently", NULL, @4));
 					$$ = (Node *) n;
 				}
-			| REINDEX opt_reindex_option_list SCHEMA opt_concurrently name
+			| REINDEX opt_utility_option_list SCHEMA opt_concurrently name
 				{
 					ReindexStmt *n = makeNode(ReindexStmt);
 
@@ -9380,7 +9379,7 @@ ReindexStmt:
 											makeDefElem("concurrently", NULL, @4));
 					$$ = (Node *) n;
 				}
-			| REINDEX opt_reindex_option_list reindex_target_all opt_concurrently opt_single_name
+			| REINDEX opt_utility_option_list reindex_target_all opt_concurrently opt_single_name
 				{
 					ReindexStmt *n = makeNode(ReindexStmt);
 
@@ -9402,10 +9401,6 @@ reindex_target_all:
 			SYSTEM_P				{ $$ = REINDEX_OBJECT_SYSTEM; }
 			| DATABASE				{ $$ = REINDEX_OBJECT_DATABASE; }
 		;
-opt_reindex_option_list:
-			'(' utility_option_list ')'				{ $$ = $2; }
-			| /* EMPTY */							{ $$ = NULL; }
-		;
 
 /*****************************************************************************
  *
@@ -11903,49 +11898,61 @@ ClusterStmt:
 					n->params = $3;
 					$$ = (Node *) n;
 				}
-			| CLUSTER '(' utility_option_list ')'
+			| CLUSTER qualified_name cluster_index_specification
+				{
+					ClusterStmt *n = makeNode(ClusterStmt);
+
+					n->relation = $2;
+					n->indexname = $3;
+					n->params = NIL;
+					$$ = (Node *) n;
+				}
+			| CLUSTER opt_utility_option_list
 				{
 					ClusterStmt *n = makeNode(ClusterStmt);
 
 					n->relation = NULL;
 					n->indexname = NULL;
-					n->params = $3;
+					n->params = $2;
 					$$ = (Node *) n;
 				}
 			/* unparenthesized VERBOSE kept for pre-14 compatibility */
-			| CLUSTER opt_verbose qualified_name cluster_index_specification
+			| CLUSTER VERBOSE qualified_name cluster_index_specification
 				{
 					ClusterStmt *n = makeNode(ClusterStmt);
 
 					n->relation = $3;
 					n->indexname = $4;
-					n->params = NIL;
-					if ($2)
-						n->params = lappend(n->params, makeDefElem("verbose", NULL, @2));
+					n->params = list_make1(makeDefElem("verbose", NULL, @2));
 					$$ = (Node *) n;
 				}
 			/* unparenthesized VERBOSE kept for pre-17 compatibility */
-			| CLUSTER opt_verbose
+			| CLUSTER VERBOSE
 				{
 					ClusterStmt *n = makeNode(ClusterStmt);
 
 					n->relation = NULL;
 					n->indexname = NULL;
-					n->params = NIL;
-					if ($2)
-						n->params = lappend(n->params, makeDefElem("verbose", NULL, @2));
+					n->params = list_make1(makeDefElem("verbose", NULL, @2));
 					$$ = (Node *) n;
 				}
 			/* kept for pre-8.3 compatibility */
-			| CLUSTER opt_verbose name ON qualified_name
+			| CLUSTER VERBOSE name ON qualified_name
 				{
 					ClusterStmt *n = makeNode(ClusterStmt);
 
 					n->relation = $5;
 					n->indexname = $3;
+					n->params = list_make1(makeDefElem("verbose", NULL, @2));
+					$$ = (Node *) n;
+				}
+			| CLUSTER name ON qualified_name
+				{
+					ClusterStmt *n = makeNode(ClusterStmt);
+
+					n->relation = $4;
+					n->indexname = $2;
 					n->params = NIL;
-					if ($2)
-						n->params = lappend(n->params, makeDefElem("verbose", NULL, @2));
 					$$ = (Node *) n;
 				}
 		;
-- 
2.39.5


--x7gweul467fn6trx--





^ permalink  raw  reply  [nested|flat] 11+ messages in thread

end of thread, other threads:[~2025-07-23 16:13 UTC | newest]

Thread overview: 11+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2022-01-18 23:48 [PATCH 8/8] vcregress: run alltaptests in parallel Justin Pryzby <[email protected]>
2022-02-19 19:41 [PATCH 10/21] vcregress: run alltaptests in parallel Justin Pryzby <[email protected]>
2022-02-19 19:41 [PATCH 10/19] vcregress: run alltaptests in parallel Justin Pryzby <[email protected]>
2022-02-19 19:41 [PATCH 10/21] vcregress: run alltaptests in parallel Justin Pryzby <[email protected]>
2022-02-19 19:41 [PATCH 14/23] vcregress: run alltaptests in parallel Justin Pryzby <[email protected]>
2022-02-19 19:41 [PATCH 10/19] vcregress: run alltaptests in parallel Justin Pryzby <[email protected]>
2022-02-19 19:41 [PATCH 10/19] vcregress: run alltaptests in parallel Justin Pryzby <[email protected]>
2022-02-19 19:41 [PATCH 14/23] vcregress: run alltaptests in parallel Justin Pryzby <[email protected]>
2022-02-19 19:41 [PATCH 16/25] vcregress: run alltaptests in parallel Justin Pryzby <[email protected]>
2022-02-19 19:41 [PATCH 6/7] vcregress: run alltaptests in parallel Justin Pryzby <[email protected]>
2025-07-23 16:13 [PATCH v2] Refactor grammar to create opt_utility_option_list Álvaro Herrera <[email protected]>

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox