agora inbox for pgsql-hackers@postgresql.org
help / color / mirror / Atom feedset TESTDIR from perl rather than Makefile
7+ messages / 3 participants
[nested] [flat]
* set TESTDIR from perl rather than Makefile
@ 2022-02-19 23:41 Justin Pryzby <pryzby@telsasoft.com>
0 siblings, 1 reply; 7+ messages in thread
From: Justin Pryzby @ 2022-02-19 23:41 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Robert Haas <robertmhaas@gmail.com>; Andrew Dunstan <andrew@dunslane.net>; pgsql-hackers@postgresql.org, Thomas Munro <thomas.munro@gmail.com>; Melanie Plageman <melanieplageman@gmail.com>; Peter Eisentraut <peter.eisentraut@enterprisedb.com>; Daniel Gustafsson <daniel@yesql.se>
Forking: <E6CA9D45-96EA-4037-8BEB-344CCA75A38C@anarazel.de>
On Tue, Feb 15, 2022 at 10:42:09PM -0800, Andres Freund wrote:
> >> I was thinking that we should make Utils.pm's INIT block responsible for
> >> figuring out both the directory a test should run in and the log location,
> >> instead having that in vcregress.pl and Makefile.global.in. Mostly because
> >> doing it in the latter means we can't start tests with different TESTDIR and
> >> working dir at the same time.
> >>
> >> If instead we pass the location of the top-level build and top-level source
> >> directory from vcregress.pl / Makefile.global, the tap test infrastructure can
> >> figure out that stuff themselves, on a per-test basis.
> >>
> >> For msvc builds we probably would need to pass in some information that allow
> >> Utils.pm to set up PATH appropriately. I think that might just require knowing
> >> that a) msvc build system is used b) Release vs Debug.
> >
> >I'm totally unsure if this resembles what you're thinking of, and I'm surprised
> >I got it working so easily. But it gets the tap test output in separate dirs,
> >and CI is passing for everyone (windows failed because I injected a "false" to
> >force it to upload artifacts).
> >
> >https://github.com/justinpryzby/postgres/runs/5211673291
>
> Yes, that's along the lines I was thinking. I only checked it on my phone, so it certainly isn't a careful look...
>
> I think this should be discussed in a separate thread, for visibility.
I rebased and fixed the check-guc script to work, made it work with vpath
builds, and cleaned it up some.
There may be other reasons to do this, but the reason I did it is to implement
an alltaptests target for vcregress, for cirrus (and everyone else). If all
the tap tests are run serially, it takes ~16min on cirrus; it takes ~13 to run
in parallel (the below run is slower than that since all the slow tests were
scheduled at once - which isn't always a good idea).
Running the tests in parallel uses a single invocation of prove, and starts all
the TAP tests from the same dir rather than their own dir. But without this
patch, all the tap test are output to the same dir, which is a pain to look
through.
This (and other) patches ran here.
https://github.com/justinpryzby/postgres/runs/5261323874
...
e806bcb280 wip: set TESTDIR from src/test/perl rather than Makefile/vcregress
a1bfa8e1a6 cirrus/windows: increase timout to 20min
5479b44198 vcregress: add alltaptests
fcef696c7d vcregress: run alltaptests in parallel
2d7dba13dd cirrus/windows: prove --state to run tests in order
7edf835d43 tmp: run tap tests first
6fb010c137 cirrus: include hints how to install OS packages..
28a25f12c3 cirrus/windows: add compiler_warnings_script
75bc8cff69 cirrus: upload changed html docs as artifacts
8008af2480 s!build docs as a separate task..
a3bf699a0e vcregress/ci: test modules/contrib with NO_INSTALLCHECK=1
d28ce46c2f wip: cirrus: code coverage
--
Justin
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: set TESTDIR from perl rather than Makefile
@ 2022-02-19 23:53 Justin Pryzby <pryzby@telsasoft.com>
parent: Justin Pryzby <pryzby@telsasoft.com>
0 siblings, 2 replies; 7+ messages in thread
From: Justin Pryzby @ 2022-02-19 23:53 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Robert Haas <robertmhaas@gmail.com>; Andrew Dunstan <andrew@dunslane.net>; pgsql-hackers@postgresql.org, Thomas Munro <thomas.munro@gmail.com>; Melanie Plageman <melanieplageman@gmail.com>; Peter Eisentraut <peter.eisentraut@enterprisedb.com>; Daniel Gustafsson <daniel@yesql.se>
On Sat, Feb 19, 2022 at 05:41:49PM -0600, Justin Pryzby wrote:
> I rebased and fixed the check-guc script to work, made it work with vpath
> builds, and cleaned it up some.
I also meant to also attach it.
> This (and other) patches ran here.
> https://github.com/justinpryzby/postgres/runs/5261323874
> ...
> e806bcb280 wip: set TESTDIR from src/test/perl rather than Makefile/vcregress
Attachments:
[text/x-diff] 0001-wip-set-TESTDIR-from-src-test-perl-rather-than-Makef.patch (5.1K, ../../20220219235309.GA27807@telsasoft.com/2-0001-wip-set-TESTDIR-from-src-test-perl-rather-than-Makef.patch)
download | inline diff:
From e806bcb280f1093c0a0e71c9a0b5617f938c4b86 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 19 Feb 2022 13:06:52 -0600
Subject: [PATCH] wip: set TESTDIR from src/test/perl rather than
Makefile/vcregress
---
src/Makefile.global.in | 6 +++---
.../libpq_pipeline/t/001_libpq_pipeline.pl | 3 ++-
src/test/modules/test_misc/t/003_check_guc.pl | 2 +-
src/test/perl/PostgreSQL/Test/Utils.pm | 16 +++++++++-------
src/tools/msvc/vcregress.pl | 4 +---
5 files changed, 16 insertions(+), 15 deletions(-)
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index c980444233..ff20387c42 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -451,7 +451,7 @@ define prove_installcheck
rm -rf '$(CURDIR)'/tmp_check
$(MKDIR_P) '$(CURDIR)'/tmp_check
cd $(srcdir) && \
- TESTDIR='$(CURDIR)' PATH="$(bindir):$(CURDIR):$$PATH" \
+ PATH="$(bindir):$(CURDIR):$$PATH" \
PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' \
PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' \
$(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
@@ -461,7 +461,7 @@ define prove_installcheck
rm -rf '$(CURDIR)'/tmp_check
$(MKDIR_P) '$(CURDIR)'/tmp_check
cd $(srcdir) && \
- TESTDIR='$(CURDIR)' PATH="$(bindir):$(CURDIR):$$PATH" \
+ PATH="$(bindir):$(CURDIR):$$PATH" \
PGPORT='6$(DEF_PGPORT)' top_builddir='$(top_builddir)' \
PG_REGRESS='$(top_builddir)/src/test/regress/pg_regress' \
$(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
@@ -472,7 +472,7 @@ define prove_check
rm -rf '$(CURDIR)'/tmp_check
$(MKDIR_P) '$(CURDIR)'/tmp_check
cd $(srcdir) && \
- TESTDIR='$(CURDIR)' $(with_temp_install) PGPORT='6$(DEF_PGPORT)' \
+ $(with_temp_install) PGPORT='6$(DEF_PGPORT)' \
PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' \
$(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
endef
diff --git a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
index 0c164dcaba..2a0eca7744 100644
--- a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
+++ b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
@@ -49,7 +49,8 @@ for my $testname (@tests)
my $expected;
my $result;
- $expected = slurp_file_eval("traces/$testname.trace");
+ my $inputdir = "$ENV{'TESTDIR'}/tmp_check";
+ $expected = slurp_file_eval("$inputdir/traces/$testname.trace");
next unless $expected ne "";
$result = slurp_file_eval($traceout);
next unless $result ne "";
diff --git a/src/test/modules/test_misc/t/003_check_guc.pl b/src/test/modules/test_misc/t/003_check_guc.pl
index 60459ef759..f4c1636240 100644
--- a/src/test/modules/test_misc/t/003_check_guc.pl
+++ b/src/test/modules/test_misc/t/003_check_guc.pl
@@ -35,7 +35,7 @@ my @not_in_sample_array = split("\n", lc($not_in_sample));
# TAP tests are executed in the directory of the test, in the source tree,
# even for VPATH builds, so rely on that to find postgresql.conf.sample.
-my $rootdir = "../../../..";
+my $rootdir = "$ENV{TESTDIR}/../../../../..";
my $sample_file = "$rootdir/src/backend/utils/misc/postgresql.conf.sample";
# List of all the GUCs found in the sample file.
diff --git a/src/test/perl/PostgreSQL/Test/Utils.pm b/src/test/perl/PostgreSQL/Test/Utils.pm
index 31e2b0315e..8a8d95ca8c 100644
--- a/src/test/perl/PostgreSQL/Test/Utils.pm
+++ b/src/test/perl/PostgreSQL/Test/Utils.pm
@@ -184,19 +184,21 @@ INIT
# test may still fail, but it's more likely to report useful facts.
$SIG{PIPE} = 'IGNORE';
- # Determine output directories, and create them. The base path is the
- # TESTDIR environment variable, which is normally set by the invoking
- # Makefile.
- $tmp_check = $ENV{TESTDIR} ? "$ENV{TESTDIR}/tmp_check" : "tmp_check";
+ my $test_dir = File::Spec->rel2abs(dirname($0));
+ my $test_name = basename($0);
+ $test_name =~ s/\.[^.]+$//;
+
+ # Determine output directories, and create them.
+ # TODO: set srcdir?
+ $tmp_check = "$test_dir/tmp_check";
$log_path = "$tmp_check/log";
+ $ENV{TESTDIR} = $test_dir;
mkdir $tmp_check;
mkdir $log_path;
# Open the test log file, whose name depends on the test name.
- $test_logfile = basename($0);
- $test_logfile =~ s/\.[^.]+$//;
- $test_logfile = "$log_path/regress_log_$test_logfile";
+ $test_logfile = "$log_path/regress_log_$test_name";
open my $testlog, '>', $test_logfile
or die "could not open STDOUT to logfile \"$test_logfile\": $!";
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index e2b0db0879..63085506e0 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -261,10 +261,8 @@ sub tap_check
$ENV{PG_REGRESS} = "$topdir/$Config/pg_regress/pg_regress";
$ENV{REGRESS_SHLIB} = "$topdir/src/test/regress/regress.dll";
- $ENV{TESTDIR} = "$dir";
my $module = basename $dir;
- # add the module build dir as the second element in the PATH
- $ENV{PATH} =~ s!;!;$topdir/$Config/$module;!;
+ #$ENV{VCREGRESS_MODE} = $Config;
rmtree('tmp_check');
system(@args);
--
2.17.1
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: set TESTDIR from perl rather than Makefile
@ 2022-02-21 00:39 Andres Freund <andres@anarazel.de>
parent: Justin Pryzby <pryzby@telsasoft.com>
1 sibling, 1 reply; 7+ messages in thread
From: Andres Freund @ 2022-02-21 00:39 UTC (permalink / raw)
To: Justin Pryzby <pryzby@telsasoft.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Robert Haas <robertmhaas@gmail.com>; Andrew Dunstan <andrew@dunslane.net>; pgsql-hackers@postgresql.org, Thomas Munro <thomas.munro@gmail.com>; Melanie Plageman <melanieplageman@gmail.com>; Peter Eisentraut <peter.eisentraut@enterprisedb.com>; Daniel Gustafsson <daniel@yesql.se>
Hi,
On 2022-02-19 17:53:09 -0600, Justin Pryzby wrote:
> I also meant to also attach it.
Is the patch actually independent of the other patches in your stack?
I like this concept a lot:
- I've had to use a wrapper around individual tap tests for meson, just to set
the CWD etc.
- Being able to run all tap tests at once, instead of many separate prove
invocations is a lot more readable. And can be faster.
- makes it easier to invoke tap tests "manually", which can be useful when
debugging problems (not fun to run make in valgrind or rr)
- I'd like to put test data and test log files in different places than they
are eventually. This seems like it gets us a tiny bit closer to that.
> - $expected = slurp_file_eval("traces/$testname.trace");
> + my $inputdir = "$ENV{'TESTDIR'}/tmp_check";
> + $expected = slurp_file_eval("$inputdir/traces/$testname.trace");
Why is this needed? Shouldn't we end up in exactly the same dir with/without
this patch?
> diff --git a/src/test/perl/PostgreSQL/Test/Utils.pm b/src/test/perl/PostgreSQL/Test/Utils.pm
> index 31e2b0315e..8a8d95ca8c 100644
> --- a/src/test/perl/PostgreSQL/Test/Utils.pm
> +++ b/src/test/perl/PostgreSQL/Test/Utils.pm
> @@ -184,19 +184,21 @@ INIT
> # test may still fail, but it's more likely to report useful facts.
> $SIG{PIPE} = 'IGNORE';
>
> - # Determine output directories, and create them. The base path is the
> - # TESTDIR environment variable, which is normally set by the invoking
> - # Makefile.
> - $tmp_check = $ENV{TESTDIR} ? "$ENV{TESTDIR}/tmp_check" : "tmp_check";
> + my $test_dir = File::Spec->rel2abs(dirname($0));
> + my $test_name = basename($0);
> + $test_name =~ s/\.[^.]+$//;
> +
> + # Determine output directories, and create them.
> + # TODO: set srcdir?
> + $tmp_check = "$test_dir/tmp_check";
> $log_path = "$tmp_check/log";
> + $ENV{TESTDIR} = $test_dir;
>
> mkdir $tmp_check;
> mkdir $log_path;
>
> # Open the test log file, whose name depends on the test name.
> - $test_logfile = basename($0);
> - $test_logfile =~ s/\.[^.]+$//;
> - $test_logfile = "$log_path/regress_log_$test_logfile";
> + $test_logfile = "$log_path/regress_log_$test_name";
> open my $testlog, '>', $test_logfile
> or die "could not open STDOUT to logfile \"$test_logfile\": $!";
>
> diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
> index e2b0db0879..63085506e0 100644
> --- a/src/tools/msvc/vcregress.pl
> +++ b/src/tools/msvc/vcregress.pl
> @@ -261,10 +261,8 @@ sub tap_check
> $ENV{PG_REGRESS} = "$topdir/$Config/pg_regress/pg_regress";
> $ENV{REGRESS_SHLIB} = "$topdir/src/test/regress/regress.dll";
>
> - $ENV{TESTDIR} = "$dir";
> my $module = basename $dir;
> - # add the module build dir as the second element in the PATH
> - $ENV{PATH} =~ s!;!;$topdir/$Config/$module;!;
> + #$ENV{VCREGRESS_MODE} = $Config;
Hm. How does the module build dir end up on PATH after this?
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: set TESTDIR from perl rather than Makefile
@ 2022-02-21 12:00 Andrew Dunstan <andrew@dunslane.net>
parent: Justin Pryzby <pryzby@telsasoft.com>
1 sibling, 1 reply; 7+ messages in thread
From: Andrew Dunstan @ 2022-02-21 12:00 UTC (permalink / raw)
To: Justin Pryzby <pryzby@telsasoft.com>; Andres Freund <andres@anarazel.de>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers@postgresql.org, Thomas Munro <thomas.munro@gmail.com>; Melanie Plageman <melanieplageman@gmail.com>; Peter Eisentraut <peter.eisentraut@enterprisedb.com>; Daniel Gustafsson <daniel@yesql.se>
On 2/19/22 18:53, Justin Pryzby wrote:
> On Sat, Feb 19, 2022 at 05:41:49PM -0600, Justin Pryzby wrote:
>> I rebased and fixed the check-guc script to work, made it work with vpath
>> builds, and cleaned it up some.
> I also meant to also attach it.
This is going to break a bunch of stuff as written.
First, it's not doing the same thing. The current system sets TESTDIR to
be the parent of the directory that holds the test. e.g. for
src/bin/pg_ctl/t/001_start_stop.pl it's src/bin/pg_ctl in the build
tree, not the 't' subdirectory. This patch apparently sets it to the 't'
subdirectory. That will break anything that goes looking for log files
in the current location, like the buildfarm client, and possibly some CI
setups as well.
Also, unless I'm mistaken it appears to to the wrong thing for vpath
builds:
my $test_dir = File::Spec->rel2abs(dirname($0));
is completely wrong for vpaths, since that will place it in the source
tree, not the build tree.
Last, and for no explained reason that I can see, the patch undoes
commit f4ce6c4d3a, but only for msvc builds. Even if that's justified it
appears to have no relevance to this patch.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: set TESTDIR from perl rather than Makefile
@ 2022-02-25 01:17 Justin Pryzby <pryzby@telsasoft.com>
parent: Andrew Dunstan <andrew@dunslane.net>
0 siblings, 1 reply; 7+ messages in thread
From: Justin Pryzby @ 2022-02-25 01:17 UTC (permalink / raw)
To: Andrew Dunstan <andrew@dunslane.net>; +Cc: Andres Freund <andres@anarazel.de>; Tom Lane <tgl@sss.pgh.pa.us>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers@postgresql.org, Thomas Munro <thomas.munro@gmail.com>; Melanie Plageman <melanieplageman@gmail.com>; Peter Eisentraut <peter.eisentraut@enterprisedb.com>; Daniel Gustafsson <daniel@yesql.se>
On Mon, Feb 21, 2022 at 07:00:54AM -0500, Andrew Dunstan wrote:
> On 2/19/22 18:53, Justin Pryzby wrote:
> > On Sat, Feb 19, 2022 at 05:41:49PM -0600, Justin Pryzby wrote:
> >> I rebased and fixed the check-guc script to work, made it work with vpath
> >> builds, and cleaned it up some.
> > I also meant to also attach it.
>
> This is going to break a bunch of stuff as written.
>
> First, it's not doing the same thing. The current system sets TESTDIR to
> be the parent of the directory that holds the test. e.g. for
> src/bin/pg_ctl/t/001_start_stop.pl it's src/bin/pg_ctl in the build
> tree, not the 't' subdirectory. This patch apparently sets it to the 't'
> subdirectory. That will break anything that goes looking for log files
> in the current location, like the buildfarm client, and possibly some CI
> setups as well.
Yes, thanks.
I changed the patch to use ENV{CURDIR} || dirname(dirname($0)). If I'm not
wrong, that seems to be doing the right thing.
> Also, unless I'm mistaken it appears to to the wrong thing for vpath
> builds:
>
> my $test_dir = File::Spec->rel2abs(dirname($0));
>
> is completely wrong for vpaths, since that will place it in the source
> tree, not the build tree.
>
> Last, and for no explained reason that I can see, the patch undoes
> commit f4ce6c4d3a, but only for msvc builds. Even if that's justified it
> appears to have no relevance to this patch.
Andres' idea is that perl should set TESTDIR and PATH. Here I commented out
PATH, and had the improbable issue that nothing seemed to be breaking,
including the pipeline test under msvc. It'd be helpful to know what
configuration that breaks so I can test that it's broken and then test that
it's fixed when set from within perl...
I got busy here, and may not be able to progress this for awhile.
Attachments:
[text/x-diff] 0001-wip-set-TESTDIR-from-src-test-perl-rather-than-Makef.patch (4.1K, ../../20220225011704.GA9008@telsasoft.com/2-0001-wip-set-TESTDIR-from-src-test-perl-rather-than-Makef.patch)
download | inline diff:
From b46b405565a2fce3f96a1dcddf6d35ae7f3acc6d Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 19 Feb 2022 13:06:52 -0600
Subject: [PATCH] wip: set TESTDIR from src/test/perl rather than
Makefile/vcregress
These seem most likely to break:
make check -C src/bin/psql
make check -C src/bin/pgbench
make check -C src/test/modules/test_misc
make check -C src/test/modules/libpq_pipeline
PROVE_TESTS=t/027_stream_regress.pl make check -C src/test/recovery
---
src/Makefile.global.in | 9 ++++++---
src/test/perl/PostgreSQL/Test/Utils.pm | 17 ++++++++++-------
src/tools/msvc/vcregress.pl | 4 +---
3 files changed, 17 insertions(+), 13 deletions(-)
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index c980444233..92649d0193 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -451,8 +451,9 @@ define prove_installcheck
rm -rf '$(CURDIR)'/tmp_check
$(MKDIR_P) '$(CURDIR)'/tmp_check
cd $(srcdir) && \
- TESTDIR='$(CURDIR)' PATH="$(bindir):$(CURDIR):$$PATH" \
+ PATH="$(bindir):$(CURDIR):$$PATH" \
PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' \
+ PG_SUBDIR='$(CURDIR)' \
PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' \
$(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
endef
@@ -461,8 +462,9 @@ define prove_installcheck
rm -rf '$(CURDIR)'/tmp_check
$(MKDIR_P) '$(CURDIR)'/tmp_check
cd $(srcdir) && \
- TESTDIR='$(CURDIR)' PATH="$(bindir):$(CURDIR):$$PATH" \
+ PATH="$(bindir):$(CURDIR):$$PATH" \
PGPORT='6$(DEF_PGPORT)' top_builddir='$(top_builddir)' \
+ PG_SUBDIR='$(CURDIR)' \
PG_REGRESS='$(top_builddir)/src/test/regress/pg_regress' \
$(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
endef
@@ -472,7 +474,8 @@ define prove_check
rm -rf '$(CURDIR)'/tmp_check
$(MKDIR_P) '$(CURDIR)'/tmp_check
cd $(srcdir) && \
- TESTDIR='$(CURDIR)' $(with_temp_install) PGPORT='6$(DEF_PGPORT)' \
+ $(with_temp_install) PGPORT='6$(DEF_PGPORT)' \
+ PG_SUBDIR='$(CURDIR)' \
PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' \
$(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
endef
diff --git a/src/test/perl/PostgreSQL/Test/Utils.pm b/src/test/perl/PostgreSQL/Test/Utils.pm
index 46cd746796..74ccaa08d9 100644
--- a/src/test/perl/PostgreSQL/Test/Utils.pm
+++ b/src/test/perl/PostgreSQL/Test/Utils.pm
@@ -184,19 +184,22 @@ INIT
# test may still fail, but it's more likely to report useful facts.
$SIG{PIPE} = 'IGNORE';
- # Determine output directories, and create them. The base path is the
- # TESTDIR environment variable, which is normally set by the invoking
- # Makefile.
- $tmp_check = $ENV{TESTDIR} ? "$ENV{TESTDIR}/tmp_check" : "tmp_check";
+ my $test_dir = File::Spec->rel2abs($ENV{PG_SUBDIR} || dirname(dirname($0)));
+
+ my $test_name = basename($0);
+ $test_name =~ s/\.[^.]+$//;
+
+ # Determine output directories, and create them.
+ # TODO: set srcdir?
+ $tmp_check = "$test_dir/tmp_check";
$log_path = "$tmp_check/log";
+ $ENV{TESTDIR} = $test_dir;
mkdir $tmp_check;
mkdir $log_path;
# Open the test log file, whose name depends on the test name.
- $test_logfile = basename($0);
- $test_logfile =~ s/\.[^.]+$//;
- $test_logfile = "$log_path/regress_log_$test_logfile";
+ $test_logfile = "$log_path/regress_log_$test_name";
open my $testlog, '>', $test_logfile
or die "could not open STDOUT to logfile \"$test_logfile\": $!";
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index e2b0db0879..c1876fbdab 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -261,10 +261,8 @@ sub tap_check
$ENV{PG_REGRESS} = "$topdir/$Config/pg_regress/pg_regress";
$ENV{REGRESS_SHLIB} = "$topdir/src/test/regress/regress.dll";
- $ENV{TESTDIR} = "$dir";
my $module = basename $dir;
- # add the module build dir as the second element in the PATH
- $ENV{PATH} =~ s!;!;$topdir/$Config/$module;!;
+ # $ENV{VCREGRESS_MODE} = $Config;
rmtree('tmp_check');
system(@args);
--
2.17.1
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: set TESTDIR from perl rather than Makefile
@ 2022-02-25 14:02 Andrew Dunstan <andrew@dunslane.net>
parent: Justin Pryzby <pryzby@telsasoft.com>
0 siblings, 0 replies; 7+ messages in thread
From: Andrew Dunstan @ 2022-02-25 14:02 UTC (permalink / raw)
To: Justin Pryzby <pryzby@telsasoft.com>; +Cc: Andres Freund <andres@anarazel.de>; Tom Lane <tgl@sss.pgh.pa.us>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers@postgresql.org, Thomas Munro <thomas.munro@gmail.com>; Melanie Plageman <melanieplageman@gmail.com>; Peter Eisentraut <peter.eisentraut@enterprisedb.com>; Daniel Gustafsson <daniel@yesql.se>
On 2/24/22 20:17, Justin Pryzby wrote:
> On Mon, Feb 21, 2022 at 07:00:54AM -0500, Andrew Dunstan wrote:
>> On 2/19/22 18:53, Justin Pryzby wrote:
>>> On Sat, Feb 19, 2022 at 05:41:49PM -0600, Justin Pryzby wrote:
>>>> I rebased and fixed the check-guc script to work, made it work with vpath
>>>> builds, and cleaned it up some.
>>> I also meant to also attach it.
>> This is going to break a bunch of stuff as written.
>>
>> First, it's not doing the same thing. The current system sets TESTDIR to
>> be the parent of the directory that holds the test. e.g. for
>> src/bin/pg_ctl/t/001_start_stop.pl it's src/bin/pg_ctl in the build
>> tree, not the 't' subdirectory. This patch apparently sets it to the 't'
>> subdirectory. That will break anything that goes looking for log files
>> in the current location, like the buildfarm client, and possibly some CI
>> setups as well.
> Yes, thanks.
>
> I changed the patch to use ENV{CURDIR} || dirname(dirname($0)). If I'm not
> wrong, that seems to be doing the right thing.
>
>> Also, unless I'm mistaken it appears to to the wrong thing for vpath
>> builds:
>>
>> my $test_dir = File::Spec->rel2abs(dirname($0));
>>
>> is completely wrong for vpaths, since that will place it in the source
>> tree, not the build tree.
>>
>> Last, and for no explained reason that I can see, the patch undoes
>> commit f4ce6c4d3a, but only for msvc builds. Even if that's justified it
>> appears to have no relevance to this patch.
> Andres' idea is that perl should set TESTDIR and PATH. Here I commented out
> PATH, and had the improbable issue that nothing seemed to be breaking,
> including the pipeline test under msvc. It'd be helpful to know what
> configuration that breaks so I can test that it's broken and then test that
> it's fixed when set from within perl...
I'm fairly sure what's broken here is the MSVC install procedure, which
is massively overeager. We should fix that rather than take it for granted.
>
> I got busy here, and may not be able to progress this for awhile.
You have fixed the vpath issue. But the changes in vcregress.pl look
wrong to me.
- $ENV{TESTDIR} = "$dir";
If we set PG_SUBDIR in the Makefile shouldn't we set it here too? Yes it
probably doesn't matter as our MSVC build system doesn't support vpath
builds, but it's good to maintain as much equivalence between the two as
possible. (Supporting vpath builds for MSVC might be a nice
student/intern project)
my $module = basename $dir;
- # add the module build dir as the second element in the PATH
- $ENV{PATH} =~ s!;!;$topdir/$Config/$module;!;
See above comment re msvc install. If we're not reverting f4ce6c4d3a in
the Makefile we shouldn't be reverting here either.
+ # $ENV{VCREGRESS_MODE} = $Config;
Why is this commented out line here?
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: set TESTDIR from perl rather than Makefile
@ 2022-02-26 01:10 Justin Pryzby <pryzby@telsasoft.com>
parent: Andres Freund <andres@anarazel.de>
0 siblings, 0 replies; 7+ messages in thread
From: Justin Pryzby @ 2022-02-26 01:10 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Robert Haas <robertmhaas@gmail.com>; Andrew Dunstan <andrew@dunslane.net>; pgsql-hackers@postgresql.org, Thomas Munro <thomas.munro@gmail.com>; Melanie Plageman <melanieplageman@gmail.com>; Peter Eisentraut <peter.eisentraut@enterprisedb.com>; Daniel Gustafsson <daniel@yesql.se>
On Sun, Feb 20, 2022 at 04:39:08PM -0800, Andres Freund wrote:
> On 2022-02-19 17:53:09 -0600, Justin Pryzby wrote:
> > I also meant to also attach it.
>
> Is the patch actually independent of the other patches in your stack?
Yes - I rearranged it that way for this thread.
However, it's best served/combined with the alltaptests patch :)
> > - $expected = slurp_file_eval("traces/$testname.trace");
> > + my $inputdir = "$ENV{'TESTDIR'}/tmp_check";
> > + $expected = slurp_file_eval("$inputdir/traces/$testname.trace");
>
> Why is this needed? Shouldn't we end up in exactly the same dir with/without
> this patch?
Right - I'd incorrectly set test_dir to t/ rather than the parent dir.
> > +++ b/src/tools/msvc/vcregress.pl
> > @@ -261,10 +261,8 @@ sub tap_check
> > $ENV{PG_REGRESS} = "$topdir/$Config/pg_regress/pg_regress";
> > $ENV{REGRESS_SHLIB} = "$topdir/src/test/regress/regress.dll";
> >
> > - $ENV{TESTDIR} = "$dir";
> > my $module = basename $dir;
> > - # add the module build dir as the second element in the PATH
> > - $ENV{PATH} =~ s!;!;$topdir/$Config/$module;!;
> > + #$ENV{VCREGRESS_MODE} = $Config;
>
> Hm. How does the module build dir end up on PATH after this?
That patch only dealt with TESTDIR. PATH was still set by makefiles.
For MSVC, it was being set to top_builddir/tmp_install.
I've added a 2nd patch to also set PATH. Maybe it should also set
PATH=$bindir:$PATH - I'm not sure.
https://github.com/justinpryzby/postgres/runs/5340884168
Attachments:
[text/x-diff] 0001-wip-set-TESTDIR-from-src-test-perl-rather-than-Makef.patch (4.0K, ../../20220226011009.GI9008@telsasoft.com/2-0001-wip-set-TESTDIR-from-src-test-perl-rather-than-Makef.patch)
download | inline diff:
From 9268dc1f10eed605d584ade20e263a6c32439e2e Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 19 Feb 2022 13:06:52 -0600
Subject: [PATCH 1/7] wip: set TESTDIR from src/test/perl rather than
Makefile/vcregress
TODO: PATH
These seem most likely to break:
make check -C src/bin/psql
make check -C src/bin/pgbench
make check -C src/test/modules/test_misc
make check -C src/test/modules/libpq_pipeline
PROVE_TESTS=t/027_stream_regress.pl make check -C src/test/recovery
---
src/Makefile.global.in | 9 ++++++---
src/test/perl/PostgreSQL/Test/Utils.pm | 16 +++++++++-------
src/tools/msvc/vcregress.pl | 1 -
3 files changed, 15 insertions(+), 11 deletions(-)
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index c980444233..92649d0193 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -451,8 +451,9 @@ define prove_installcheck
rm -rf '$(CURDIR)'/tmp_check
$(MKDIR_P) '$(CURDIR)'/tmp_check
cd $(srcdir) && \
- TESTDIR='$(CURDIR)' PATH="$(bindir):$(CURDIR):$$PATH" \
+ PATH="$(bindir):$(CURDIR):$$PATH" \
PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' \
+ PG_SUBDIR='$(CURDIR)' \
PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' \
$(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
endef
@@ -461,8 +462,9 @@ define prove_installcheck
rm -rf '$(CURDIR)'/tmp_check
$(MKDIR_P) '$(CURDIR)'/tmp_check
cd $(srcdir) && \
- TESTDIR='$(CURDIR)' PATH="$(bindir):$(CURDIR):$$PATH" \
+ PATH="$(bindir):$(CURDIR):$$PATH" \
PGPORT='6$(DEF_PGPORT)' top_builddir='$(top_builddir)' \
+ PG_SUBDIR='$(CURDIR)' \
PG_REGRESS='$(top_builddir)/src/test/regress/pg_regress' \
$(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
endef
@@ -472,7 +474,8 @@ define prove_check
rm -rf '$(CURDIR)'/tmp_check
$(MKDIR_P) '$(CURDIR)'/tmp_check
cd $(srcdir) && \
- TESTDIR='$(CURDIR)' $(with_temp_install) PGPORT='6$(DEF_PGPORT)' \
+ $(with_temp_install) PGPORT='6$(DEF_PGPORT)' \
+ PG_SUBDIR='$(CURDIR)' \
PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' \
$(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
endef
diff --git a/src/test/perl/PostgreSQL/Test/Utils.pm b/src/test/perl/PostgreSQL/Test/Utils.pm
index 46cd746796..0e2abb6c9e 100644
--- a/src/test/perl/PostgreSQL/Test/Utils.pm
+++ b/src/test/perl/PostgreSQL/Test/Utils.pm
@@ -184,19 +184,21 @@ INIT
# test may still fail, but it's more likely to report useful facts.
$SIG{PIPE} = 'IGNORE';
- # Determine output directories, and create them. The base path is the
- # TESTDIR environment variable, which is normally set by the invoking
- # Makefile.
- $tmp_check = $ENV{TESTDIR} ? "$ENV{TESTDIR}/tmp_check" : "tmp_check";
+ my $test_dir = File::Spec->rel2abs($ENV{PG_SUBDIR} || dirname(dirname($0)));
+ my $test_name = basename($0);
+ $test_name =~ s/\.[^.]+$//;
+
+ $ENV{TESTDIR} = $test_dir;
+
+ # Determine output directories, and create them.
+ $tmp_check = "$test_dir/tmp_check";
$log_path = "$tmp_check/log";
mkdir $tmp_check;
mkdir $log_path;
# Open the test log file, whose name depends on the test name.
- $test_logfile = basename($0);
- $test_logfile =~ s/\.[^.]+$//;
- $test_logfile = "$log_path/regress_log_$test_logfile";
+ $test_logfile = "$log_path/regress_log_$test_name";
open my $testlog, '>', $test_logfile
or die "could not open STDOUT to logfile \"$test_logfile\": $!";
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index e2b0db0879..e805a9f8f6 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -261,7 +261,6 @@ sub tap_check
$ENV{PG_REGRESS} = "$topdir/$Config/pg_regress/pg_regress";
$ENV{REGRESS_SHLIB} = "$topdir/src/test/regress/regress.dll";
- $ENV{TESTDIR} = "$dir";
my $module = basename $dir;
# add the module build dir as the second element in the PATH
$ENV{PATH} =~ s!;!;$topdir/$Config/$module;!;
--
2.17.1
[text/x-diff] 0002-s-also-remove-PATH.patch (2.9K, ../../20220226011009.GI9008@telsasoft.com/3-0002-s-also-remove-PATH.patch)
download | inline diff:
From 06c8a2458fbbf382011928190b576db1918c853f Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 25 Feb 2022 17:23:26 -0600
Subject: [PATCH 2/7] s!also remove PATH.
This essentially reverts commit f4ce6c4d3a30ec3a12c7f64b90a6fc82887ddd7b
XXX: also set PATH=bindir ?
---
src/Makefile.global.in | 6 +++---
src/test/perl/PostgreSQL/Test/Utils.pm | 10 ++++++++++
src/tools/msvc/vcregress.pl | 4 ----
3 files changed, 13 insertions(+), 7 deletions(-)
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 92649d0193..c0e5661338 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -440,7 +440,7 @@ ld_library_path_var = LD_LIBRARY_PATH
# need something more here. If not defined then the expansion does
# nothing.
with_temp_install = \
- PATH="$(abs_top_builddir)/tmp_install$(bindir):$(CURDIR):$$PATH" \
+ PATH="$(abs_top_builddir)/tmp_install$(bindir):$$PATH" \
$(call add_to_path,$(strip $(ld_library_path_var)),$(abs_top_builddir)/tmp_install$(libdir)) \
$(with_temp_install_extra)
@@ -451,7 +451,7 @@ define prove_installcheck
rm -rf '$(CURDIR)'/tmp_check
$(MKDIR_P) '$(CURDIR)'/tmp_check
cd $(srcdir) && \
- PATH="$(bindir):$(CURDIR):$$PATH" \
+ PATH="$(bindir):$$PATH" \
PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' \
PG_SUBDIR='$(CURDIR)' \
PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' \
@@ -462,7 +462,7 @@ define prove_installcheck
rm -rf '$(CURDIR)'/tmp_check
$(MKDIR_P) '$(CURDIR)'/tmp_check
cd $(srcdir) && \
- PATH="$(bindir):$(CURDIR):$$PATH" \
+ PATH="$(bindir):$$PATH" \
PGPORT='6$(DEF_PGPORT)' top_builddir='$(top_builddir)' \
PG_SUBDIR='$(CURDIR)' \
PG_REGRESS='$(top_builddir)/src/test/regress/pg_regress' \
diff --git a/src/test/perl/PostgreSQL/Test/Utils.pm b/src/test/perl/PostgreSQL/Test/Utils.pm
index 0e2abb6c9e..41c373fadd 100644
--- a/src/test/perl/PostgreSQL/Test/Utils.pm
+++ b/src/test/perl/PostgreSQL/Test/Utils.pm
@@ -190,6 +190,16 @@ INIT
$ENV{TESTDIR} = $test_dir;
+ if ($PostgreSQL::Test::Utils::windows_os &&
+ $Config{osname} eq 'MSWin32')
+ {
+ $ENV{PATH} =~ s!;!;$test_dir;!;
+ }
+ else
+ {
+ $ENV{PATH} =~ s!:!:$test_dir:!;
+ }
+
# Determine output directories, and create them.
$tmp_check = "$test_dir/tmp_check";
$log_path = "$tmp_check/log";
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index e805a9f8f6..74aa572d2f 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -261,10 +261,6 @@ sub tap_check
$ENV{PG_REGRESS} = "$topdir/$Config/pg_regress/pg_regress";
$ENV{REGRESS_SHLIB} = "$topdir/src/test/regress/regress.dll";
- my $module = basename $dir;
- # add the module build dir as the second element in the PATH
- $ENV{PATH} =~ s!;!;$topdir/$Config/$module;!;
-
rmtree('tmp_check');
system(@args);
my $status = $? >> 8;
--
2.17.1
[text/x-diff] 0003-cirrus-windows-increase-timeout-to-20min.patch (1.1K, ../../20220226011009.GI9008@telsasoft.com/4-0003-cirrus-windows-increase-timeout-to-20min.patch)
download | inline diff:
From a52ac4fd5fd2e75fc13656c1fc1b4dd8ab1d7acd Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sun, 13 Feb 2022 17:56:40 -0600
Subject: [PATCH 3/7] cirrus/windows: increase timeout to 20min
---
.cirrus.yml | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index d10b0a82f9..634c0b4933 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -366,9 +366,9 @@ task:
# If tests hang forever, cirrus eventually times out. In that case log
# output etc is not uploaded, making the problem hard to debug. Of course
# tests internally should have shorter timeouts, but that's proven to not
- # be sufficient. 15min currently is fast enough to finish individual test
+ # be sufficient. 20min currently is fast enough to finish individual test
# "suites".
- T_C: "\"C:/Program Files/Git/usr/bin/timeout.exe\" -v -k60s 15m"
+ T_C: "\"C:/Program Files/Git/usr/bin/timeout.exe\" -v -k60s 20m"
only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*windows.*'
--
2.17.1
[text/x-diff] 0004-vcregress-add-alltaptests.patch (4.2K, ../../20220226011009.GI9008@telsasoft.com/5-0004-vcregress-add-alltaptests.patch)
download | inline diff:
From d2ebfa605d86e41e47fa416ca4e362a3cb37634e Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 25 Feb 2022 17:00:33 -0600
Subject: [PATCH 4/7] vcregress: add alltaptests
In passing, document taptest PROVE_FLAGS
ci-os-only: windows
See also:
d835dd6685246f0737ca42ab68242210681bb220
13d856e177e69083f543d6383eeda9e12ce3c55c
fed6df486dca1b9e53d3f560031b9a236c99f4bb
---
.cirrus.yml | 10 ++-------
src/tools/msvc/vcregress.pl | 43 +++++++++++++++++++++++++++++++++----
2 files changed, 41 insertions(+), 12 deletions(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index 634c0b4933..bc4ae9fc28 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -417,14 +417,8 @@ task:
test_ssl_script: |
set with_ssl=openssl
%T_C% perl src/tools/msvc/vcregress.pl taptest ./src/test/ssl/
- test_subscription_script: |
- %T_C% perl src/tools/msvc/vcregress.pl taptest ./src/test/subscription/
- test_authentication_script: |
- %T_C% perl src/tools/msvc/vcregress.pl taptest ./src/test/authentication/
- test_recovery_script: |
- %T_C% perl src/tools/msvc/vcregress.pl recoverycheck
- test_bin_script: |
- %T_C% perl src/tools/msvc/vcregress.pl bincheck
+ test_tap_script: |
+ %T_C% perl src/tools/msvc/vcregress.pl alltaptests
test_pg_upgrade_script: |
%T_C% perl src/tools/msvc/vcregress.pl upgradecheck
test_ecpg_script: |
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index 74aa572d2f..e27c1d15e5 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -49,7 +49,7 @@ if (-e "src/tools/msvc/buildenv.pl")
my $what = shift || "";
if ($what =~
- /^(check|installcheck|plcheck|contribcheck|modulescheck|ecpgcheck|isolationcheck|upgradecheck|bincheck|recoverycheck|taptest)$/i
+ /^(check|installcheck|plcheck|contribcheck|modulescheck|ecpgcheck|isolationcheck|upgradecheck|bincheck|recoverycheck|taptest|alltaptests)$/i
)
{
$what = uc $what;
@@ -107,6 +107,7 @@ my %command = (
BINCHECK => \&bincheck,
RECOVERYCHECK => \&recoverycheck,
UPGRADECHECK => \&upgradecheck,
+ ALLTAPTESTS => \&alltaptests,
TAPTEST => \&taptest,);
my $proc = $command{$what};
@@ -261,6 +262,9 @@ sub tap_check
$ENV{PG_REGRESS} = "$topdir/$Config/pg_regress/pg_regress";
$ENV{REGRESS_SHLIB} = "$topdir/src/test/regress/regress.dll";
+ print "============================================================\n";
+ print "Checking @args\n";
+
rmtree('tmp_check');
system(@args);
my $status = $? >> 8;
@@ -273,8 +277,7 @@ sub bincheck
my $mstat = 0;
- # Find out all the existing TAP tests by looking for t/ directories
- # in the tree.
+ # Find the TAP tests by looking for t/ directories
my @bin_dirs = glob("$topdir/src/bin/*");
# Process each test
@@ -288,6 +291,36 @@ sub bincheck
return;
}
+sub alltaptests
+{
+ InstallTemp();
+
+ my $mstat = 0;
+
+ # Find out all the existing TAP tests by looking for t/ directories
+ # in the tree.
+ my @tap_dirs = ();
+ my @top_dir = ($topdir);
+ File::Find::find(
+ { wanted => sub {
+ /^t\z/s
+ && $File::Find::name !~ /\/(kerberos|ldap|ssl|ssl_passphrase_callback)\// # opt-in
+ && push(@tap_dirs, $File::Find::name);
+ }
+ },
+ @top_dir);
+
+ # Process each test
+ foreach my $test_path (@tap_dirs)
+ {
+ my $dir = dirname($test_path);
+ my $status = tap_check($dir);
+ $mstat ||= $status;
+ }
+ exit $mstat if $mstat;
+ return;
+}
+
sub taptest
{
my $dir = shift;
@@ -777,6 +810,7 @@ sub usage
print STDERR
"Usage: vcregress.pl <mode> [<arg>]\n\n",
"Options for <mode>:\n",
+ " alltaptests run all tap tests (except kerberos, ldap, ssl, ssl_passphrase_callback)\n",
" bincheck run tests of utilities in src/bin/\n",
" check deploy instance and run regression tests on it\n",
" contribcheck run tests of modules in contrib/\n",
@@ -792,6 +826,7 @@ sub usage
" serial serial mode\n",
" parallel parallel mode\n",
"\nOption for <arg>: for taptest\n",
- " TEST_DIR (required) directory where tests reside\n";
+ " TEST_DIR (required) directory where tests reside\n",
+ " PROVE_FLAGS flags to pass to prove\n";
exit(1);
}
--
2.17.1
[text/x-diff] 0005-cirrus-include-hints-how-to-install-OS-packages.patch (2.2K, ../../20220226011009.GI9008@telsasoft.com/6-0005-cirrus-include-hints-how-to-install-OS-packages.patch)
download | inline diff:
From 6d46888521e470ba07929b73ea96c37185088774 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Mon, 17 Jan 2022 00:53:04 -0600
Subject: [PATCH 5/7] cirrus: include hints how to install OS packages..
This is useful for patches during development, but once a feature is merged,
new libraries should be added to the OS image files, rather than installed
during every CI run forever into the future.
---
.cirrus.yml | 16 +++++++++++++---
1 file changed, 13 insertions(+), 3 deletions(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index bc4ae9fc28..ded4c7d27f 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -73,10 +73,11 @@ task:
chown -R postgres:postgres .
mkdir -p ${CCACHE_DIR}
chown -R postgres:postgres ${CCACHE_DIR}
- setup_cores_script: |
+ setup_os_script: |
mkdir -m 770 /tmp/cores
chown root:postgres /tmp/cores
sysctl kern.corefile='/tmp/cores/%N.%P.core'
+ #pkg install -y ...
# NB: Intentionally build without --with-llvm. The freebsd image size is
# already large enough to make VM startup slow, and even without llvm
@@ -180,10 +181,12 @@ task:
chown -R postgres:postgres ${CCACHE_DIR}
echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf
su postgres -c "ulimit -l -H && ulimit -l -S"
- setup_cores_script: |
+ setup_os_script: |
mkdir -m 770 /tmp/cores
chown root:postgres /tmp/cores
sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core'
+ #apt-get update
+ #apt-get -y install ...
configure_script: |
su postgres <<-EOF
@@ -237,7 +240,7 @@ task:
ulimit -a -H && ulimit -a -S
export
- setup_cores_script:
+ setup_os_script:
- mkdir ${HOME}/cores
- sudo sysctl kern.corefile="${HOME}/cores/core.%P"
@@ -384,6 +387,9 @@ task:
powershell -Command get-psdrive -psprovider filesystem
set
+ setup_os_script: |
+ REM choco install -y ...
+
configure_script:
# copy errors out when using forward slashes
- copy src\tools\ci\windows_build_config.pl src\tools\msvc\config.pl
@@ -473,6 +479,10 @@ task:
ccache_cache:
folder: $CCACHE_DIR
+ setup_os_script: |
+ #apt-get update
+ #apt-get -y install ...
+
###
# Test that code can be built with gcc/clang without warnings
###
--
2.17.1
[text/x-diff] 0006-vcregress-run-alltaptests-in-parallel.patch (5.2K, ../../20220226011009.GI9008@telsasoft.com/7-0006-vcregress-run-alltaptests-in-parallel.patch)
download | inline diff:
From d0743f0377b0b90891d244f5f0c76f6af90e2cc3 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 19 Feb 2022 13:41:36 -0600
Subject: [PATCH 6/7] vcregress: run alltaptests in parallel
ci-os-only: windows
The test changes are needed to avoid these failures:
https://github.com/justinpryzby/postgres/runs/5174636590
[15:59:59.408] Bailout called. Further testing stopped: could not create test directory "c:/cirrus/tmp_check/t_C:\cirrus\src\bin\pgbench\t\002_pgbench_no_server_stuff": Invalid argument
[15:59:59.408] FAILED--Further testing stopped: could not create test directory "c:/cirrus/tmp_check/t_C:\cirrus\src\bin\pgbench\t\002_pgbench_no_server_stuff": Invalid argument
https://github.com/justinpryzby/postgres/runs/5174788205
[16:37:09.891] # Failed test 'reading traces/disallowed_in_pipeline.trace: could not open "traces/disallowed_in_pipeline.trace": The system cannot find the path specified at C:\cirrus\src\test\modules\libpq_pipeline\t\001_libpq_pipeline.pl line 70.
https://github.com/justinpryzby/postgres/runs/5174877506
pg_regress: could not open file "../regress/parallel_schedule" for reading: No such file or directory
XXX: avoid breaking src/bin/psql/t/010_tab_completion.pl:
Should pass the srcdir to all taptests, or go back to writing the results to
tmp_check ?
See also:
f4ce6c4d3a30ec3a12c7f64b90a6fc82887ddd7b
795862c280c5949bafcd8c44543d887fd32b590a
db973ffb3ca43e65a0bf15175a35184a53bf977d
---
src/bin/pgbench/t/002_pgbench_no_server.pl | 4 ++--
.../modules/libpq_pipeline/t/001_libpq_pipeline.pl | 3 ++-
src/test/modules/test_misc/t/003_check_guc.pl | 2 +-
src/test/recovery/t/027_stream_regress.pl | 4 ++--
src/tools/msvc/vcregress.pl | 14 ++++----------
5 files changed, 11 insertions(+), 16 deletions(-)
diff --git a/src/bin/pgbench/t/002_pgbench_no_server.pl b/src/bin/pgbench/t/002_pgbench_no_server.pl
index acad19edd0..84e36dcd3f 100644
--- a/src/bin/pgbench/t/002_pgbench_no_server.pl
+++ b/src/bin/pgbench/t/002_pgbench_no_server.pl
@@ -8,12 +8,12 @@
use strict;
use warnings;
+use File::Basename;
use PostgreSQL::Test::Utils;
use Test::More;
# create a directory for scripts
-my $testname = $0;
-$testname =~ s,.*/,,;
+my $testname = basename($0);
$testname =~ s/\.pl$//;
my $testdir = "$PostgreSQL::Test::Utils::tmp_check/t_${testname}_stuff";
diff --git a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
index 0c164dcaba..2a0eca7744 100644
--- a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
+++ b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
@@ -49,7 +49,8 @@ for my $testname (@tests)
my $expected;
my $result;
- $expected = slurp_file_eval("traces/$testname.trace");
+ my $inputdir = "$ENV{'TESTDIR'}/tmp_check";
+ $expected = slurp_file_eval("$inputdir/traces/$testname.trace");
next unless $expected ne "";
$result = slurp_file_eval($traceout);
next unless $result ne "";
diff --git a/src/test/modules/test_misc/t/003_check_guc.pl b/src/test/modules/test_misc/t/003_check_guc.pl
index 60459ef759..167b6ea44b 100644
--- a/src/test/modules/test_misc/t/003_check_guc.pl
+++ b/src/test/modules/test_misc/t/003_check_guc.pl
@@ -35,7 +35,7 @@ my @not_in_sample_array = split("\n", lc($not_in_sample));
# TAP tests are executed in the directory of the test, in the source tree,
# even for VPATH builds, so rely on that to find postgresql.conf.sample.
-my $rootdir = "../../../..";
+my $rootdir = "$ENV{TESTDIR}/../../../..";
my $sample_file = "$rootdir/src/backend/utils/misc/postgresql.conf.sample";
# List of all the GUCs found in the sample file.
diff --git a/src/test/recovery/t/027_stream_regress.pl b/src/test/recovery/t/027_stream_regress.pl
index c40951b7ba..f7679306f5 100644
--- a/src/test/recovery/t/027_stream_regress.pl
+++ b/src/test/recovery/t/027_stream_regress.pl
@@ -58,9 +58,9 @@ system_or_bail($ENV{PG_REGRESS} . " $extra_opts " .
"--bindir= " .
"--host=" . $node_primary->host . " " .
"--port=" . $node_primary->port . " " .
- "--schedule=../regress/parallel_schedule " .
+ "--schedule=\"$dlpath/parallel_schedule\" " .
"--max-concurrent-tests=20 " .
- "--inputdir=../regress " .
+ "--inputdir=\"$dlpath\" " .
"--outputdir=\"$outputdir\"");
# Clobber all sequences with their next value, so that we don't have
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index e27c1d15e5..cc8d690c29 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -295,8 +295,6 @@ sub alltaptests
{
InstallTemp();
- my $mstat = 0;
-
# Find out all the existing TAP tests by looking for t/ directories
# in the tree.
my @tap_dirs = ();
@@ -310,14 +308,10 @@ sub alltaptests
},
@top_dir);
- # Process each test
- foreach my $test_path (@tap_dirs)
- {
- my $dir = dirname($test_path);
- my $status = tap_check($dir);
- $mstat ||= $status;
- }
- exit $mstat if $mstat;
+ # Run all the tap tests in a single prove instance for good performance
+ $ENV{PROVE_TESTS} = "@tap_dirs";
+ my $status = tap_check('PROVE_FLAGS=--ext=.pl', "$topdir");
+ exit $status if $status;
return;
}
--
2.17.1
[text/x-diff] 0007-tmp-run-tap-tests-first.patch (1.1K, ../../20220226011009.GI9008@telsasoft.com/8-0007-tmp-run-tap-tests-first.patch)
download | inline diff:
From 7e62e81f20b43fdf0252b87197ea46ce24289af0 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 25 Feb 2022 14:33:26 -0600
Subject: [PATCH 7/7] tmp: run tap tests first
ci-os-only: windows
---
.cirrus.yml | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index ded4c7d27f..9152cc4b90 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -402,6 +402,9 @@ task:
# Installation on windows currently only completely works from src/tools/msvc
- cd src/tools/msvc && perl install.pl %CIRRUS_WORKING_DIR%/tmp_install
+ test_tap_script: |
+ %T_C% perl src/tools/msvc/vcregress.pl alltaptests
+
test_regress_parallel_script: |
%T_C% perl src/tools/msvc/vcregress.pl check parallel
startcreate_script: |
@@ -423,8 +426,6 @@ task:
test_ssl_script: |
set with_ssl=openssl
%T_C% perl src/tools/msvc/vcregress.pl taptest ./src/test/ssl/
- test_tap_script: |
- %T_C% perl src/tools/msvc/vcregress.pl alltaptests
test_pg_upgrade_script: |
%T_C% perl src/tools/msvc/vcregress.pl upgradecheck
test_ecpg_script: |
--
2.17.1
^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2022-02-26 01:10 UTC | newest]
Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2022-02-19 23:41 set TESTDIR from perl rather than Makefile Justin Pryzby <pryzby@telsasoft.com>
2022-02-19 23:53 ` Justin Pryzby <pryzby@telsasoft.com>
2022-02-21 00:39 ` Andres Freund <andres@anarazel.de>
2022-02-26 01:10 ` Justin Pryzby <pryzby@telsasoft.com>
2022-02-21 12:00 ` Andrew Dunstan <andrew@dunslane.net>
2022-02-25 01:17 ` Justin Pryzby <pryzby@telsasoft.com>
2022-02-25 14:02 ` Andrew Dunstan <andrew@dunslane.net>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox