Date: Fri, 25 Feb 2022 20:51:16 -0600
From: Justin Pryzby <pryzby@telsasoft.com>
To: Andres Freund <andres@anarazel.de>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Robert Haas <robertmhaas@gmail.com>,
	Andrew Dunstan <andrew@dunslane.net>, pgsql-hackers@postgresql.org,
	Thomas Munro <thomas.munro@gmail.com>,
	Melanie Plageman <melanieplageman@gmail.com>,
	Peter Eisentraut <peter.eisentraut@enterprisedb.com>,
	Daniel Gustafsson <daniel@yesql.se>
Subject: Re: Adding CI to our tree
Message-ID: <20220226025116.GJ9008@telsasoft.com>
References: <20220117201619.3ltudwhgk2krmoki@alap3.anarazel.de>
 <20220118210847.GC23027@telsasoft.com>
 <20220203035827.GG23027@telsasoft.com>
 <20220203195718.smqo5xg4ygp5qktq@alap3.anarazel.de>
 <20220204050403.GL23027@telsasoft.com>
 <20220206032339.tuyo534rfbvu4mbh@alap3.anarazel.de>
 <20220212220640.GL31460@telsasoft.com>
 <20220212222625.aph3ft466ntifrpi@alap3.anarazel.de>
 <20220213214213.GS31460@telsasoft.com>
 <20220213220709.vjz5rziuhfdpqxrg@alap3.anarazel.de>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="x1F0m3RQhDZyj8sd"
Content-Disposition: inline
In-Reply-To: <20220213220709.vjz5rziuhfdpqxrg@alap3.anarazel.de>
User-Agent: Mutt/1.9.4 (2018-02-28)
Archived-At: 
 <https://www.postgresql.org/message-id/20220226025116.GJ9008%40telsasoft.com>
Precedence: bulk


--x1F0m3RQhDZyj8sd
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

This is the other half of my CI patches, which are unrelated to the TAP ones on
the other thread.

--x1F0m3RQhDZyj8sd
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="0001-cirrus-include-hints-how-to-install-OS-packages.patch"

From 88c01c09ee26db2817629265fc12b2dbcd8c9a91 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Mon, 17 Jan 2022 00:53:04 -0600
Subject: [PATCH 1/7] cirrus: include hints how to install OS packages..

This is useful for patches during development, but once a feature is merged,
new libraries should be added to the OS image files, rather than installed
during every CI run forever into the future.
---
 .cirrus.yml | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index d10b0a82f9..1b7c36283e 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -73,10 +73,11 @@ task:
     chown -R postgres:postgres .
     mkdir -p ${CCACHE_DIR}
     chown -R postgres:postgres ${CCACHE_DIR}
-  setup_cores_script: |
+  setup_os_script: |
     mkdir -m 770 /tmp/cores
     chown root:postgres /tmp/cores
     sysctl kern.corefile='/tmp/cores/%N.%P.core'
+    #pkg install -y ...
 
   # NB: Intentionally build without --with-llvm. The freebsd image size is
   # already large enough to make VM startup slow, and even without llvm
@@ -180,10 +181,12 @@ task:
     chown -R postgres:postgres ${CCACHE_DIR}
     echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf
     su postgres -c "ulimit -l -H && ulimit -l -S"
-  setup_cores_script: |
+  setup_os_script: |
     mkdir -m 770 /tmp/cores
     chown root:postgres /tmp/cores
     sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core'
+    #apt-get update
+    #apt-get -y install ...
 
   configure_script: |
     su postgres <<-EOF
@@ -237,7 +240,7 @@ task:
     ulimit -a -H && ulimit -a -S
     export
 
-  setup_cores_script:
+  setup_os_script:
     - mkdir ${HOME}/cores
     - sudo sysctl kern.corefile="${HOME}/cores/core.%P"
 
@@ -384,6 +387,9 @@ task:
     powershell -Command get-psdrive -psprovider filesystem
     set
 
+  setup_os_script: |
+    REM choco install -y ...
+
   configure_script:
     # copy errors out when using forward slashes
     - copy src\tools\ci\windows_build_config.pl src\tools\msvc\config.pl
@@ -479,6 +485,10 @@ task:
   ccache_cache:
     folder: $CCACHE_DIR
 
+  setup_os_script: |
+    #apt-get update
+    #apt-get -y install ...
+
   ###
   # Test that code can be built with gcc/clang without warnings
   ###
-- 
2.17.1


--x1F0m3RQhDZyj8sd
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="0002-cirrus-upload-changed-html-docs-as-artifacts.patch"

From 7a80dfccb17f454849679ebe9abc89bd0901828a Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 15 Jan 2022 11:27:28 -0600
Subject: [PATCH 2/7] cirrus: upload changed html docs as artifacts

Always run doc build; to allow them to be shown in cfbot, they should not be
skipped if the linux build fails.

This could be done on the client side (cfbot).  One advantage of doing it here
is that fewer docs are uploaded - many patches won't upload docs at all.

XXX: if this is run in the same task, the configure flags should probably be
consistent ?

https://cirrus-ci.com/task/5396696388599808

ci-os-only: linux
---
 .cirrus.yml | 46 ++++++++++++++++++++++++++++++++++------------
 1 file changed, 34 insertions(+), 12 deletions(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index 1b7c36283e..f21b249b6f 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -450,10 +450,6 @@ task:
 task:
   name: CompilerWarnings
 
-  # To limit unnecessary work only run this once the normal linux test succeeds
-  depends_on:
-    - Linux - Debian Bullseye
-
   env:
     CPUS: 4
     BUILD_JOBS: 4
@@ -482,6 +478,13 @@ task:
     clang -v
     export
 
+    git remote -v
+    git branch -a
+    git remote add postgres https://github.com/postgres/postgres
+    time git fetch -v postgres master
+    git log -1 postgres/master
+    git diff --name-only postgres/master..
+
   ccache_cache:
     folder: $CCACHE_DIR
 
@@ -557,16 +560,35 @@ task:
   ###
   # Verify docs can be built
   ###
-  # XXX: Only do this if there have been changes in doc/ since last build
   always:
     docs_build_script: |
-      time ./configure \
-        --cache gcc.cache \
-        CC="ccache gcc" \
-        CXX="ccache g++" \
-        CLANG="ccache clang"
-      make -s -j${BUILD_JOBS} clean
-      time make -s -j${BUILD_JOBS} -C doc
+      # Do nothing if the patch doesn't update doc:
+      git diff --name-only --cherry-pick --exit-code postgres/master... doc && exit
+
+      # Exercise HTML and other docs:
+      ./configure >/dev/null
+      make -s -C doc
+      cp -r doc new-docs
+
+      # Build HTML docs from the parent commit
+      git checkout postgres/master -- doc
+      make -s -C doc clean
+      make -s -C doc html
+      cp -r doc old-docs
+
+      # Copy HTML which differ into html_docs
+      # This will show any files which differ from files generated from postgres/master.
+      # Commits to postgres/master which affect doc/ will cause more files to be
+      # included until the patch is rebased.
+      mkdir html_docs
+      cp new-docs/src/sgml/html/*.css new-docs/src/sgml/html/*.svg html_docs/
+      for f in `git diff --no-index --name-only old-docs/src/sgml/html new-docs/src/sgml/html`
+      do
+        cp $f html_docs/
+      done
+
+    html_docs_artifacts:
+      paths: ['html_docs/**/*.html', 'html_docs/**/*.png', 'html_docs/**/*.css']
 
   always:
     upload_caches: ccache
-- 
2.17.1


--x1F0m3RQhDZyj8sd
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="0003-s-build-docs-as-a-separate-task.patch"

From a32cb674edd03839da559dfe04b543b4d508045c Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sun, 13 Feb 2022 03:28:00 -0600
Subject: [PATCH 3/7] s!build docs as a separate task..

I believe this'll automatically show up as a separate "column" on the cfbot
page.
---
 .cirrus.yml | 101 +++++++++++++++++++++++++++++++---------------------
 1 file changed, 61 insertions(+), 40 deletions(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index f21b249b6f..73b0baeaa7 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -450,6 +450,10 @@ task:
 task:
   name: CompilerWarnings
 
+  # To limit unnecessary work only run this once the normal linux test succeeds
+  depends_on:
+    - Linux - Debian Bullseye
+
   env:
     CPUS: 4
     BUILD_JOBS: 4
@@ -478,13 +482,6 @@ task:
     clang -v
     export
 
-    git remote -v
-    git branch -a
-    git remote add postgres https://github.com/postgres/postgres
-    time git fetch -v postgres master
-    git log -1 postgres/master
-    git diff --name-only postgres/master..
-
   ccache_cache:
     folder: $CCACHE_DIR
 
@@ -557,38 +554,62 @@ task:
       make -s -j${BUILD_JOBS} clean
       time make -s -j${BUILD_JOBS} world-bin
 
-  ###
-  # Verify docs can be built
-  ###
-  always:
-    docs_build_script: |
-      # Do nothing if the patch doesn't update doc:
-      git diff --name-only --cherry-pick --exit-code postgres/master... doc && exit
-
-      # Exercise HTML and other docs:
-      ./configure >/dev/null
-      make -s -C doc
-      cp -r doc new-docs
-
-      # Build HTML docs from the parent commit
-      git checkout postgres/master -- doc
-      make -s -C doc clean
-      make -s -C doc html
-      cp -r doc old-docs
-
-      # Copy HTML which differ into html_docs
-      # This will show any files which differ from files generated from postgres/master.
-      # Commits to postgres/master which affect doc/ will cause more files to be
-      # included until the patch is rebased.
-      mkdir html_docs
-      cp new-docs/src/sgml/html/*.css new-docs/src/sgml/html/*.svg html_docs/
-      for f in `git diff --no-index --name-only old-docs/src/sgml/html new-docs/src/sgml/html`
-      do
-        cp $f html_docs/
-      done
-
-    html_docs_artifacts:
-      paths: ['html_docs/**/*.html', 'html_docs/**/*.png', 'html_docs/**/*.css']
-
   always:
     upload_caches: ccache
+
+
+# Verify docs can be built, and upload changed docs as artifacts
+task:
+  name: HTML docs
+
+  env:
+    CPUS: 1
+
+  only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*(docs|html).*'
+
+  container:
+    image: $CONTAINER_REPO/linux_debian_bullseye_ci:latest
+    cpu: $CPUS
+
+  sysinfo_script: |
+    id
+    uname -a
+    cat /proc/cmdline
+    ulimit -a -H && ulimit -a -S
+    export
+
+    git remote -v
+    git branch -a
+    git remote add postgres https://github.com/postgres/postgres
+    time git fetch -v postgres master
+    git log -1 postgres/master
+    git diff --name-only postgres/master..
+
+  docs_build_script: |
+    # Do nothing if the patch doesn't update doc:
+    git diff --name-only --cherry-pick --exit-code postgres/master... doc && exit
+
+    # Exercise HTML and other docs:
+    ./configure >/dev/null
+    make -s -C doc
+    cp -r doc new-docs
+
+    # Build HTML docs from the parent commit
+    git checkout postgres/master -- doc
+    make -s -C doc clean
+    make -s -C doc html
+    cp -r doc old-docs
+
+    # Copy HTML which differ into html_docs
+    # This will show any files which differ from files generated from postgres/master.
+    # Commits to postgres/master which affect doc/ will cause more files to be
+    # included until the patch is rebased.
+    mkdir html_docs
+    cp new-docs/src/sgml/html/*.css new-docs/src/sgml/html/*.svg html_docs/
+    for f in `git diff --no-index --name-only old-docs/src/sgml/html new-docs/src/sgml/html`
+    do
+      cp $f html_docs/
+    done
+
+  html_docs_artifacts:
+    paths: ['html_docs/**/*.html', 'html_docs/**/*.png', 'html_docs/**/*.css']
-- 
2.17.1


--x1F0m3RQhDZyj8sd
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="0004-wip-cirrus-code-coverage.patch"

From d1c7b123382fee3989c7e1b1ef14057af0651862 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Mon, 17 Jan 2022 00:54:28 -0600
Subject: [PATCH 4/7] wip: cirrus: code coverage

XXX: lcov should be installed in the OS image
---
 .cirrus.yml | 36 ++++++++++++++++++++++++++++++++++--
 1 file changed, 34 insertions(+), 2 deletions(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index 73b0baeaa7..635ee64b50 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -174,6 +174,14 @@ task:
     cat /proc/cmdline
     ulimit -a -H && ulimit -a -S
     export
+    git remote -v
+    git branch -a
+    git remote add postgres https://github.com/postgres/postgres
+    time git fetch -v postgres master
+    git log -1 postgres/master
+    git diff --name-only --cherry-pick postgres/master...
+    #git branch parent `git merge-base postgres/master HEAD`
+    #git diff --name-only parent..
   create_user_script: |
     useradd -m postgres
     chown -R postgres:postgres .
@@ -185,13 +193,14 @@ task:
     mkdir -m 770 /tmp/cores
     chown root:postgres /tmp/cores
     sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core'
-    #apt-get update
-    #apt-get -y install ...
+    apt-get update
+    apt-get -y install lcov
 
   configure_script: |
     su postgres <<-EOF
       ./configure \
         --enable-cassert --enable-debug --enable-tap-tests \
+        --enable-coverage \
         --enable-nls \
         \
         ${LINUX_CONFIGURE_FEATURES} \
@@ -211,6 +220,29 @@ task:
       make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS}
     EOF
 
+  generate_coverage_report_script: |
+      mkdir coverage
+      # Coverage only for changed files
+      # This is useful to see coverage of newly-added code, but won't
+      # show added/lost coverage in files which this patch doesn't modify.
+      for f in `git diff --name-only --cherry-pick postgres/master... '*.c'`;
+      do
+        lcov --quiet --capture --directory "$f"
+      done >coverage/coverage.gcov
+
+      time genhtml coverage/coverage.gcov --output-directory coverage --show-details --legend --quiet
+      cp coverage/index.html coverage/00-index.html
+
+      # Coverage for all files:
+      #time lcov --quiet --capture --directory . >coverage/coverage-all.gcov
+      #time genhtml coverage/coverage-all.gcov --output-directory coverage-all --show-details --legend --quiet
+      #cp coverage-all/index.html coverage-all/00-index.html
+
+      #git diff --name-only --cherry-pick postgres/master... '*.c' |xargs -r gcov --demangled-names --
+
+  coverage_artifacts:
+    paths: ['coverage/**/*.html', 'coverage/**/*.png', 'coverage/**/*.gcov', 'coverage/**/*.css' ]
+
   on_failure:
     <<: *on_failure
     cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores
-- 
2.17.1


--x1F0m3RQhDZyj8sd
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="0005-cirrus-vcregress-test-modules-contrib-with-NO_INSTAL.patch"

From 226699150e3e224198fc297689add21bece51c4b Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sun, 9 Jan 2022 18:25:02 -0600
Subject: [PATCH 5/7] cirrus/vcregress: test modules/contrib with
 NO_INSTALLCHECK=1

---
 .cirrus.yml                                |  4 +-
 contrib/basic_archive/Makefile             |  2 +-
 contrib/pg_stat_statements/Makefile        |  2 +-
 contrib/test_decoding/Makefile             |  2 +-
 src/test/modules/snapshot_too_old/Makefile |  2 +-
 src/test/modules/worker_spi/Makefile       |  2 +-
 src/tools/msvc/vcregress.pl                | 46 +++++++++++++++++++---
 7 files changed, 48 insertions(+), 12 deletions(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index 635ee64b50..b56bdd337f 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -447,9 +447,9 @@ task:
   test_isolation_script: |
     %T_C% perl src/tools/msvc/vcregress.pl isolationcheck
   test_modules_script: |
-    %T_C% perl src/tools/msvc/vcregress.pl modulescheck
+    %T_C% perl src/tools/msvc/vcregress.pl modulescheck install
   test_contrib_script: |
-    %T_C% perl src/tools/msvc/vcregress.pl contribcheck
+    %T_C% perl src/tools/msvc/vcregress.pl contribcheck install
   stop_script: |
     tmp_install\bin\pg_ctl.exe stop -D tmp_check/db -l tmp_check/postmaster.log
   test_ssl_script: |
diff --git a/contrib/basic_archive/Makefile b/contrib/basic_archive/Makefile
index 14d036e1c4..246358973f 100644
--- a/contrib/basic_archive/Makefile
+++ b/contrib/basic_archive/Makefile
@@ -4,7 +4,7 @@ MODULES = basic_archive
 PGFILEDESC = "basic_archive - basic archive module"
 
 REGRESS = basic_archive
-REGRESS_OPTS = --temp-config $(top_srcdir)/contrib/basic_archive/basic_archive.conf
+REGRESS_OPTS = --temp-config=$(top_srcdir)/contrib/basic_archive/basic_archive.conf
 
 NO_INSTALLCHECK = 1
 
diff --git a/contrib/pg_stat_statements/Makefile b/contrib/pg_stat_statements/Makefile
index 7fabd96f38..d732e1ade7 100644
--- a/contrib/pg_stat_statements/Makefile
+++ b/contrib/pg_stat_statements/Makefile
@@ -15,7 +15,7 @@ PGFILEDESC = "pg_stat_statements - execution statistics of SQL statements"
 
 LDFLAGS_SL += $(filter -lm, $(LIBS))
 
-REGRESS_OPTS = --temp-config $(top_srcdir)/contrib/pg_stat_statements/pg_stat_statements.conf
+REGRESS_OPTS = --temp-config=$(top_srcdir)/contrib/pg_stat_statements/pg_stat_statements.conf
 REGRESS = pg_stat_statements oldextversions
 # Disabled because these tests require "shared_preload_libraries=pg_stat_statements",
 # which typical installcheck users do not have (e.g. buildfarm clients).
diff --git a/contrib/test_decoding/Makefile b/contrib/test_decoding/Makefile
index 36929dd97d..465e5d365d 100644
--- a/contrib/test_decoding/Makefile
+++ b/contrib/test_decoding/Makefile
@@ -11,7 +11,7 @@ ISOLATION = mxact delayed_startup ondisk_startup concurrent_ddl_dml \
 	oldest_xmin snapshot_transfer subxact_without_top concurrent_stream \
 	twophase_snapshot slot_creation_error
 
-REGRESS_OPTS = --temp-config $(top_srcdir)/contrib/test_decoding/logical.conf
+REGRESS_OPTS = --temp-config=$(top_srcdir)/contrib/test_decoding/logical.conf
 ISOLATION_OPTS = --temp-config $(top_srcdir)/contrib/test_decoding/logical.conf
 
 # Disabled because these tests require "wal_level=logical", which
diff --git a/src/test/modules/snapshot_too_old/Makefile b/src/test/modules/snapshot_too_old/Makefile
index dfb4537f63..752a0039fd 100644
--- a/src/test/modules/snapshot_too_old/Makefile
+++ b/src/test/modules/snapshot_too_old/Makefile
@@ -5,7 +5,7 @@
 EXTRA_CLEAN = $(pg_regress_clean_files)
 
 ISOLATION = sto_using_cursor sto_using_select sto_using_hash_index
-ISOLATION_OPTS = --temp-config $(top_srcdir)/src/test/modules/snapshot_too_old/sto.conf
+ISOLATION_OPTS = --temp-config=$(top_srcdir)/src/test/modules/snapshot_too_old/sto.conf
 
 # Disabled because these tests require "old_snapshot_threshold" >= 0, which
 # typical installcheck users do not have (e.g. buildfarm clients).
diff --git a/src/test/modules/worker_spi/Makefile b/src/test/modules/worker_spi/Makefile
index cbf9b2e37f..d9f7d9bab6 100644
--- a/src/test/modules/worker_spi/Makefile
+++ b/src/test/modules/worker_spi/Makefile
@@ -9,7 +9,7 @@ PGFILEDESC = "worker_spi - background worker example"
 REGRESS = worker_spi
 
 # enable our module in shared_preload_libraries for dynamic bgworkers
-REGRESS_OPTS = --temp-config $(top_srcdir)/src/test/modules/worker_spi/dynamic.conf
+REGRESS_OPTS = --temp-config=$(top_srcdir)/src/test/modules/worker_spi/dynamic.conf
 
 # Disable installcheck to ensure we cover dynamic bgworkers.
 NO_INSTALLCHECK = 1
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index e2b0db0879..7bcc709200 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -442,6 +442,7 @@ sub plcheck
 sub subdircheck
 {
 	my $module = shift;
+	my $installcheck = shift || 1;
 
 	if (   !-d "$module/sql"
 		|| !-d "$module/expected"
@@ -451,7 +452,7 @@ sub subdircheck
 	}
 
 	chdir $module;
-	my @tests = fetchTests();
+	my @tests = fetchTests($installcheck);
 
 	# Leave if no tests are listed in the module.
 	if (scalar @tests == 0)
@@ -461,6 +462,7 @@ sub subdircheck
 	}
 
 	my @opts = fetchRegressOpts();
+	push @opts, "--temp-instance=tmp_check" if $installcheck == -1;
 
 	# Special processing for python transform modules, see their respective
 	# Makefiles for more details regarding Python-version specific
@@ -488,7 +490,7 @@ sub subdircheck
 	print "Checking $module\n";
 	my @args = (
 		"$topdir/$Config/pg_regress/pg_regress",
-		"--bindir=${topdir}/${Config}/psql",
+		"--bindir=$tmp_installdir/bin",
 		"--dbname=contrib_regression", @opts, @tests);
 	print join(' ', @args), "\n";
 	system(@args);
@@ -498,6 +500,8 @@ sub subdircheck
 
 sub contribcheck
 {
+	my $mode = shift || '';
+
 	chdir "../../../contrib";
 	my $mstat = 0;
 	foreach my $module (glob("*"))
@@ -515,12 +519,25 @@ sub contribcheck
 		my $status = $? >> 8;
 		$mstat ||= $status;
 	}
+
+	# As above, but creates new DB instance for each module.  For CI.
+	if ($mode eq "install")
+	{
+		foreach my $module (glob("*"))
+		{
+			subdircheck("$module", -1);
+			$mstat ||= $? >> 8;
+		}
+	}
+
 	exit $mstat if $mstat;
 	return;
 }
 
 sub modulescheck
 {
+	my $mode = shift || '';
+
 	chdir "../../../src/test/modules";
 	my $mstat = 0;
 	foreach my $module (glob("*"))
@@ -529,6 +546,17 @@ sub modulescheck
 		my $status = $? >> 8;
 		$mstat ||= $status;
 	}
+
+	# As above, but creates new DB instance for each module.  For CI.
+	if ($mode eq "install")
+	{
+		foreach my $module (glob("*"))
+		{
+			subdircheck("$module", -1);
+			$mstat ||= $? >> 8;
+		}
+	}
+
 	exit $mstat if $mstat;
 	return;
 }
@@ -698,6 +726,7 @@ sub fetchRegressOpts
 		# option starting with "--".
 		@opts = grep { !/\$\(/ && /^--/ }
 		  map { (my $x = $_) =~ s/\Q$(top_builddir)\E/\"$topdir\"/; $x; }
+		  map { (my $x = $_) =~ s/\Q$(top_srcdir)\E/\"$topdir\"/; $x; }
 		  split(/\s+/, $1);
 	}
 	if ($m =~ /^\s*ENCODING\s*=\s*(\S+)/m)
@@ -723,14 +752,19 @@ sub fetchTests
 	my $m = <$handle>;
 	close($handle);
 	my $t = "";
+	my $installcheck = shift || 1;
 
 	$m =~ s{\\\r?\n}{}g;
 
-	# A module specifying NO_INSTALLCHECK does not support installcheck,
-	# so bypass its run by returning an empty set of tests.
 	if ($m =~ /^\s*NO_INSTALLCHECK\s*=\s*\S+/m)
 	{
-		return ();
+		# Skip modules marked installcheck unless running installcheck tests.
+		return () if $installcheck == 1;
+	}
+	else
+	{
+		# Skip modules not marked installcheck if running installcheck tests.
+		return () if $installcheck == -1;
 	}
 
 	if ($m =~ /^REGRESS\s*=\s*(.*)$/gm)
@@ -796,6 +830,8 @@ sub usage
 	  "\nOptions for <arg>: (used by check and installcheck)\n",
 	  "  serial         serial mode\n",
 	  "  parallel       parallel mode\n",
+	  "\nOptions for <arg>: (used by contribcheck and modulescheck)\n",
+	  "  install        also run tests which require a new instance\n",
 	  "\nOption for <arg>: for taptest\n",
 	  "  TEST_DIR       (required) directory where tests reside\n";
 	exit(1);
-- 
2.17.1


--x1F0m3RQhDZyj8sd
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="0006-wip-cirrus-windows-save-tmp_install-as-an-artifact.patch"

From 08933bcd93d4f57ad73ab6df2f1627b93e61b459 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sun, 16 Jan 2022 12:51:13 -0600
Subject: [PATCH 6/7] wip: cirrus/windows: save tmp_install as an artifact..

..to allow users to easily download compiled binaries to try a patch.
If they don't have a development environment handy or not familiar with
compilation.

XXX: maybe this should be conditional or commented out ?

ci-os-only: windows
---
 .cirrus.yml | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/.cirrus.yml b/.cirrus.yml
index b56bdd337f..0d6d6a749d 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -472,6 +472,14 @@ task:
     cd src/tools/msvc
     %T_C% perl vcregress.pl ecpgcheck
 
+  always:
+    compress_install_script:
+      - powershell Compress-Archive tmp_install tmp_install.zip
+      - zipinfo tmp_install.zip
+
+    install_artifacts:
+      path: 'tmp_install.zip'
+
   on_failure:
     <<: *on_failure
     crashlog_artifacts:
-- 
2.17.1


--x1F0m3RQhDZyj8sd
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="0007-wip-cirrus-windows-add-compiler_warnings_script.patch"

From a7d2bba6f51d816412fb645b0d4821c36ee5c400 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sun, 20 Feb 2022 15:01:59 -0600
Subject: [PATCH 7/7] wip: cirrus/windows: add compiler_warnings_script

I'm not sure how to write this test in windows shell; it's also not easy to
write it in posix sh, since windows shell is somehow interpretting && and ||...

ci-os-only: windows

https://cirrus-ci.com/task/6183879907213312
https://cirrus-ci.com/task/4876271443247104
---
 .cirrus.yml | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index 0d6d6a749d..be38f74aa5 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -396,7 +396,8 @@ task:
     # ForceNoAlign prevents msbuild from introducing line-breaks for long lines
     # disable file tracker, we're never going to rebuild, and it slows down the
     #   build
-    MSBFLAGS: -m -verbosity:minimal "-consoleLoggerParameters:Summary;ForceNoAlign" /p:TrackFileAccess=false -nologo
+    # -fileLoggerParameters1: write to msbuild.warn.log.
+    MSBFLAGS: -m -verbosity:minimal "-consoleLoggerParameters:Summary;ForceNoAlign" /p:TrackFileAccess=false -nologo -fileLoggerParameters1:warningsonly;logfile=msbuild.warn.log
 
     # If tests hang forever, cirrus eventually times out. In that case log
     # output etc is not uploaded, making the problem hard to debug. Of course
@@ -480,6 +481,10 @@ task:
     install_artifacts:
       path: 'tmp_install.zip'
 
+    # Success if the file doesn't exist or is empty, else fail
+    compiler_warnings_script:
+      - sh -exc 'f=msbuild.warn.log; if [ -s "$f" ]; then cat "$f"; exit 1; else exit 0; fi'
+
   on_failure:
     <<: *on_failure
     crashlog_artifacts:
-- 
2.17.1


--x1F0m3RQhDZyj8sd--