Date: Fri, 3 Feb 2023 08:26:57 -0600
From: Justin Pryzby <pryzby@telsasoft.com>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Andres Freund <andres@anarazel.de>,
	Andrew Dunstan <andrew@dunslane.net>,
	pgsql-hackers@lists.postgresql.org, Noah Misch <noah@leadboat.com>,
	Michael Paquier <michael@paquier.xyz>,
	Anastasia Lubennikova <lubennikovaav@gmail.com>,
	Robert Haas <robertmhaas@gmail.com>,
	Melanie Plageman <melanieplageman@gmail.com>,
	Peter Eisentraut <peter.eisentraut@enterprisedb.com>,
	Daniel Gustafsson <daniel@yesql.se>,
	samay sharma <smilingsamay@gmail.com>
Subject: Re: CI and test improvements
Message-ID: <20230203142656.GA1653@telsasoft.com>
References: <20220910200542.GX31833@telsasoft.com>
 <20221104235412.GE16921@telsasoft.com>
 <20221105015946.yrxijqb7h4rqhp6d@awork3.anarazel.de>
 <20221113235303.GA26337@telsasoft.com>
 <20221121224542.p2zapvyvb7objluw@alap3.anarazel.de>
 <20221122225744.GF11463@telsasoft.com>
 <CA+hUKGL=z6-kU80nPTjWqV=Xo=mpM36Lca=ngTr4LqcCf=tHzQ@mail.gmail.com>
 <CA+hUKG+kFpXfCsJQPJApkk7WrBFYYSao0V8q+cj88KQBV8XYRQ@mail.gmail.com>
 <1441145.1675300332@sss.pgh.pa.us>
 <CA+hUKGLNyuvxTcchJe+DK_=_+fZTYCKA4c4o8eDwO5OYc2=qXQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="61jdw2sOBCFtR2d/"
Content-Disposition: inline
In-Reply-To: 
 <CA+hUKGLNyuvxTcchJe+DK_=_+fZTYCKA4c4o8eDwO5OYc2=qXQ@mail.gmail.com>
User-Agent: Mutt/1.9.4 (2018-02-28)
Archived-At: 
 <https://www.postgresql.org/message-id/20230203142656.GA1653%40telsasoft.com>
Precedence: bulk


--61jdw2sOBCFtR2d/
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

rebased, and re-including a patch to show code coverage of changed
files.

a5b3e50d922 cirrus/windows: add compiler_warnings_script
4c98dcb0e03 cirrus/freebsd: run with more CPUs+RAM and do not repartition
aaeef938ed4 cirrus/freebsd: define ENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS
9baf41674ad pg_upgrade: tap test: exercise --link and --clone
7e09035f588 WIP: ci/meson: allow showing only failed tests ..
e4534821ef5 cirrus/ccache: use G rather than GB suffix..
185d1c3ed13 cirrus: code coverage
5dace84a038 cirrus: upload changed html docs as artifacts
852360330ef +html index file


--61jdw2sOBCFtR2d/
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="0001-cirrus-windows-add-compiler_warnings_script.patch"

From a5b3e50d922a6dcff22feabbcae741b23b2347c6 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Wed, 25 May 2022 21:53:22 -0500
Subject: [PATCH 1/9] cirrus/windows: add compiler_warnings_script

I'm not sure how to write this test in windows shell; it's also easy to
write something that doesn't work in posix sh, since windows shell is
interpretting && and ||...

https://www.postgresql.org/message-id/20220212212310.f645c6vw3njkgxka%40alap3.anarazel.de

See also:
8a1ce5e54f6d144e4f8e19af7c767b026ee0c956
https://cirrus-ci.com/task/6241060062494720
https://cirrus-ci.com/task/6496366607204352

ci-os-only: windows
---
 .cirrus.yml | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index 1204824d2eb..eefc5c21fe6 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -566,12 +566,21 @@ task:
 
   build_script: |
     vcvarsall x64
-    ninja -C build
+    ninja -C build |tee build.txt
+    REM Since pipes lose the exit status of the preceding command, rerun the compilation
+    REM without the pipe, exiting now if it fails, to avoid trying to run checks
+    ninja -C build > nul
 
   check_world_script: |
     vcvarsall x64
     meson test %MTEST_ARGS% --num-processes %TEST_JOBS%
 
+  # This should be last, so check_world is run even if there are warnings
+  always:
+    compiler_warnings_script:
+      # this avoids using metachars which would be interpretted by the windows shell
+      - sh -c 'if grep ": warning " build.txt; then exit 1; fi; exit 0'
+
   on_failure:
     <<: *on_failure_meson
     crashlog_artifacts:
-- 
2.25.1


--61jdw2sOBCFtR2d/
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="0002-cirrus-freebsd-run-with-more-CPUs-RAM-and-do-not-rep.patch"

From 4c98dcb0e033ca12c6a6093c5e94e4231756fa4d Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 24 Jun 2022 00:09:12 -0500
Subject: [PATCH 2/9] cirrus/freebsd: run with more CPUs+RAM and do not
 repartition

There was some historic problem where tests under freebsd took 8+ minutes (and
before 4a288a37f took 15 minutes).

This reduces test time from 10min to 3min.
4 CPUs 4 tests https://cirrus-ci.com/task/4880240739614720
4 CPUs 6 tests https://cirrus-ci.com/task/4664440120410112 https://cirrus-ci.com/task/4586784884523008
4 CPUs 8 tests https://cirrus-ci.com/task/5001995491737600

6 CPUs https://cirrus-ci.com/task/6678321684545536
8 CPUs https://cirrus-ci.com/task/6264854121021440

See also:
https://www.postgresql.org/message-id/flat/20220310033347.hgxk4pyarzq4hxwp@alap3.anarazel.de#f36c0b17e33e31e7925e7e5812998686
8 jobs 7min https://cirrus-ci.com/task/6186376667332608

//-os-only: freebsd
---
 .cirrus.yml | 13 ++++---------
 1 file changed, 4 insertions(+), 9 deletions(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index eefc5c21fe6..0c12ca04fd9 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -131,11 +131,9 @@ task:
   name: FreeBSD - 13 - Meson
 
   env:
-    # FreeBSD on GCP is slow when running with larger number of CPUS /
-    # jobs. Using one more job than cpus seems to work best.
-    CPUS: 2
-    BUILD_JOBS: 3
-    TEST_JOBS: 3
+    CPUS: 4
+    BUILD_JOBS: 4
+    TEST_JOBS: 6
 
     CCACHE_DIR: /tmp/ccache_dir
     CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DCOPY_PARSE_PLAN_TREES -DWRITE_READ_PARSE_PLAN_TREES -DRAW_EXPRESSION_COVERAGE_TEST
@@ -160,8 +158,6 @@ task:
 
   ccache_cache:
     folder: $CCACHE_DIR
-  # Work around performance issues due to 32KB block size
-  repartition_script: src/tools/ci/gcp_freebsd_repartition.sh
   create_user_script: |
     pw useradd postgres
     chown -R postgres:postgres .
@@ -175,8 +171,7 @@ task:
     #pkg install -y ...
 
   # NB: Intentionally build without -Dllvm. The freebsd image size is already
-  # large enough to make VM startup slow, and even without llvm freebsd
-  # already takes longer than other platforms except for windows.
+  # large enough to make VM startup slow
   configure_script: |
     su postgres <<-EOF
       meson setup \
-- 
2.25.1


--61jdw2sOBCFtR2d/
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="0003-cirrus-freebsd-define-ENFORCE_REGRESSION_TEST_NAME_R.patch"

From aaeef938ed4a1b604c41d3973a49d790ad58af20 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 9 Dec 2022 15:32:50 -0600
Subject: [PATCH 3/9] cirrus/freebsd: define
 ENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS

See also: 54100f5c6052404f68de9ce7310ceb61f1c291f8

ci-os-only: freebsd
---
 .cirrus.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index 0c12ca04fd9..6186505ccd7 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -136,7 +136,7 @@ task:
     TEST_JOBS: 6
 
     CCACHE_DIR: /tmp/ccache_dir
-    CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DCOPY_PARSE_PLAN_TREES -DWRITE_READ_PARSE_PLAN_TREES -DRAW_EXPRESSION_COVERAGE_TEST
+    CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DCOPY_PARSE_PLAN_TREES -DWRITE_READ_PARSE_PLAN_TREES -DRAW_EXPRESSION_COVERAGE_TEST -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS
     CFLAGS: -Og -ggdb
 
   depends_on: SanityCheck
-- 
2.25.1


--61jdw2sOBCFtR2d/
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="0004-pg_upgrade-tap-test-exercise-link-and-clone.patch"

From 9baf41674ad28d68f2c657363079643955252fee Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 30 Jul 2022 19:25:51 -0500
Subject: [PATCH 4/9] pg_upgrade: tap test: exercise --link and --clone

This both increases code coverage and accelerates tests.

See also: b059a2409faf5833b3ba7792e247d6466c9e8090

linux,
macos,
//-os-only: freebsd
---
 .cirrus.yml | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/.cirrus.yml b/.cirrus.yml
index 6186505ccd7..a8a514ed178 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -138,6 +138,7 @@ task:
     CCACHE_DIR: /tmp/ccache_dir
     CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DCOPY_PARSE_PLAN_TREES -DWRITE_READ_PARSE_PLAN_TREES -DRAW_EXPRESSION_COVERAGE_TEST -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS
     CFLAGS: -Og -ggdb
+    PG_TEST_PG_UPGRADE_MODE: --link
 
   depends_on: SanityCheck
   only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*freebsd.*'
@@ -269,6 +270,7 @@ task:
     LDFLAGS: $SANITIZER_FLAGS
     CC: ccache gcc
     CXX: ccache g++
+    PG_TEST_PG_UPGRADE_MODE: --link
 
     LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES
     LINUX_MESON_FEATURES: *LINUX_MESON_FEATURES
@@ -419,6 +421,7 @@ task:
     CPPFLAGS: -DRANDOMIZE_ALLOCATED_MEMORY
     CFLAGS: -Og -ggdb
     CXXFLAGS: -Og -ggdb
+    PG_TEST_PG_UPGRADE_MODE: --clone
 
   depends_on: SanityCheck
   only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*(macos|darwin|osx).*'
-- 
2.25.1


--61jdw2sOBCFtR2d/
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="0005-WIP-ci-meson-allow-showing-only-failed-tests.patch"

From 7e09035f5882a0262fe339eb305e64f3bbaa2bb9 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 25 Nov 2022 13:57:17 -0600
Subject: [PATCH 5/9] WIP: ci/meson: allow showing only failed tests ..

It's simpler and seems to make more sense to integrate this with
testwrap, rather than to run it after check-world, but only if it
failed, and finding a way to preserve the exit code.

https://www.postgresql.org/message-id/20221114235328.lxdj3puenfhirhqm@awork3.anarazel.de

> It is wasteful to upload thousdands of logfiles to show a single
> failure.  That would make our cirrus tasks faster - compressing and
> uploading the logs takes over a minute.
>
> It's also a lot friendlier to show fewer than 8 pages of test folders to
> search through to find the one that failed.

macos
ci-os-only: linux-meson freebsd
---
 .cirrus.yml        | 13 +++++++------
 src/tools/testwrap |  6 ++++++
 2 files changed, 13 insertions(+), 6 deletions(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index a8a514ed178..fd5585e5818 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -23,6 +23,7 @@ env:
   CHECKFLAGS: -Otarget
   PROVE_FLAGS: --timer
   MTEST_ARGS: --print-errorlogs --no-rebuild -C build
+  PG_FAILED_TESTDIR: ${CIRRUS_WORKING_DIR}/failed.build
   PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests
   TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf
   PG_TEST_EXTRA: kerberos ldap ssl
@@ -40,9 +41,9 @@ on_failure_ac: &on_failure_ac
 on_failure_meson: &on_failure_meson
   testrun_artifacts:
     paths:
-      - "build*/testrun/**/*.log"
-      - "build*/testrun/**/*.diffs"
-      - "build*/testrun/**/regress_log_*"
+      - "failed.build*/**/*.log"
+      - "failed.build*/**/*.diffs"
+      - "failed.build*/**/regress_log_*"
     type: text/plain
 
   # In theory it'd be nice to upload the junit files meson generates, so that
@@ -198,10 +199,10 @@ task:
       ulimit -c unlimited
       meson test $MTEST_ARGS --quiet --suite setup
       export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH"
-      mkdir -p build/testrun
+      mkdir -p build/testrun ${PG_FAILED_TESTDIR}
       build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust
       echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf
-      build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start
+      build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l ${PG_FAILED_TESTDIR}/runningcheck.log start
       meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running
       build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop
     EOF
@@ -390,7 +391,7 @@ task:
       test_world_32_script: |
         su postgres <<-EOF
           ulimit -c unlimited
-          PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS}
+          PYTHONCOERCECLOCALE=0 LANG=C PG_FAILED_TESTDIR=`pwd`/failed.build-32 meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS}
         EOF
 
       on_failure:
diff --git a/src/tools/testwrap b/src/tools/testwrap
index 7a64fe76a2d..445ac595afc 100755
--- a/src/tools/testwrap
+++ b/src/tools/testwrap
@@ -44,4 +44,10 @@ if sp.returncode == 0:
 else:
     print('# test failed')
     open(os.path.join(testdir, 'test.fail'), 'x')
+    faileddir = os.getenv('PG_FAILED_TESTDIR')
+    if faileddir:
+        parentdir = os.path.dirname(testdir)
+        newdest = os.path.join(faileddir, os.path.basename(parentdir), os.path.basename(testdir))
+        shutil.copytree(testdir, newdest)
+
 sys.exit(sp.returncode)
-- 
2.25.1


--61jdw2sOBCFtR2d/
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="0006-cirrus-ccache-use-G-rather-than-GB-suffix.patch"

From e4534821ef5c1997ff34a2515884c5592f171e0b Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sun, 13 Nov 2022 14:03:27 -0600
Subject: [PATCH 6/9] cirrus/ccache: use G rather than GB suffix..

the former being the documented spelling.
---
 .cirrus.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index fd5585e5818..a9fa4b5af27 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -667,7 +667,7 @@ task:
 
     # Use larger ccache cache, as this task compiles with multiple compilers /
     # flag combinations
-    CCACHE_MAXSIZE: "1GB"
+    CCACHE_MAXSIZE: "1G"
     CCACHE_DIR: "/tmp/ccache_dir"
 
     LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES
-- 
2.25.1


--61jdw2sOBCFtR2d/
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0007-cirrus-code-coverage.patch"

From 185d1c3ed13f5472fea32fd15f55158721b7245b Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Mon, 17 Jan 2022 00:54:28 -0600
Subject: [PATCH 7/9] cirrus: code coverage

Some alternatives:
- could build with "--coverage -fprofile-filter-files=", but that means
  that ccache will never work (both because it doesn't support that option
  and also because the arguments will be different for every patch).
- could use ninja coverage-html, but that can't filter only changed
  files, and would take a long time to upload a lot of useless files.

https://www.postgresql.org/message-id/202202111821.w3gqblvfp4pr%40alvherre.pgsql
https://www.postgresql.org/message-id/flat/20220409021853.GP24419@telsasoft.com

ci-os-only: freebsd
---
 .cirrus.yml                       | 18 +++++++++++++++-
 src/tools/ci/code-coverage-report | 35 +++++++++++++++++++++++++++++++
 2 files changed, 52 insertions(+), 1 deletion(-)
 create mode 100755 src/tools/ci/code-coverage-report

diff --git a/.cirrus.yml b/.cirrus.yml
index a9fa4b5af27..ec15263a665 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -28,6 +28,14 @@ env:
   TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf
   PG_TEST_EXTRA: kerberos ldap ssl
 
+  # The commit that this branch is rebased on.  There's no easy way to find this.
+  # This does the right thing for cfbot, which always squishes all patches into a single commit.
+  # And does the right thing for any 1-patch commits.
+  # Patch series manually submitted to cirrus would benefit from setting this to the
+  # number of patches in the series (or directly to the commit the series was rebased on).
+  #BASE_COMMIT: HEAD~1
+  # For demo purposes:
+  BASE_COMMIT: HEAD~11
 
 # What files to preserve in case tests fail
 on_failure_ac: &on_failure_ac
@@ -157,6 +165,7 @@ task:
     uname -a
     ulimit -a -H && ulimit -a -S
     export
+    git diff --name-only "$BASE_COMMIT"
 
   ccache_cache:
     folder: $CCACHE_DIR
@@ -170,7 +179,7 @@ task:
     chown root:postgres /tmp/cores
     sysctl kern.corefile='/tmp/cores/%N.%P.core'
   setup_additional_packages_script: |
-    #pkg install -y ...
+    pkg install -y lcov
 
   # NB: Intentionally build without -Dllvm. The freebsd image size is already
   # large enough to make VM startup slow
@@ -178,6 +187,7 @@ task:
     su postgres <<-EOF
       meson setup \
         --buildtype=debug \
+        -Db_coverage=true \
         -Dcassert=true -Dssl=openssl -Duuid=bsd -Dtcl_version=tcl86 -Ddtrace=auto \
         -DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
         -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \
@@ -188,10 +198,16 @@ task:
 
   test_world_script: |
     su postgres <<-EOF
+      set -e
       ulimit -c unlimited
       meson test $MTEST_ARGS --num-processes ${TEST_JOBS}
+      # Create coverage report for files changed since the base commit.
+      time ./src/tools/ci/code-coverage-report "$BASE_COMMIT" ./build ./coverage
     EOF
 
+  coverage_artifacts:
+    path: 'coverage/**'
+
   # test runningcheck, freebsd chosen because it's currently fast enough
   test_running_script: |
     su postgres <<-EOF
diff --git a/src/tools/ci/code-coverage-report b/src/tools/ci/code-coverage-report
new file mode 100755
index 00000000000..4f29d3f17a7
--- /dev/null
+++ b/src/tools/ci/code-coverage-report
@@ -0,0 +1,35 @@
+#! /bin/sh
+# Called during CI to generate a code coverage report of changed files.
+set -e
+
+base_branch=$1
+build_dir=$2
+outdir=$3
+
+changed=`git diff --name-only "$base_branch" '*.c'`
+[ -z "$changed" ] && exit 0 # Nothing changed
+
+mkdir "$outdir"
+
+# Coverage is shown only for changed files
+# This is useful to see coverage of newly-added code, but won't
+# show added/lost coverage in files which this patch doesn't modify.
+
+# This could be used to map from object file back to source file:
+# readelf --debug-dump=info src/backend/postgres_lib.a.p/utils_adt_array_userfuncs.c.o |awk '/DW_AT_name/{print $NF;exit}'
+# gcov ./src/port/libpgport_shlib.a.p/inet_net_ntop.c.gcno --stdout
+
+gcov=$outdir/coverage.gcov
+lcov --quiet --capture --directory "$build_dir" >"$gcov.all"
+
+# Filter to include only changed files
+echo "$changed" |sed 's,^,*/,' |
+	xargs -rt lcov --extract "$gcov.all" >"$gcov"
+
+ls -l "$outdir"
+
+# Exit successfully if no relevant files were changed
+[ -s "$gcov" ] || exit 0
+
+genhtml "$gcov" --show-details --legend --quiet --num-spaces=4 --output-directory "$outdir" --title="Coverage report of files changed since: $base_branch"
+cp "$outdir"/index.html "$outdir"/00-index.html
-- 
2.25.1


--61jdw2sOBCFtR2d/
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="0008-cirrus-upload-changed-html-docs-as-artifacts.patch"

From 5dace84a03882f7c0a274102fe497bfe9f606a76 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 26 Feb 2022 19:39:10 -0600
Subject: [PATCH 8/9] cirrus: upload changed html docs as artifacts

This could be done on the client side (cfbot).  One advantage of doing
it here is that fewer docs are uploaded - many patches won't upload docs
at all.

https://www.postgresql.org/message-id/flat/20220409021853.GP24419@telsasoft.com
https://www.postgresql.org/message-id/CAB8KJ=i4qmEuopQ+PCSMBzGd4O-Xv0FCnC+q1x7hN9hsdvkBug@mail.gmail.com

https://cirrus-ci.com/task/5396696388599808

ci-os-only: html
---
 .cirrus.yml                    | 16 +++++++++++++++-
 src/tools/ci/copy-changed-docs | 29 +++++++++++++++++++++++++++++
 2 files changed, 44 insertions(+), 1 deletion(-)
 create mode 100755 src/tools/ci/copy-changed-docs

diff --git a/.cirrus.yml b/.cirrus.yml
index ec15263a665..57359f43867 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -701,6 +701,7 @@ task:
     gcc -v
     clang -v
     export
+    git diff --name-only "$BASE_COMMIT"
 
   ccache_cache:
     folder: $CCACHE_DIR
@@ -775,7 +776,7 @@ task:
       time make -s -j${BUILD_JOBS} world-bin
 
   ###
-  # Verify docs can be built
+  # Verify docs can be built, and upload changed docs as artifacts
   ###
   # XXX: Only do this if there have been changes in doc/ since last build
   always:
@@ -787,6 +788,19 @@ task:
         CLANG="ccache clang"
       make -s -j${BUILD_JOBS} clean
       time make -s -j${BUILD_JOBS} -C doc
+      cp -r doc new-docs
+
+      # Re-build HTML docs from the base commit.
+      git checkout "$BASE_COMMIT" -- doc
+      make -s -C doc clean
+      time make -s -C doc html
+      cp -r doc old-docs
+
+    copy_changed_docs_script:
+      - src/tools/ci/copy-changed-docs "old-docs" "new-docs" "html_docs"
+
+    html_docs_artifacts:
+      paths: ['html_docs/*.html', 'html_docs/*.png', 'html_docs/*.css']
 
   ###
   # Verify headerscheck / cpluspluscheck succeed
diff --git a/src/tools/ci/copy-changed-docs b/src/tools/ci/copy-changed-docs
new file mode 100755
index 00000000000..1c921a8df6f
--- /dev/null
+++ b/src/tools/ci/copy-changed-docs
@@ -0,0 +1,29 @@
+#! /bin/sh
+# Copy HTML which differ between $old and $new into $outdir
+set -e
+
+old=$1
+new=$2
+outdir=$3
+
+# The index is large and changes often
+skippages="bookindex.html"
+
+mkdir "$outdir"
+cp "$new"/src/sgml/html/*.css "$new"/src/sgml/html/*.svg "$outdir"
+
+changed=`git diff --no-index --name-only "$old"/src/sgml/html "$new"/src/sgml/html` ||
+	[ $? -eq 1 ]
+
+for f in $changed
+do
+	# Avoid removed files
+	[ -f "$f" ] || continue
+
+	echo "$f" |grep -Ew "$skippages" >/dev/null &&
+		continue
+
+	cp -v "$f" "$outdir"
+done
+
+exit 0
-- 
2.25.1


--61jdw2sOBCFtR2d/
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0009-html-index-file.patch"

From 852360330ef7e41edfb4d591ef246435aa956731 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Mon, 28 Feb 2022 23:18:19 -0600
Subject: [PATCH 9/9] +html index file

This allows linking to the artifacts from the last successful build.

//freebsd
ci-os-only: warnings
---
 .cirrus.yml                    |  4 ++--
 src/tools/ci/copy-changed-docs | 42 ++++++++++++++++++++++++++++++++--
 2 files changed, 42 insertions(+), 4 deletions(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index 57359f43867..f36d7d110e0 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -674,8 +674,8 @@ task:
   # succeeds. This is particularly important for this task as we intentionally
   # use always: to continue after failures. Task that did not run count as a
   # success, so we need to recheck SanityChecks's condition here ...
-  depends_on: SanityCheck
-  only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*'
+  #depends_on: SanityCheck
+  only_if: $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:.*warnings'
 
   env:
     CPUS: 4
diff --git a/src/tools/ci/copy-changed-docs b/src/tools/ci/copy-changed-docs
index 1c921a8df6f..57dd2823a50 100755
--- a/src/tools/ci/copy-changed-docs
+++ b/src/tools/ci/copy-changed-docs
@@ -5,6 +5,10 @@ set -e
 old=$1
 new=$2
 outdir=$3
+branch=$CIRRUS_BRANCH
+
+# The index is large and changes often
+skippages="bookindex.html"
 
 # The index is large and changes often
 skippages="bookindex.html"
@@ -12,6 +16,19 @@ skippages="bookindex.html"
 mkdir "$outdir"
 cp "$new"/src/sgml/html/*.css "$new"/src/sgml/html/*.svg "$outdir"
 
+# The index is useful to allow a static link (not specific to a cirrus run) to the artifacts for the most-recent, successful CI run for a branch
+branchurl=https://api.cirrus-ci.com/v1/artifact/github/$CIRRUS_REPO_FULL_NAME/$CIRRUS_TASK_NAME/html_docs/html_docs/00-index.html?branch=$branch
+
+index="$outdir/00-index.html"
+cat >"$index" <<EOF
+<html>
+<head><title>Index of docs changed since: $BASE_COMMIT</title></head>
+<body>
+<!-- A link to documentation for the most recent successful build: $branchurl -->
+<h1>Index of docs changed since: $BASE_COMMIT</h1>
+<ul>
+EOF
+
 changed=`git diff --no-index --name-only "$old"/src/sgml/html "$new"/src/sgml/html` ||
 	[ $? -eq 1 ]
 
@@ -23,7 +40,28 @@ do
 	echo "$f" |grep -Ew "$skippages" >/dev/null &&
 		continue
 
-	cp -v "$f" "$outdir"
-done
+	cp -v "$f" "$outdir" >&2
+	fn=${f##*/}
+	# ?branch=... is needed when accessing the artifacts for the static link for the branch
+	# It's not needed and ignored if accessing artifacts for *this* CI run
+	echo "<li><a href='$fn?branch=$branch'>$fn</a>"
+done >>"$index"
+
+github=https://github.com/$CIRRUS_REPO_FULL_NAME/commit
+cirrus=https://cirrus-ci.com/build
+
+cat >>"$index" <<EOF
+</ul>
+<hr>
+<code>
+<br>This file was written on: `date --rfc-822 --utc`
+<br>CIRRUS_CHANGE_TITLE: $CIRRUS_CHANGE_TITLE
+<br>CIRRUS_CHANGE_IN_REPO: <a href="$github/$CIRRUS_CHANGE_IN_REPO">$CIRRUS_CHANGE_IN_REPO</a>
+<br>CIRRUS_BUILD_ID: <a href="$cirrus/$CIRRUS_BUILD_ID">$CIRRUS_BUILD_ID</a>
+<br>CIRRUS_LAST_GREEN_CHANGE: <a href="$github/$CIRRUS_LAST_GREEN_CHANGE">$CIRRUS_LAST_GREEN_CHANGE</a>
+<br>CIRRUS_LAST_GREEN_BUILD_ID: <a href="$cirrus/$CIRRUS_LAST_GREEN_BUILD_ID">$CIRRUS_LAST_GREEN_BUILD_ID</a>
+</code>
+</body></html>
+EOF
 
 exit 0
-- 
2.25.1


--61jdw2sOBCFtR2d/--