Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nPVwC-0004km-7n for pgsql-hackers@arkaria.postgresql.org; Wed, 02 Mar 2022 20:51:08 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1nPVwB-0002bM-69 for pgsql-hackers@arkaria.postgresql.org; Wed, 02 Mar 2022 20:51:07 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nPVwA-0002aw-HB for pgsql-hackers@lists.postgresql.org; Wed, 02 Mar 2022 20:51:06 +0000 Received: from mail-io1-xd2a.google.com ([2607:f8b0:4864:20::d2a]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1nPVw5-0003aw-Fx for pgsql-hackers@postgresql.org; Wed, 02 Mar 2022 20:51:05 +0000 Received: by mail-io1-xd2a.google.com with SMTP id r7so3420350iot.3 for ; Wed, 02 Mar 2022 12:51:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telsasoft-com.20210112.gappssmtp.com; s=20210112; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=DEp3tMTB4QQTcsBZ6aKKjqgCugw6CR1exaj/dr1ohwY=; b=izleH7x7/m/3ULUV7Ad3HdhymHKiDdhPk5KrILINw21D/hQCU9Ho7aUIusnrbovRZy iYUNYP24kZhTeWwPkprhB3bJ/MgsaCFiNHBkpZzsccRZpOFnC3dOSS3hkjT5EBl89quh TS5/XzziUMaSDnZjhITTdFSYK3Tc64j71l/klalM/wRKVGBH4rG8nmF4zNIXjljDlrXa XuXvWPXmMp/HrJ+GAYVcL5qpKX5wrHmihdfZ9Pe7NxTVyBK5/yyjMAcY+4sN9qVfKaRV TyGLC85rMFdRxOt2+cSOCqNEWed0mUlta4/iaZ2HhmEPzmKau3hdhK9IJ0c/qLAXTbr5 Qhzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=DEp3tMTB4QQTcsBZ6aKKjqgCugw6CR1exaj/dr1ohwY=; b=r+QP6A9WNHLaW3f/xCYmCXxgjRBjqINBfexTvw92E3lgOmr8CXIWlflmo2wzVGsh6K ov5AzQ703KEtgV753D0L/XTrNitJYDEkQNiSl4m/j9lSsSP72jmH/O0fEpLs+S0tB92A dmfJAHFYOgcd4frW2wefJJQO6+RhObHd4tyQCu1lvWda5DfV7zt/4wC0U7uiz3/RA5eX FGOF/jVJZW6sMGWGUGNucPsZN5mDDBa7P2xqIPkEojpQgyK+bLruuUnXjxCXy7tlrlcN kauMn/wqI2WMUQopF0v6DKbatfo7h9P26s3Binf7RKCh78YizR2tgkkbrNGZHT10Pohh 9SVg== X-Gm-Message-State: AOAM531mAMebtsBpfmDdgXvJY1ZDOwoSUWFVae8cOFnLUdUzwrQJc2+M hw2Tow5IMfRtB+G8cCYq50zuuA== X-Google-Smtp-Source: ABdhPJzyd4KPE+xsmMiXA0MFFtEiIEadvKfJkTzhvWEbz68nmeE56BcWL30lTCBVD/4mjCxEvBi2oA== X-Received: by 2002:a5e:dc49:0:b0:640:a0de:c177 with SMTP id s9-20020a5edc49000000b00640a0dec177mr24781339iop.90.1646254260635; Wed, 02 Mar 2022 12:51:00 -0800 (PST) Received: from pryzbyj.telsasoft (charmander.telsasoft.com. [50.244.222.1]) by smtp.gmail.com with ESMTPSA id b9-20020a056e02184900b002c21c1d8929sm60228ilv.63.2022.03.02.12.50.59 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 02 Mar 2022 12:50:59 -0800 (PST) Received: by pryzbyj.telsasoft (Postfix, from userid 1000) id 86827800CA5; Wed, 2 Mar 2022 14:50:58 -0600 (CST) Date: Wed, 2 Mar 2022 14:50:58 -0600 From: Justin Pryzby To: Andres Freund Cc: Tom Lane , Robert Haas , Andrew Dunstan , pgsql-hackers@postgresql.org, Thomas Munro , Melanie Plageman , Peter Eisentraut , Daniel Gustafsson Subject: Re: Adding CI to our tree Message-ID: <20220302205058.GJ15744@telsasoft.com> References: <20220212222625.aph3ft466ntifrpi@alap3.anarazel.de> <20220213214213.GS31460@telsasoft.com> <20220213220709.vjz5rziuhfdpqxrg@alap3.anarazel.de> <20220226025116.GJ9008@telsasoft.com> <20220227010908.vz2a7dmfzgwg742w@alap3.anarazel.de> <20220227024352.GC25269@telsasoft.com> <20220227025000.mjmfimls5gnz7em7@alap3.anarazel.de> <20220227031057.GD25269@telsasoft.com> <20220227040838.3z4gqihimyerxe3h@alap3.anarazel.de> <20220228205802.GE25269@telsasoft.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="47eKBCiAZYFK5l32" Content-Disposition: inline In-Reply-To: <20220228205802.GE25269@telsasoft.com> User-Agent: Mutt/1.9.4 (2018-02-28) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --47eKBCiAZYFK5l32 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Mon, Feb 28, 2022 at 02:58:02PM -0600, Justin Pryzby wrote: > I still think that if "Build Docs" is a separate cirrus task, it should rebuild > docs on every CI run, even if they haven't changed, for any patch that touches > docs/. It'll be confusing if cfbot shows 5 green circles and 4 of them were > built 1 day ago, and 1 was built 3 weeks ago. Docs are the task that runs > quickest, so I don't think it's worth doing anything special there (especially > without understanding the behavior of changesInclude()). > > Also, to allow users to view the built HTML docs, cfbot would need to 1) keep > track of previous CI runs; and 2) logic to handle "skipped" CI runs, to allow > showing artifacts from the previous run. If it's not already done, I think the > first half is a good idea on its own. But the 2nd part doesn't seem desirable. Maybe changesInclude() could work if we use this URL (from cirrus' documentation), which uses the artifacts from the last successful build. https://api.cirrus-ci.com/v1/artifact/github/justinpryzby/postgres/Documentation/html_docs/html_docs/00-doc.html?branch=citest-cirrus2 That requires knowing the file being modified, so we'd have to generate an index of changed files - which I've started doing here. > However, I realized that we can filter on cfbot with either of these: > | $CIRRUS_CHANGE_TITLE =~ '^\[CF...' > | git log -1 |grep '^Author: Commitfest Bot ' > If we can assume that cfbot will continue submitting branches as a single > patch, this resolves the question of a "base branch", for cfbot. I don't know what you think of that idea, but I think I want to amend my proposal: show HTML and coverage artifacts for HEAD~1, unless set otherwise by an environment var. Today, that'd do the right thing for cfbot, and also for any 1-patch commits. > These patches implement that idea, and make "code coverage" and "HTML diffs" > stuff only run for cfbot commits. This still needs another round of testing, > though. The patch was missing a file due to an issue while rebasing - oops. BTW (regarding the last patch), I just noticed that -Og optimization can cause warnings with gcc-4.8.5-39.el7.x86_64. be-fsstubs.c: In function 'be_lo_export': be-fsstubs.c:522:24: warning: 'fd' may be used uninitialized in this function [-Wmaybe-uninitialized] if (CloseTransientFile(fd) != 0) ^ trigger.c: In function 'ExecCallTriggerFunc': trigger.c:2400:2: warning: 'result' may be used uninitialized in this function [-Wmaybe-uninitialized] return (HeapTuple) DatumGetPointer(result); ^ xml.c: In function 'xml_pstrdup_and_free': xml.c:1205:2: warning: 'result' may be used uninitialized in this function [-Wmaybe-uninitialized] return result; -- Justin --47eKBCiAZYFK5l32 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0001-cirrus-include-hints-how-to-install-OS-packages.patch" From 883edaa653bcf7f1a2369d8edf46eaaac1ba0ba2 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Mon, 17 Jan 2022 00:53:04 -0600 Subject: [PATCH 1/7] cirrus: include hints how to install OS packages.. This is useful for patches during development, but once a feature is merged, new libraries should be added to the OS image files, rather than installed during every CI run forever into the future. --- .cirrus.yml | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/.cirrus.yml b/.cirrus.yml index d10b0a82f9f..1b7c36283e9 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -73,10 +73,11 @@ task: chown -R postgres:postgres . mkdir -p ${CCACHE_DIR} chown -R postgres:postgres ${CCACHE_DIR} - setup_cores_script: | + setup_os_script: | mkdir -m 770 /tmp/cores chown root:postgres /tmp/cores sysctl kern.corefile='/tmp/cores/%N.%P.core' + #pkg install -y ... # NB: Intentionally build without --with-llvm. The freebsd image size is # already large enough to make VM startup slow, and even without llvm @@ -180,10 +181,12 @@ task: chown -R postgres:postgres ${CCACHE_DIR} echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf su postgres -c "ulimit -l -H && ulimit -l -S" - setup_cores_script: | + setup_os_script: | mkdir -m 770 /tmp/cores chown root:postgres /tmp/cores sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' + #apt-get update + #apt-get -y install ... configure_script: | su postgres <<-EOF @@ -237,7 +240,7 @@ task: ulimit -a -H && ulimit -a -S export - setup_cores_script: + setup_os_script: - mkdir ${HOME}/cores - sudo sysctl kern.corefile="${HOME}/cores/core.%P" @@ -384,6 +387,9 @@ task: powershell -Command get-psdrive -psprovider filesystem set + setup_os_script: | + REM choco install -y ... + configure_script: # copy errors out when using forward slashes - copy src\tools\ci\windows_build_config.pl src\tools\msvc\config.pl @@ -479,6 +485,10 @@ task: ccache_cache: folder: $CCACHE_DIR + setup_os_script: | + #apt-get update + #apt-get -y install ... + ### # Test that code can be built with gcc/clang without warnings ### -- 2.17.1 --47eKBCiAZYFK5l32 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-cirrus-build-docs-as-a-separate-task.patch" From 12127d5e4fe9dfccafa12d70311e58f91c34abec Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Sat, 26 Feb 2022 19:34:35 -0600 Subject: [PATCH 2/7] cirrus: build docs as a separate task.. Because a failure in documentation should be shown even if Linux also fails check-world. This'll automatically show up as a separate "column" on cfbot. Also, in the future, this will hopefully upload each patch's changed HTML docs as an artifact, for easy review. ci-os-only: html --- .cirrus.yml | 46 ++++++++++++++++++++++++++++++++-------------- 1 file changed, 32 insertions(+), 14 deletions(-) diff --git a/.cirrus.yml b/.cirrus.yml index 1b7c36283e9..72671a66eda 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -554,19 +554,37 @@ task: make -s -j${BUILD_JOBS} clean time make -s -j${BUILD_JOBS} world-bin - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" \ - CXX="ccache g++" \ - CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - always: upload_caches: ccache + + +### +# Verify docs can be built +### + +task: + name: Documentation + + env: + CPUS: 1 + BUILD_JOBS: 1 + + only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*(docs|html).*' + #skip: "!changesInclude('.cirrus.yml', 'doc/**')" + + container: + image: $CONTAINER_REPO/linux_debian_bullseye_ci:latest + cpu: $CPUS + memory: 2G + + sysinfo_script: | + id + uname -a + cat /proc/cmdline + ulimit -a -H && ulimit -a -S + export + + # Exercise HTML and other docs: + docs_build_script: | + time ./configure + make -s -j${BUILD_JOBS} -C doc -- 2.17.1 --47eKBCiAZYFK5l32 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-cirrus-upload-changed-html-docs-as-artifacts.patch" From 4a64b4306d5d6157a0ee8c3795005068541cdda4 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Sat, 26 Feb 2022 19:39:10 -0600 Subject: [PATCH 3/7] cirrus: upload changed html docs as artifacts Always run doc build; to allow them to be shown in cfbot, they should not be skipped if the linux build fails. This could be done on the client side (cfbot). One advantage of doing it here is that fewer docs are uploaded - many patches won't upload docs at all. https://cirrus-ci.com/task/5396696388599808 ci-os-only: html --- .cirrus.yml | 25 +++++++++++++++++++++++-- src/tools/ci/copy-changed-docs | 16 ++++++++++++++++ 2 files changed, 39 insertions(+), 2 deletions(-) create mode 100755 src/tools/ci/copy-changed-docs diff --git a/.cirrus.yml b/.cirrus.yml index 72671a66eda..e168a52a85a 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -26,6 +26,13 @@ env: TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf PG_TEST_EXTRA: kerberos ldap ssl + # The commit that this branch is rebased on. There's no easy way to find this. + # This does the right thing for cfbot, which always squishes all patches into a single commit. + # And does the right thing for any 1-patch commits. + # Patches series manually submitted to cirrus may benefit from setting this + # to the number of patches in the series (or directly to the commit the series was rebased on). + BASE_COMMIT: HEAD~1 + # What files to preserve in case tests fail on_failure: &on_failure @@ -559,7 +566,7 @@ task: ### -# Verify docs can be built +# Verify docs can be built, and (only on cfbot) upload changed docs as artifacts ### task: @@ -570,7 +577,7 @@ task: BUILD_JOBS: 1 only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*(docs|html).*' - #skip: "!changesInclude('.cirrus.yml', 'doc/**')" + #skip: "!changesInclude('.cirrus.yml', 'doc/**', 'src/tools/ci/copy-changed-docs')" container: image: $CONTAINER_REPO/linux_debian_bullseye_ci:latest @@ -583,8 +590,22 @@ task: cat /proc/cmdline ulimit -a -H && ulimit -a -S export + git diff --name-only "$BASE_COMMIT" # Exercise HTML and other docs: docs_build_script: | time ./configure make -s -j${BUILD_JOBS} -C doc + cp -r doc new-docs + + # Build HTML docs from the base commit. + git checkout "$BASE_COMMIT" -- doc + make -s -C doc clean + make -s -C doc html + cp -r doc old-docs + + copy_changed_docs_script: | + src/tools/ci/copy-changed-docs + + html_docs_artifacts: + paths: ['html_docs/**/*.html', 'html_docs/**/*.png', 'html_docs/**/*.css'] diff --git a/src/tools/ci/copy-changed-docs b/src/tools/ci/copy-changed-docs new file mode 100755 index 00000000000..0a942838f5c --- /dev/null +++ b/src/tools/ci/copy-changed-docs @@ -0,0 +1,16 @@ +#! /bin/sh +# Copy HTML which differ into html_docs +set -e + +outdir=html_docs + +mkdir "$outdir" +cp new-docs/src/sgml/html/*.css new-docs/src/sgml/html/*.svg "$outdir/" + +changed=`git diff --no-index --name-only old-docs/src/sgml/html new-docs/src/sgml/html` || + [ $? -eq 1 ] + +for f in $changed +do + cp "$f" "$outdir" +done -- 2.17.1 --47eKBCiAZYFK5l32 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0004-f-html-index-file.patch" From 3eb756f669aadfe065d96561817aae39a8b24a75 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Mon, 28 Feb 2022 23:18:19 -0600 Subject: [PATCH 4/7] f!html: index file This allows linking to the artifacts from the last successful build, which itself allows *not* rebuilding when sources haven't changed. ci-os-only: html --- src/tools/ci/copy-changed-docs | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/src/tools/ci/copy-changed-docs b/src/tools/ci/copy-changed-docs index 0a942838f5c..ca6214d40a6 100755 --- a/src/tools/ci/copy-changed-docs +++ b/src/tools/ci/copy-changed-docs @@ -7,10 +7,19 @@ outdir=html_docs mkdir "$outdir" cp new-docs/src/sgml/html/*.css new-docs/src/sgml/html/*.svg "$outdir/" +# The index is useful to allow a static link to all changed docs +# https://api.cirrus-ci.com/v1/artifact/github/USERNAME/postgres/Documentation/html_docs/html_docs/00-doc.html?branch=BRANCH +index="$outdir/00-doc.html" +echo "Index of changed docs
    " >"$index" + changed=`git diff --no-index --name-only old-docs/src/sgml/html new-docs/src/sgml/html` || [ $? -eq 1 ] for f in $changed do cp "$f" "$outdir" -done + fn=${f##*/} + echo "
  • $fn" +done >>"$index" + +echo "
" >>"$index" -- 2.17.1 --47eKBCiAZYFK5l32 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0005-wip-cirrus-code-coverage.patch" From b8733e960daa678ef4d568ffaa7aae0610166fb4 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Mon, 17 Jan 2022 00:54:28 -0600 Subject: [PATCH 5/7] wip: cirrus: code coverage XXX: lcov should be installed in the OS image XXX: Use --num-spaces=4 like in src/Makefile.global.in ? ci-os-only: linux --- .cirrus.yml | 13 +++++++++++-- src/tools/ci/code-coverage-report | 24 ++++++++++++++++++++++++ 2 files changed, 35 insertions(+), 2 deletions(-) create mode 100755 src/tools/ci/code-coverage-report diff --git a/.cirrus.yml b/.cirrus.yml index e168a52a85a..d9136365724 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -181,6 +181,7 @@ task: cat /proc/cmdline ulimit -a -H && ulimit -a -S export + git diff --name-only "$BASE_COMMIT" create_user_script: | useradd -m postgres chown -R postgres:postgres . @@ -192,13 +193,14 @@ task: mkdir -m 770 /tmp/cores chown root:postgres /tmp/cores sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - #apt-get update - #apt-get -y install ... + apt-get update + apt-get -y install lcov configure_script: | su postgres <<-EOF ./configure \ --enable-cassert --enable-debug --enable-tap-tests \ + --enable-coverage \ --enable-nls \ \ ${LINUX_CONFIGURE_FEATURES} \ @@ -218,6 +220,13 @@ task: make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} EOF + # Build coverage report for files changed since the base commit. + generate_coverage_report_script: | + src/tools/ci/code-coverage-report "$BASE_COMMIT" + + coverage_artifacts: + paths: ['coverage/**/*.html', 'coverage/**/*.png', 'coverage/**/*.gcov', 'coverage/**/*.css' ] + on_failure: <<: *on_failure cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores diff --git a/src/tools/ci/code-coverage-report b/src/tools/ci/code-coverage-report new file mode 100755 index 00000000000..c09a159704b --- /dev/null +++ b/src/tools/ci/code-coverage-report @@ -0,0 +1,24 @@ +#! /bin/sh +# Called during the linux CI task to generate a code coverage report. +set -e + +base_branch=$1 +changed=`git diff --name-only "$base_branch" '*.c'` + +mkdir coverage + +# Coverage only for changed files +# This is useful to see coverage of newly-added code, but won't +# show added/lost coverage in files which this patch doesn't modify. + +gcov=coverage/coverage.gcov +for f in $changed +do + lcov --quiet --capture --directory "$f" +done >"$gcov" + +# Exit successfully if no relevant files were changed +[ -s "$gcov" ] || exit 0 + +genhtml "$gcov" --output-directory coverage --show-details --legend --quiet +cp coverage/index.html coverage/00-index.html -- 2.17.1 --47eKBCiAZYFK5l32 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0006-wip-cirrus-windows-add-compiler_warnings_script.patch" From feceea4413b84f478e6a0888cdfab4be1c80767a Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Sun, 20 Feb 2022 15:01:59 -0600 Subject: [PATCH 6/7] wip: cirrus/windows: add compiler_warnings_script I'm not sure how to write this test in windows shell; it's also not easy to write it in posix sh, since windows shell is somehow interpretting && and ||... ci-os-only: windows https://cirrus-ci.com/task/6183879907213312 https://cirrus-ci.com/task/4876271443247104 --- .cirrus.yml | 9 ++++++++- src/tools/ci/windows-compiler-warnings | 16 ++++++++++++++++ 2 files changed, 24 insertions(+), 1 deletion(-) create mode 100755 src/tools/ci/windows-compiler-warnings diff --git a/.cirrus.yml b/.cirrus.yml index d9136365724..6f05d420c85 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -380,7 +380,8 @@ task: # ForceNoAlign prevents msbuild from introducing line-breaks for long lines # disable file tracker, we're never going to rebuild, and it slows down the # build - MSBFLAGS: -m -verbosity:minimal "-consoleLoggerParameters:Summary;ForceNoAlign" /p:TrackFileAccess=false -nologo + # -fileLoggerParameters1: write to msbuild.warn.log. + MSBFLAGS: -m -verbosity:minimal "-consoleLoggerParameters:Summary;ForceNoAlign" /p:TrackFileAccess=false -nologo -fileLoggerParameters1:warningsonly;logfile=msbuild.warn.log # If tests hang forever, cirrus eventually times out. In that case log # output etc is not uploaded, making the problem hard to debug. Of course @@ -456,6 +457,12 @@ task: cd src/tools/msvc %T_C% perl vcregress.pl ecpgcheck + # These should be last, so all the important checks are always run + always: + # Success if the file doesn't exist or is empty, else fail + compiler_warnings_script: + - sh src\tools\ci\windows-compiler-warnings msbuild.warn.log + on_failure: <<: *on_failure crashlog_artifacts: diff --git a/src/tools/ci/windows-compiler-warnings b/src/tools/ci/windows-compiler-warnings new file mode 100755 index 00000000000..d6f9a1fc569 --- /dev/null +++ b/src/tools/ci/windows-compiler-warnings @@ -0,0 +1,16 @@ +#! /bin/sh +# Success if the given file doesn't exist or is empty, else fail +# This is a separate file only to avoid dealing with windows shell quoting and escaping. +set -e + +fn=$1 + +if [ -s "$fn" ] +then + # Display the file's content, then exit indicating failure + cat "$fn" + exit 1 +else + # Success + exit 0 +fi -- 2.17.1 --47eKBCiAZYFK5l32 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0007-cirrus-compile-with-Og.patch" From d180953d273c221a30c5e9ad8d74b1b4dfc60bd1 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Sun, 27 Feb 2022 15:17:50 -0600 Subject: [PATCH 7/7] cirrus: compile with -Og.. To improve performance of check-world, and improve debugging, without significantly slower builds (they're cached anyway). This makes freebsd check-world run in 8.5 minutes rather than 15 minutes. --- .cirrus.yml | 12 +++++++----- src/tools/msvc/MSBuildProject.pm | 4 ++-- 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/.cirrus.yml b/.cirrus.yml index 6f05d420c85..8b673bf58cf 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -113,7 +113,7 @@ task: \ CC="ccache cc" \ CXX="ccache c++" \ - CFLAGS="-O0 -ggdb" + CFLAGS="-Og -ggdb" EOF build_script: su postgres -c "gmake -s -j${BUILD_JOBS} world-bin" upload_caches: ccache @@ -208,8 +208,8 @@ task: CC="ccache gcc" \ CXX="ccache g++" \ CLANG="ccache clang" \ - CFLAGS="-O0 -ggdb" \ - CXXFLAGS="-O0 -ggdb" + CFLAGS="-Og -ggdb" \ + CXXFLAGS="-Og -ggdb" EOF build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" upload_caches: ccache @@ -329,8 +329,8 @@ task: CC="ccache cc" \ CXX="ccache c++" \ CLANG="ccache ${brewpath}/llvm/bin/ccache" \ - CFLAGS="-O0 -ggdb" \ - CXXFLAGS="-O0 -ggdb" \ + CFLAGS="-Og -ggdb" \ + CXXFLAGS="-Og -ggdb" \ \ LLVM_CONFIG=${brewpath}/llvm/bin/llvm-config \ PYTHON=python3 @@ -383,6 +383,8 @@ task: # -fileLoggerParameters1: write to msbuild.warn.log. MSBFLAGS: -m -verbosity:minimal "-consoleLoggerParameters:Summary;ForceNoAlign" /p:TrackFileAccess=false -nologo -fileLoggerParameters1:warningsonly;logfile=msbuild.warn.log + MSBUILD_OPTIMIZE: MaxSpeed + # If tests hang forever, cirrus eventually times out. In that case log # output etc is not uploaded, making the problem hard to debug. Of course # tests internally should have shorter timeouts, but that's proven to not diff --git a/src/tools/msvc/MSBuildProject.pm b/src/tools/msvc/MSBuildProject.pm index 5e312d232e9..05e0c41eb5c 100644 --- a/src/tools/msvc/MSBuildProject.pm +++ b/src/tools/msvc/MSBuildProject.pm @@ -85,7 +85,7 @@ EOF $f, 'Debug', { defs => "_DEBUG;DEBUG=1", - opt => 'Disabled', + opt => $ENV{MSBUILD_OPTIMIZE} || 'Disabled', strpool => 'false', runtime => 'MultiThreadedDebugDLL' }); @@ -94,7 +94,7 @@ EOF 'Release', { defs => "", - opt => 'Full', + opt => $ENV{MSBUILD_OPTIMIZE} || 'Full', strpool => 'true', runtime => 'MultiThreadedDLL' }); -- 2.17.1 --47eKBCiAZYFK5l32--