Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1wUAdV-000zK3-0e for pgsql-hackers@arkaria.postgresql.org; Mon, 01 Jun 2026 21:57:30 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.96) (envelope-from ) id 1wUAdU-00BHC1-03 for pgsql-hackers@arkaria.postgresql.org; Mon, 01 Jun 2026 21:57:28 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1wUAdS-00BHBt-1Q for pgsql-hackers@lists.postgresql.org; Mon, 01 Jun 2026 21:57:27 +0000 Received: from fhigh-b2-smtp.messagingengine.com ([202.12.124.153]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1wUAdK-00000000duU-1atE for pgsql-hackers@postgresql.org; Mon, 01 Jun 2026 21:57:25 +0000 Received: from phl-compute-01.internal (phl-compute-01.internal [10.202.2.41]) by mailfhigh.stl.internal (Postfix) with ESMTP id 18C7F7A0103; Mon, 1 Jun 2026 17:57:16 -0400 (EDT) Received: from phl-frontend-03 ([10.202.2.162]) by phl-compute-01.internal (MEProxy); Mon, 01 Jun 2026 17:57:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=anarazel.de; h= cc:cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm1; t=1780351036; x=1780437436; bh=7dUITN4HfKDdyLSlE4U1Qi4+N4mq7RY5q5hNo3kdiLU=; b= UZ68xfyKBeZokaSe8NX+riJYIpD/tKdI6QsQfrq8kaSlTTlRO6ptXYmG1lvSTarp DFVusTSHJFatQczZblvDBL46BLeYE3EJPF5KGG/Gz5WszADsj32NuKaFi6XvW4Sp y9to2eG+H/HpyvMXrwrCnPSfqCQl7E2AXwLbpxNkwWWpFPik5PzQ4YfeJ/tu7tu0 XMrF1nWlqVe+Dvaa18qjGr0SmTChxqBGmXfA6scQC5dek6QsqES8B/msrkPlrakD djRdQu+3gEV4yZzRtjNX4SNP2jz+zqnLU4ikNYixs0w32VPeu6vhHBxsRqBSRqGt jtjzNQKQ1gxfvj3FN4i+lQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1780351036; x= 1780437436; bh=7dUITN4HfKDdyLSlE4U1Qi4+N4mq7RY5q5hNo3kdiLU=; b=F E2CKhP0NziBvc/ZqRqJCSFi/G+Yfx59irulJzjljsm2cWscIYvzPHSIn9Oxwu2ev AhsB45oRhrA9y8qzOZ8AHeGlYOsG9o3Ck8Yvo4nwNpu9pPf5mxacphK5IHAF/cko xP/W6UEJ9nWHHrb018AuB/ElrgXE7soQjJ1jaf7HEdxBgx4lPCI9r2Ru/sGgKZcz SXQ+HgcmWbBoTtQE1/e2bij82Jsi7xF8ozqONHCiXfwYbkS41Q+JC3qsBJ0giDQT Og3wcSsVMN5BrJz4LN2Bmx3INbjmG+jRXClP5ARE2scnuBjo2LQY0p1wZpE7pzOG ljXgU3Nf171WYb4sZYQZw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: dmFkZTF1AiayzRx6WFXOc86JzcphA63HzgMbhEHnlf+ACfStbyir6S96r5PLsNLgP5jEbu jt61fAxVU5ABb2aoZfEMsOvmagnBlid8uBertHqEx2HAUCecp2RrfrB5SfhRW6C/k7wt2m nzKMp3c6d5NKvMAs9UkczkRNM1FDsrJVYyX4OxYg4iE2gF/PRilYGIa1sN5oYxy0I2veii a6lhBeJfa309bcrF7MyG1+pTJlG5GN6J9mCIplovEmZkjOwHN8ysNGlCrZ2iH9s+uIx5/F 3bL7cVuqVhIIGhNxMZ1oT0imj6kOVWUdSTb9WCbnpmBLaXnzIQmw/MiTh/XaTA7lf1kytA fCftJwR5DNkCVqpd5V1f9H0/SztCDV6/yFfAS/dNl4LJOVQtMWFU92TaLxVDaEyP6Rkmpz BhdXxAywYoiNlR0JLLXGJ5Zy4X9q/A703H7UWLgrr3xpAlLhL4DetAfH2O5Jgz8cgM6HH6 M1wFN82OohP8dWXwYZeQ7F9cNwO10+rIOGCJXBrrHzhc04KrHj38r3wfzd3V+gnF1iN5Oa ZG9AkcR1mYkHiSs6yueBDjfJywTdtwV/qLTzxPNWJlTA0mJ8UA5JB40wwNgShSZom548bZ JIdjKry7zNqfH9kw9t1YEhab/lWSHSS2ccxBkux8BzMrRg1WKNOsQEam0kzA X-ME-Proxy: Feedback-ID: id4a34324:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 1 Jun 2026 17:57:15 -0400 (EDT) Date: Mon, 1 Jun 2026 17:57:15 -0400 From: Andres Freund To: Nazir Bilal Yavuz Cc: Jacob Champion , Jelte Fennema-Nio , Thomas Munro , pgsql-hackers@postgresql.org, Zsolt Parragi , Peter Eisentraut Subject: Re: Heads Up: cirrus-ci is shutting down June 1st Message-ID: References: <3ydjipcr7kbss57nvi67noplncqhesl5eyb6wgol4ccjxynspv@yatlykpribmm> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="rv3g7aw7ud36z5b5" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --rv3g7aw7ud36z5b5 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, Attached is an large incremental patch onto Bilal's version: - I hacked enough on the pg-vm-images repo to make it store containers in github (on the gha_main branch, for now). That makes the container faster and cheaper to retrieve. We might want to split the containers further (e.g. cross building and 32bit support), but for now I just split the docs stuff into a separate container. - The reason that sometimes cancelling took a long time was, afaict, the use of always() in the compiler-warning job. That apparently prevents GHA from cancelling the job in a timely fashion. I turned those into !cancelled(). Also addded a !cancelled() to SanityCheck. - In the end I didn't like the matrix all that much, makes it considerably harder to understand everything and the restrictions GHA puts on it are just too annoying. I replaced it much more heavy use of yaml anchors/aliases and by updating the environment programattically. I'm not sure how much better it is now. - The macports cache keys included the run_id, I think that's a bad idea, because it leads to the cache being newly uploaded even if there's been no change. That'll lead to even more quickly churning through the cache space. - ccache: - There was too much duplication around the ccache handling for my taste. I moved that into a yaml anchor and reused it everywhere. By using ${{github.job_id}} the cache names don't need to be manually disambiguated. - The ccache names weren't unique enough. The run_id doesn't change during reruns which would lead to warnings. - Made it so that the cache is saved immediately after the build, so that cancelled builds still save the cache. - I wanted to share commands like meson test between the tasks, made that work with a bit of hackery. - I didn't see why find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; was needed. - I found it hard to find actual warnings in the output of CompilerWarnings, due to to all the configure output. I added some magic output stuff to make the configure output collapse once it ran. - I converted most things to use ${{env.varname}} instead of $VARNAME or %VARNAME%, as that allows sharing code between windows and other OSs if one is a bit careful. - Deduplicated a few other things like logging. - Added a commit removing cirrus, mainly because I was tired of it also running while hacking on this. - Also added a commit to reduce the segment size in tests to 1MB, that makes them a decent bit less IO intensive. - Added a commit to just run regress/regress, makes it a lot faster to test "complete-ish" cycles. - Changed sanitizer using builds to use -O2, to reduce the CPU cost a bit. This makes uncached builds noticeably slower, but does appear to be a win. But I could see counter-arguments to that too. - Removed :detect_stack_use_after_return=0, as that's been made unnecessary since you "forked off" from .cirrus.tasks.yml. A few other comments: - All the caches in GHA apparently are branch specific, except that branches can access the caches of the main branch. I think that'll make particularly macports very expensive for cfbot. I wonder if we ought to build the "base" macports cache in pg-vm-images. Similarly, I think we probably should do that for mingw64. - src/tools/ci/README needs updating - I experimented with making the slowest jobs faster by using meson test --slice 1/2 that actually does nicely work. But it didn't quite seem critical for now. Here's a run running with just regress/regress: https://github.com/anarazel/postgres/actions/runs/26782558353 (the CompilerWarnings ccache hit had a miss, that's why it's not faster) Here's a (not yet completed) run with the full tests: https://github.com/anarazel/postgres/actions/runs/26784248887 Thoughts? Greetings, Andres Freund --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=utf-8 Content-Disposition: attachment; filename="v6a-0001-Add-GitHub-Actions-workflow-for-CI.patch" Content-Transfer-Encoding: 8bit From 844833b49963726af8832bcfeadd93e6564d3f75 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz Date: Thu, 28 May 2026 19:31:34 +0300 Subject: [PATCH v6a 1/5] Add GitHub Actions workflow for CI Cirrus CI, which the project has used to run CI, is shutting down on June 1, 2026. Replace it with a GitHub Actions. Github Actions is selected because it has unlimited runner time for public repositories. Github Action currently covers: - SanityCheck - Linux (Autoconf) - Linux (Meson, 32- and 64-bit) - macOS (Meson) - Windows (Visual Studio + Meson and MinGW + Meson) - CompilerWarnings BSD coverage is left for later, as it requires more work. Back-branches will be updated later, after being sure that workflow runs correctly on master. Author: Jelte Fennema-Nio Author: Nazir Bilal Yavuz Reviewed-by: Jacob Champion Reviewed-by: Peter Eisentraut Reviewed-by: Andres Freund Reviewed-by: Zsolt Parragi Discussion: https://postgr.es/m/3ydjipcr7kbss57nvi67noplncqhesl5eyb6wgol4ccjxynspv%40yatlykpribmm --- .github/workflows/postgresql-ci.yml | 1026 ++++++++++++++++++++++++++ src/tools/ci/ci_macports_packages.sh | 19 +- 2 files changed, 1042 insertions(+), 3 deletions(-) create mode 100644 .github/workflows/postgresql-ci.yml diff --git a/.github/workflows/postgresql-ci.yml b/.github/workflows/postgresql-ci.yml new file mode 100644 index 00000000000..a7ef0bee94d --- /dev/null +++ b/.github/workflows/postgresql-ci.yml @@ -0,0 +1,1026 @@ +# GitHub Actions CI configuration for PostgreSQL + +name: GitHub Actions CI + +on: + push: + +# Restrict GITHUB_TOKEN to the minimum the jobs need: reading repo +# contents during checkout. +permissions: + contents: read + +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + # Never cancel in-progress runs on master to ensure all commits are tested. + cancel-in-progress: ${{ github.ref != 'refs/heads/master' }} + +env: + # The lower depth accelerates git clone. Use a bit of depth so that + # concurrent jobs and retrying older runs have a chance of working. + CLONE_DEPTH: 500 + + CCACHE_MAXSIZE: "250M" + + # check target for the autoconf builds + CHECK: check-world PROVE_FLAGS=--timer + CHECKFLAGS: -Otarget + + # Build test dependencies as part of the build step, to see compiler + # errors/warnings in one place. + MBUILD_TARGET: all testprep + MTEST_ARGS: --print-errorlogs --no-rebuild -C build + PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests + TEMP_CONFIG: ${{ github.workspace }}/src/tools/ci/pg_ci_base.conf + PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth + + # Postgres config args for the meson builds, shared between all meson tasks + # except the 'SanityCheck' task + MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true + + # Meson feature flags shared by all meson tasks, except: + # SanityCheck: uses almost no dependencies. + # Windows - VS: has fewer dependencies than listed here, so defines its own. + # Linux: uses the 'auto' feature option to test meson feature autodetection. + MESON_COMMON_FEATURES: >- + -Dauto_features=disabled + -Dldap=enabled + -Dssl=openssl + -Dtap_tests=enabled + -Dplperl=enabled + -Dplpython=enabled + -Ddocs=enabled + -Dicu=enabled + -Dlibxml=enabled + -Dlibxslt=enabled + -Dlz4=enabled + -Dpltcl=enabled + -Dreadline=enabled + -Dzlib=enabled + -Dzstd=enabled + + # Shared between the Linux autoconf job and the CompilerWarnings jobs + LINUX_CONFIGURE_FEATURES: >- + --with-gssapi + --with-icu + --with-ldap + --with-libcurl + --with-libxml + --with-libxslt + --with-llvm + --with-lz4 + --with-pam + --with-perl + --with-python + --with-selinux + --with-ssl=openssl + --with-systemd + --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ + --with-uuid=ossp + --with-zstd + + # Debian Trixie container image used by all Linux jobs. Built by + # 'https://github.com/anarazel/pg-vm-images/'. + LINUX_CI_IMAGE: us-docker.pkg.dev/pg-ci-images/ci/linux_debian_trixie_ci:latest + + # The full set of OS / job selectors recognized by the `ci-os-only:` + # commit-message directive parsed in the `setup` job below. + CI_OS_ONLY_JOBS: "linux macos windows mingw compilerwarnings sanitycheck" + + _LOG_PATHS: &log_paths | + build*/testrun/**/*.log + build*/testrun/**/*.diffs + build*/testrun/**/regress_log_* + build*/meson-logs/*.txt + + +jobs: + # Parse "ci-os-only: ..." from the commit message and expose flags + # consumed by the jobs' `if:` conditions. + setup: + name: Determine enabled jobs + runs-on: ubuntu-latest + timeout-minutes: 1 + outputs: + linux: ${{ steps.os.outputs.linux }} + macos: ${{ steps.os.outputs.macos }} + windows: ${{ steps.os.outputs.windows }} + mingw: ${{ steps.os.outputs.mingw }} + compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} + sanitycheck: ${{ steps.os.outputs.sanitycheck }} + # Re-export workflow-level env vars that other jobs need to reference + # from contexts (e.g. `jobs..container.image`) where the `env` + # context is not available. + linux_ci_image: ${{ env.LINUX_CI_IMAGE }} + steps: + - id: os + env: + MSG: ${{ github.event.head_commit.message }} + shell: bash + run: | + set -e + all_os=${CI_OS_ONLY_JOBS} + if printf '%s\n' "$MSG" | grep -qE '^ci-os-only: '; then + sel=$(printf '%s\n' "$MSG" | sed -n 's/^ci-os-only: //p' | head -n 1) + echo "ci-os-only selection: $sel" + else + sel="$all_os" + fi + for o in $all_os; do + if echo " $sel " | grep -qE "[ ,]$o[ ,]"; then + echo "$o=true" >> "$GITHUB_OUTPUT" + else + echo "$o=false" >> "$GITHUB_OUTPUT" + fi + done + cat "$GITHUB_OUTPUT" + + + # To avoid unnecessarily spinning up a lot of VMs / containers for entirely + # broken commits, have a minimal task that all others depend on. + # + # SPECIAL: + # - Builds with --auto-features=disabled and thus almost no enabled + # dependencies + sanity-check: + name: SanityCheck + needs: setup + if: needs.setup.outputs.sanitycheck == 'true' + runs-on: ubuntu-latest + timeout-minutes: 15 + container: + image: ${{ needs.setup.outputs.linux_ci_image }} + # --privileged is needed so the prepare step can write to sysctls + # under /proc/sys (it's mounted read-only without it). We use it to + # set kernel.core_pattern. + options: --privileged + env: + BUILD_JOBS: 8 + TEST_JOBS: 8 + CCACHE_DIR: ${{ github.workspace }}/ccache_dir + # no options enabled, should be small + CCACHE_MAXSIZE: "150M" + steps: + # Anchor reused by other jobs further down. GitHub Actions supports + # YAML anchors/aliases but not merge keys, so the alias copies the + # whole step verbatim. The anchor is resolved at YAML parse time, so the + # alias keeps working even if this job is skipped at runtime. + - &checkout_step + uses: actions/checkout@v6 + with: + fetch-depth: ${{ env.CLONE_DEPTH }} + + - name: Restore ccache + uses: actions/cache@v5 + with: + path: ${{ env.CCACHE_DIR }} + key: ccache-sanitycheck-${{ github.ref_name }}-${{ github.run_id }} + restore-keys: | + ccache-sanitycheck-${{ github.ref_name }}- + ccache-sanitycheck- + + - name: Prepare workspace + run: | + whoami + useradd -m postgres + chown -R postgres:postgres . + mkdir -p "$CCACHE_DIR" + chown -R postgres:postgres "$CCACHE_DIR" + + - name: Configure + run: | + su postgres <<-'EOF' + set -e + meson setup \ + --buildtype=debug \ + --auto-features=disabled \ + -Ddefault_library=shared \ + -Dtap_tests=enabled \ + build + EOF + + - name: Build + run: | + su postgres <> /etc/hosts <<-EOF + 127.0.0.1 pg-loadbalancetest + 127.0.0.2 pg-loadbalancetest + 127.0.0.3 pg-loadbalancetest + EOF + + - name: Configure + run: | + su postgres <- + -Dbonjour=enabled + -Ddtrace=enabled + -Dgssapi=enabled + -Dlibcurl=enabled + -Dnls=enabled + -Duuid=e2fs + + MACOS_PACKAGE_LIST: >- + ccache + icu + kerberos5 + lz4 + meson + openldap + openssl + p5.34-io-tty + p5.34-ipc-run + python312 + tcl + zstd + + CC: ccache cc + CXX: ccache c++ + CFLAGS: -Og -ggdb + CXXFLAGS: -Og -ggdb + PG_TEST_PG_UPGRADE_MODE: --clone + PG_TEST_PG_COMBINEBACKUP_MODE: --clone + + # Several buildfarm animals enable these options. Without testing them + # during CI, it would be easy to cause breakage on the buildfarm with CI + # passing. + PG_TEST_INITDB_EXTRA_OPTS: >- + -c debug_copy_parse_plan_trees=on + -c debug_write_read_parse_plan_trees=on + -c debug_raw_expression_coverage_test=on + -c debug_parallel_query=regress + + steps: + - *checkout_step + + - name: Sysinfo + run: | + id + uname -a + ulimit -a -H && ulimit -a -S + env + + - name: Setup core files + run: | + mkdir -p $HOME/cores + sudo sysctl kern.corefile="$HOME/cores/core.%P" + + - name: Restore ccache + uses: actions/cache@v5 + with: + path: ${{ env.CCACHE_DIR }} + key: ccache-macos-${{ github.ref_name }}-${{ github.run_id }} + restore-keys: | + ccache-macos-${{ github.ref_name }}- + ccache-macos- + + - name: Compute MacPorts cache key + id: mpkey + run: | + macos_major=$(sw_vers -productVersion | sed 's/\..*//') + pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) + script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) + echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}-${GITHUB_RUN_ID}" >> "$GITHUB_OUTPUT" + echo "restore-key=macports-${macos_major}-${pkglist_hash}-${script_hash}-" >> "$GITHUB_OUTPUT" + + - name: Restore MacPorts cache + uses: actions/cache@v5 + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mpkey.outputs.key }} + restore-keys: ${{ steps.mpkey.outputs.restore-key }} + + # Use MacPorts, even though Homebrew is installed. The installation + # of the additional packages we need would take quite a while with + # Homebrew, even if we cache the downloads. We can't cache all of + # Homebrew, because it's already large. So we use MacPorts. To cache + # the installation we create a .dmg file that we mount if it already + # exists. + # XXX: The reason for the direct p5.34* references is that we'd need + # the large MacPort tree around to figure out that p5-io-tty is + # actually p5.34-io-tty. Using the unversioned name works, but + # updates MacPorts every time. + - name: Install dependencies (MacPorts) + env: + # Pass token so the script's GitHub API call to list MacPorts + # releases isn't subject to the 60/h/IP unauthenticated rate + # limit (shared across all jobs on the runner's IP). + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST + # system python doesn't provide headers + sudo /opt/local/bin/port select python3 python312 + # Make macports install visible to subsequent steps + echo /opt/local/sbin >> "$GITHUB_PATH" + echo /opt/local/bin >> "$GITHUB_PATH" + + - name: Configure + run: | + export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" + meson setup \ + ${MESON_COMMON_PG_CONFIG_ARGS} \ + --buildtype=debug \ + -Dextra_include_dirs=/opt/local/include \ + -Dextra_lib_dirs=/opt/local/lib \ + -Ddarwin_sysroot=none \ + ${MESON_COMMON_FEATURES} \ + ${MESON_FEATURES} \ + build + + - name: Build + run: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} + + - name: Test world + run: | + ulimit -c unlimited # default is 0 + ulimit -n 1024 # default is 256, pretty low + meson test ${MTEST_ARGS} --num-processes ${TEST_JOBS} + + - name: Core backtraces + if: failure() + run: src/tools/ci/cores_backtrace.sh macos "$HOME/cores" + + - name: Upload logs + if: failure() + uses: actions/upload-artifact@v7 + with: + name: macos-logs-${{ github.run_id }} + path: *log_paths + if-no-files-found: ignore + + + windows-vs: + name: Windows - VS - Meson & ninja + needs: [setup, sanity-check] + if: | + !cancelled() && + needs.setup.outputs.windows == 'true' && + needs.sanity-check.result != 'failure' + runs-on: windows-2022 + timeout-minutes: 60 + env: + TEST_JOBS: 8 + # Avoid port conflicts between concurrent tap tests + PG_TEST_USE_UNIX_SOCKETS: 1 + PG_REGRESS_SOCK_DIR: 'd:\pgsock' + + MESON_FEATURES: >- + -Dcpp_args=/std:c++20 + -Dauto_features=disabled + -Dtap_tests=enabled + -Dldap=enabled + -Dssl=openssl + -Dplperl=enabled + -Dplpython=enabled + TAR: "c:/windows/system32/tar.exe" + + defaults: + run: + shell: cmd + steps: + - name: Disable Windows Defender + shell: powershell + run: | + Set-MpPreference -DisableRealtimeMonitoring $true -SubmitSamplesConsent NeverSend -MAPSReporting Disable + # Verify Defender status + $status = Get-MpComputerStatus -ErrorAction SilentlyContinue + if ($status) { + Write-Host "RealTimeProtectionEnabled: $($status.RealTimeProtectionEnabled)" + Write-Host "AntivirusEnabled: $($status.AntivirusEnabled)" + } + + - *checkout_step + + - name: Sysinfo + run: | + chcp + systeminfo + set + + # The TAP tests build an initdb template under build/tmp_install and + # then `robocopy` it into per-test data directories. Robocopy with the + # default /COPY:DAT flag doesn't copy ACLs — destinations inherit from + # their parent dir. On GitHub-hosted Windows runners the workspace's + # inherited ACL grants Administrators:(F) and Users:(RX) but does NOT + # grant the runner user (runneradmin) directly. That matters because + # pg_ctl on Windows uses CreateRestrictedProcess to drop admin + # privileges from postmaster, so the postmaster process has the user + # SID in its token but no longer the Administrators group — leaving it + # with only "Users:(RX)" on pg_control and friends, which causes + # "PANIC: could not open file global/pg_control: Permission denied". + # + # Fix it once on the workspace dir with (OI)(CI) inheritance flags so + # every file/dir created underneath gets an explicit grant for the + # current user. + - name: Grant workspace ACL to runner user + shell: pwsh + run: | + icacls "${{ github.workspace }}" /grant "${env:USERNAME}:(OI)(CI)F" /Q | Out-Null + Write-Host "Granted Full Control to $env:USERNAME on ${{ github.workspace }}" + + # postgres' plpython3u loads python3.dll (the stable-ABI forwarder) + # which in turn loads whichever python3NN.dll the Windows loader finds + # first on PATH. On windows-2022 `C:\Program Files\Mercurial\` ships + # its own python3.dll + python39.dll and appears on PATH *before* the + # hostedtoolcache Python 3.12 — so without intervention the backend + # ends up running Python 3.9 while postgres' stdlib search uses 3.12, + # producing `ImportError: cannot import name 'text_encoding' from + # 'io'` (the 3.12 `io.py` calling into 3.9's `_io`). + # + # Drop Mercurial's directory from PATH so the hostedtoolcache + # python3.dll wins the DLL search. + - name: Remove Mercurial from PATH + shell: pwsh + run: | + $filtered = ($env:PATH -split ';' | + Where-Object { $_ -and ($_ -notmatch '\\Mercurial\\?$') }) -join ';' + Add-Content $env:GITHUB_ENV "PATH=$filtered" + Write-Host "Removed Mercurial entries from PATH" + + - name: Install dependencies + shell: pwsh + run: | + choco install -y --no-progress --limitoutput diffutils winflexbison3 + # meson + ninja aren't preinstalled on windows-2022. Install via pip + python -m pip install --upgrade meson ninja + + # OpenSSL 1.1 via the slproweb installer (pinned to match the + # version used elsewhere in postgres CI). + curl.exe -fsSL -o openssl-setup.exe https://slproweb.com/download/Win64OpenSSL-1_1_1w.exe + Start-Process -Wait -FilePath ./openssl-setup.exe ` + -ArgumentList '/DIR=d:\openssl\1.1\ /VERYSILENT /SP- /SUPPRESSMSGBOXES' + # The slproweb installer puts libcrypto-1_1-x64.dll / libssl-1_1-x64.dll + # in d:\openssl\1.1\bin\ and updates the system PATH. GH Actions + # snapshots PATH at job start though, so the running job won't + # see those DLLs and initdb.exe would crash silently at runtime. + # Push the bin dir onto GITHUB_PATH so it persists for later steps. + Add-Content $env:GITHUB_PATH "d:\openssl\1.1\bin" + + # Install IPC::Run. + # - recommends_policy=0 keeps cpan from pulling in IO::Tty / IO::Pty, + # which don't build on Windows ("This module requires a POSIX + # compliant system to work"). + # - Pin to NJM/IPC-Run-20250809.0 because TODDR/IPC-Run-20260322.0 + # broke postgres tap tests on Windows (changed pipe stdio + # handling). See upstream pg-vm-images commit ff5238afa3 and + # the thread at + # https://postgr.es/m/CAN55FZ06xanSbJdHe-CurjX_qNuBWZDEvS1kAk36L38YCtZXnw%40mail.gmail.com + "o conf recommends_policy 0`no conf commit`nnotest install NJM/IPC-Run-20250809.0.tar.gz" | cpan + perl -mIPC::Run -e 1 + + - name: Setup hosts file + shell: pwsh + run: | + Add-Content c:\Windows\System32\Drivers\etc\hosts "127.0.0.1 pg-loadbalancetest" + Add-Content c:\Windows\System32\Drivers\etc\hosts "127.0.0.2 pg-loadbalancetest" + Add-Content c:\Windows\System32\Drivers\etc\hosts "127.0.0.3 pg-loadbalancetest" + + - name: Setup socket directory + shell: cmd + run: mkdir %PG_REGRESS_SOCK_DIR% + + - name: Configure + run: | + call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64 + meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% %MESON_FEATURES% --buildtype debug -Db_pch=true -Dextra_lib_dirs=d:\openssl\1.1\lib -Dextra_include_dirs=d:\openssl\1.1\include -DTAR=%TAR% build + + - name: Build + run: | + call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64 + ninja -C build %MBUILD_TARGET% + ninja -C build -t missingdeps + + - name: Test world + run: | + call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64 + meson test %MTEST_ARGS% --num-processes %TEST_JOBS% + + # FIX: We need to collect crashlogs but they are not collected. cdb.exe + # is installed on the runner so it needs to be configured. + - name: Upload logs + if: failure() + uses: actions/upload-artifact@v7 + with: + name: windows-vs-logs-${{ github.run_id }} + path: *log_paths + if-no-files-found: ignore + + + windows-mingw: + name: Windows - MinGW - Meson + needs: [setup, sanity-check] + if: | + !cancelled() && + needs.setup.outputs.mingw == 'true' && + needs.sanity-check.result != 'failure' + runs-on: windows-2022 + timeout-minutes: 60 + env: + TEST_JOBS: 4 # higher concurrency causes occasional failures + PG_TEST_USE_UNIX_SOCKETS: 1 + PG_REGRESS_SOCK_DIR: 'd:\pgsock' + TAR: "c:/windows/system32/tar.exe" + + MSYS: winjitdebug + CHERE_INVOKING: 1 + MSYSTEM: UCRT64 + + # Keep -Dnls explicitly disabled, as the number of files it creates + # causes a noticeable slowdown. + MESON_FEATURES: >- + -Dnls=disabled + + CCACHE_DIR: D:/a/ccache + CCACHE_MAXSIZE: "500M" + CCACHE_SLOPPINESS: pch_defines,time_macros + CCACHE_DEPEND: 1 + + defaults: + run: + shell: 'D:\msys64\usr\bin\bash.exe --login -eo pipefail "{0}"' + + steps: + - name: Disable Windows Defender + shell: powershell + run: | + Set-MpPreference -DisableRealtimeMonitoring $true -SubmitSamplesConsent NeverSend -MAPSReporting Disable + # Verify Defender status + $status = Get-MpComputerStatus -ErrorAction SilentlyContinue + if ($status) { + Write-Host "RealTimeProtectionEnabled: $($status.RealTimeProtectionEnabled)" + Write-Host "AntivirusEnabled: $($status.AntivirusEnabled)" + } + + - *checkout_step + + # Relocate the preinstalled MSYS2 tree from C:\ (slow system disk) to + # D:\ (faster ephemeral data disk). Every subsequent MSYS2 step uses + # D:\msys64\usr\bin\bash.exe via the job's `defaults.run.shell`. + # + # This reduces the total runtime of this task by ~15 minutes. + # + # robocopy returns 0-7 on success (with various "files copied" bits + # set) and 8+ on real failure, so we have to translate its exit code. + - name: Relocate MSYS2 to D + shell: powershell + run: | + robocopy C:\msys64 D:\msys64 /E /MT:16 /NJS /NJH /NFL /NDL /NP + if ($LASTEXITCODE -ge 8) { exit $LASTEXITCODE } + exit 0 + + - name: Setup MSYS2 + run: | + # ${MINGW_PACKAGE_PREFIX} is an environment variable used in the + # MSYS2. It dynamically expands to the correct prefix for the active + # shell environment. + pacman -S --noconfirm --needed \ + git bison flex make diffutils \ + ${MINGW_PACKAGE_PREFIX}-ccache \ + ${MINGW_PACKAGE_PREFIX}-gcc \ + ${MINGW_PACKAGE_PREFIX}-icu \ + ${MINGW_PACKAGE_PREFIX}-libbacktrace \ + ${MINGW_PACKAGE_PREFIX}-libxml2 \ + ${MINGW_PACKAGE_PREFIX}-libxslt \ + ${MINGW_PACKAGE_PREFIX}-lz4 \ + ${MINGW_PACKAGE_PREFIX}-make \ + ${MINGW_PACKAGE_PREFIX}-meson \ + ${MINGW_PACKAGE_PREFIX}-perl \ + ${MINGW_PACKAGE_PREFIX}-pkg-config \ + ${MINGW_PACKAGE_PREFIX}-readline \ + ${MINGW_PACKAGE_PREFIX}-zlib + + - name: Install additional dependencies + run: | + # Pin IPC::Run to NJM/IPC-Run-20250809.0; TODDR/IPC-Run-20260322.0 + # broke postgres tap tests on Windows (pipe stdio handling). + # See pg-vm-images commit ff5238afa3. + (echo; echo o conf recommends_policy 0; echo notest install NJM/IPC-Run-20250809.0.tar.gz) | cpan + perl -mIPC::Run -e 1 + + - name: Setup socket directory + shell: cmd + run: mkdir %PG_REGRESS_SOCK_DIR% + + - name: Restore ccache + uses: actions/cache@v5 + with: + path: ${{ env.CCACHE_DIR }} + key: ccache-mingw-${{ github.ref_name }}-${{ github.run_id }} + restore-keys: | + ccache-mingw-${{ github.ref_name }}- + ccache-mingw- + + - name: Configure + run: | + meson setup \ + ${MESON_COMMON_PG_CONFIG_ARGS} \ + -Ddebug=true -Doptimization=g -Db_pch=true \ + ${MESON_COMMON_FEATURES} \ + ${MESON_FEATURES} \ + -DTAR=${TAR} \ + build + + - name: Build + run: ninja -C build ${MBUILD_TARGET} + + - name: Test world + run: meson test ${MTEST_ARGS} --num-processes ${TEST_JOBS} + + # FIX: We need to collect crashlogs but they are not collected. cdb.exe + # is installed on the runner so it needs to be configured. + - name: Upload logs + if: failure() + uses: actions/upload-artifact@v7 + with: + name: windows-mingw-logs-${{ github.run_id }} + path: *log_paths + if-no-files-found: ignore + + # Test that code can be built with both gcc and clang without warnings, + # with various combinations of cassert/dtrace flags. Trace probes have + # a history of getting accidentally broken; the matrix is there to + # catch that. + # + # The autoconf cache files (gcc.cache / clang.cache) are intentionally + # reused across the matrix entries that share a compiler, so we don't + # pay for full feature detection on every entry. + compiler-warnings: + name: CompilerWarnings + needs: [setup, sanity-check] + if: | + !cancelled() && + needs.setup.outputs.compilerwarnings == 'true' && + needs.sanity-check.result != 'failure' + runs-on: ubuntu-latest + timeout-minutes: 60 + container: + image: ${{ needs.setup.outputs.linux_ci_image }} + env: + BUILD_JOBS: 4 + CCACHE_DIR: /tmp/ccache_dir + # Use larger ccache cache as this job compiles with multiple + # compilers / flag combinations. + CCACHE_MAXSIZE: "1G" + steps: + - *checkout_step + + - name: Restore ccache + uses: actions/cache@v5 + with: + path: ${{ env.CCACHE_DIR }} + key: ccache-compiler-warnings-${{ github.ref_name }}-${{ github.run_id }} + restore-keys: | + ccache-compiler-warnings-${{ github.ref_name }}- + ccache-compiler-warnings- + + - name: Sysinfo + run: | + id + uname -a + cat /proc/cmdline + ulimit -a -H && ulimit -a -S + gcc -v + clang -v + env + + - name: Setup workspace + run: | + echo "COPT=-Werror" > src/Makefile.custom + mkdir -p "$CCACHE_DIR" + + # gcc, cassert off, dtrace on + - name: gcc warnings + (dtrace) + if: always() + run: | + ./configure \ + --cache gcc.cache \ + --enable-dtrace \ + ${LINUX_CONFIGURE_FEATURES} \ + CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" + make -s -j${BUILD_JOBS} clean + make -s -j${BUILD_JOBS} world-bin + + # gcc, cassert on, dtrace off + - name: gcc warnings + (cassert) + if: always() + run: | + ./configure \ + --cache gcc.cache \ + --enable-cassert \ + ${LINUX_CONFIGURE_FEATURES} \ + CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" + make -s -j${BUILD_JOBS} clean + make -s -j${BUILD_JOBS} world-bin + + # clang, cassert off, dtrace off + - name: clang warnings + if: always() + run: | + ./configure \ + --cache clang.cache \ + ${LINUX_CONFIGURE_FEATURES} \ + CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" + make -s -j${BUILD_JOBS} clean + make -s -j${BUILD_JOBS} world-bin + + # clang, cassert on, dtrace on + - name: clang warnings + (cassert + dtrace) + if: always() + run: | + ./configure \ + --cache clang.cache \ + --enable-cassert \ + --enable-dtrace \ + ${LINUX_CONFIGURE_FEATURES} \ + CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" + make -s -j${BUILD_JOBS} clean + make -s -j${BUILD_JOBS} world-bin + + - name: mingw warnings (cross compilation) + if: always() + run: | + ./configure \ + --host=x86_64-w64-mingw32ucrt \ + --enable-cassert \ + --without-icu \ + CC="ccache x86_64-w64-mingw32ucrt-gcc" \ + CXX="ccache x86_64-w64-mingw32ucrt-g++" + make -s -j${BUILD_JOBS} clean + make -s -j${BUILD_JOBS} world-bin + + ### + # Verify docs can be built + ### + # XXX: Only do this if there have been changes in doc/ since last build + - name: Build documentation + if: always() + run: | + ./configure \ + --cache gcc.cache \ + CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" + make -s -j${BUILD_JOBS} clean + make -s -j${BUILD_JOBS} -C doc + + ### + # Verify headerscheck / cpluspluscheck succeed + # + # - Run both in same script to increase parallelism, use -k to get + # result of both + # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose + ### + - name: headerscheck + cpluspluscheck + if: always() + run: | + ./configure \ + ${LINUX_CONFIGURE_FEATURES} \ + --cache gcc.cache \ + --quiet \ + CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" + make -s -j${BUILD_JOBS} clean + make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' diff --git a/src/tools/ci/ci_macports_packages.sh b/src/tools/ci/ci_macports_packages.sh index 63e97b37c78..18a06f96119 100755 --- a/src/tools/ci/ci_macports_packages.sh +++ b/src/tools/ci/ci_macports_packages.sh @@ -20,13 +20,26 @@ echo "macOS major version: $macos_major_version" # macOS release. macports_release_list_url="https://api.github.com/repos/macports/macports-base/releases" macports_version_pattern="2\.10\.1" -macports_url="$( curl -s $macports_release_list_url | grep "\"https://github.com/macports/macports-base/releases/download/v$macports_version_pattern/MacPorts-$macports_version_pattern-$macos_major_version-[A-Za-z]*\.pkg\"" | sed 's/.*: "//;s/".*//' | head -1 )" +# Authenticate the GitHub API request when a token is available (e.g. on +# GitHub Actions). Unauthenticated requests share a 60/h/IP rate limit +# with every other job on the runner's IP and frequently return an error +# JSON, leaving $macports_url empty and breaking the subsequent curl. +auth_header="" +if [ -n "$GITHUB_TOKEN" ]; then + auth_header="Authorization: Bearer $GITHUB_TOKEN" +fi +macports_url="$( curl -fsSL ${auth_header:+-H "$auth_header"} "$macports_release_list_url" | grep "\"https://github.com/macports/macports-base/releases/download/v$macports_version_pattern/MacPorts-$macports_version_pattern-$macos_major_version-[A-Za-z]*\.pkg\"" | sed 's/.*: "//;s/".*//' | head -1 )" echo "MacPorts package URL: $macports_url" +if [ -z "$macports_url" ]; then + echo "error: could not determine MacPorts package URL for macOS $macos_major_version (version pattern: $macports_version_pattern)" 1>&2 + exit 1 +fi + cache_dmg="macports.hfs.dmg" -if [ "$CIRRUS_CI" != "true" ]; then - echo "expect to be called within cirrus-ci" 1>2 +if [ "$CIRRUS_CI" != "true" ] && [ "$GITHUB_ACTIONS" != "true" ]; then + echo "expect to be called within cirrus-ci or github actions" 1>2 exit 1 fi -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0002-gha-Andres-revisions.patch" From 828954a35ae25ba9331834eeb19ce42a3ed17a3f Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Mon, 1 Jun 2026 15:09:49 -0400 Subject: [PATCH v6a 2/5] gha: Andres' revisions --- .github/workflows/postgresql-ci.yml | 834 +++++++++++++++------------- 1 file changed, 447 insertions(+), 387 deletions(-) diff --git a/.github/workflows/postgresql-ci.yml b/.github/workflows/postgresql-ci.yml index a7ef0bee94d..e2795ca0ffb 100644 --- a/.github/workflows/postgresql-ci.yml +++ b/.github/workflows/postgresql-ci.yml @@ -4,6 +4,7 @@ name: GitHub Actions CI on: push: + # FIXME: Should we also run on PRs? # Restrict GITHUB_TOKEN to the minimum the jobs need: reading repo # contents during checkout. @@ -13,6 +14,7 @@ permissions: concurrency: group: ${{ github.workflow }}-${{ github.ref }} # Never cancel in-progress runs on master to ensure all commits are tested. + # FIXME: Should also not cancel REL_XY_STABLE cancel-in-progress: ${{ github.ref != 'refs/heads/master' }} env: @@ -20,9 +22,19 @@ env: # concurrent jobs and retrying older runs have a chance of working. CLONE_DEPTH: 500 + # At the moment all jobs use 4vcore runners, and none seems to benefit from + # increasing concurrency further. + BUILD_JOBS: 4 + + # It's possible that some jobs benefit from an increased test concurrency, + # but a default of 4 is a safe bet. Individual jobs can override. + TEST_JOBS: 4 + CCACHE_MAXSIZE: "250M" + CCACHE_DIR: ${{ github.workspace }}/ccache_dir - # check target for the autoconf builds + # Check target for the autoconf builds. Can be set to e.g. check to only + # only test the main regression tests. CHECK: check-world PROVE_FLAGS=--timer CHECKFLAGS: -Otarget @@ -30,6 +42,11 @@ env: # errors/warnings in one place. MBUILD_TARGET: all testprep MTEST_ARGS: --print-errorlogs --no-rebuild -C build + + # Can be set to a non-empty value to run a limited set of tests + # (e.g. --suite regress to only run the main regression tests). + MTEST_TARGET: + PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests TEMP_CONFIG: ${{ github.workspace }}/src/tools/ci/pg_ci_base.conf PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth @@ -79,20 +96,16 @@ env: --with-uuid=ossp --with-zstd - # Debian Trixie container image used by all Linux jobs. Built by + # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/'. - LINUX_CI_IMAGE: us-docker.pkg.dev/pg-ci-images/ci/linux_debian_trixie_ci:latest + CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/gha_main + CONTAINER_LINUX_CI: linux_debian_trixie_ci:latest + CONTAINER_LINUX_CI_DOCS: linux_debian_trixie_ci_docs:latest # The full set of OS / job selectors recognized by the `ci-os-only:` # commit-message directive parsed in the `setup` job below. CI_OS_ONLY_JOBS: "linux macos windows mingw compilerwarnings sanitycheck" - _LOG_PATHS: &log_paths | - build*/testrun/**/*.log - build*/testrun/**/*.diffs - build*/testrun/**/regress_log_* - build*/meson-logs/*.txt - jobs: # Parse "ci-os-only: ..." from the commit message and expose flags @@ -111,14 +124,26 @@ jobs: # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs..container.image`) where the `env` # context is not available. - linux_ci_image: ${{ env.LINUX_CI_IMAGE }} + container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} + container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} steps: + # Anchor reused by other jobs further down. GitHub Actions supports YAML + # anchors/aliases but not merge keys, so the alias copies the whole step + # verbatim. The anchor is resolved at YAML parse time, so the alias + # keeps working even if this job were to be skipped at runtime. + - &nix_sysinfo_step + name: sysinfo + run: | + id + uname -a + ulimit -a -H && ulimit -a -S + env + - id: os env: MSG: ${{ github.event.head_commit.message }} shell: bash run: | - set -e all_os=${CI_OS_ONLY_JOBS} if printf '%s\n' "$MSG" | grep -qE '^ci-os-only: '; then sel=$(printf '%s\n' "$MSG" | sed -n 's/^ci-os-only: //p' | head -n 1) @@ -145,199 +170,22 @@ jobs: sanity-check: name: SanityCheck needs: setup - if: needs.setup.outputs.sanitycheck == 'true' + if: | + !cancelled() && + needs.setup.outputs.sanitycheck == 'true' runs-on: ubuntu-latest timeout-minutes: 15 - container: - image: ${{ needs.setup.outputs.linux_ci_image }} + container: &linux_ci_container + image: ${{ needs.setup.outputs.container_linux_ci }} + + # Options passed to all linux containers. Not all of the jobs need + # all of them, but it's easier to just define them centrally. + # # --privileged is needed so the prepare step can write to sysctls # under /proc/sys (it's mounted read-only without it). We use it to - # set kernel.core_pattern. - options: --privileged - env: - BUILD_JOBS: 8 - TEST_JOBS: 8 - CCACHE_DIR: ${{ github.workspace }}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - steps: - # Anchor reused by other jobs further down. GitHub Actions supports - # YAML anchors/aliases but not merge keys, so the alias copies the - # whole step verbatim. The anchor is resolved at YAML parse time, so the - # alias keeps working even if this job is skipped at runtime. - - &checkout_step - uses: actions/checkout@v6 - with: - fetch-depth: ${{ env.CLONE_DEPTH }} - - - name: Restore ccache - uses: actions/cache@v5 - with: - path: ${{ env.CCACHE_DIR }} - key: ccache-sanitycheck-${{ github.ref_name }}-${{ github.run_id }} - restore-keys: | - ccache-sanitycheck-${{ github.ref_name }}- - ccache-sanitycheck- - - - name: Prepare workspace - run: | - whoami - useradd -m postgres - chown -R postgres:postgres . - mkdir -p "$CCACHE_DIR" - chown -R postgres:postgres "$CCACHE_DIR" - - - name: Configure - run: | - su postgres <<-'EOF' - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - - - name: Build - run: | - su postgres <> "$GITHUB_ENV" + echo "CXXFLAGS=$CXXFLAGS ${SANITIZER_FLAGS}" >> "$GITHUB_ENV" + echo "LDFLAGS=$LDFLAGS ${SANITIZER_FLAGS}" >> "$GITHUB_ENV" + + echo "CC=${CC}" >> "$GITHUB_ENV" + echo "CXX=${CXX}" >> "$GITHUB_ENV" + + echo "PG_TEST_INITDB_EXTRA_OPTS=${PG_TEST_INITDB_EXTRA_OPTS}" >> "$GITHUB_ENV" + echo "PG_TEST_PG_COMBINEBACKUP_MODE=${PG_TEST_PG_COMBINEBACKUP_MODE}" >> "$GITHUB_ENV" + + - *nix_sysinfo_step + - *checkout_step + - *ccache_restore_step + - *linux_prepare_workspace + + - name: Configure + shell: *su_postgres_shell + run: | + ./configure \ + --enable-cassert --enable-injection-points --enable-debug \ + --enable-tap-tests --enable-nls \ + --with-segsize-blocks=6 \ + --with-libnuma \ + --with-liburing \ + ${LINUX_CONFIGURE_FEATURES} + + - name: Build + shell: *su_postgres_shell + run: | + make -s -j${BUILD_JOBS} world-bin + + - *ccache_save_step + + - name: Test world + shell: *su_postgres_shell + run: | + make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} + + - *linux_collect_cores + - *upload_logs_step + + + # Linux Meson, 32 bit + # + # SPECIAL: + # - Uses undefined behaviour and alignment sanitizers, (sanitizer failures + # are typically printed in the server log) + # - Uses io_method=io_uring + # - Uses meson feature autodetection + # - tests with LANG=C to give ICU some buildfarm-uncovered coverage. Also, + # newer Python insists on changing LC_CTYPE away from C, prevent that with + # PYTHONCOERCECLOCALE. + linux-meson-32: + name: Linux - Meson (32-bit) + needs: [setup, sanity-check] + if: *linux_job_if + runs-on: ubuntu-latest + container: *linux_ci_container + timeout-minutes: 60 + env: *linux_env + + steps: + - name: Update Environment + env: + SANITIZER_FLAGS: -fsanitize=alignment,undefined + PG_TEST_INITDB_EXTRA_OPTS: -c io_method=io_uring + CC: ccache gcc -m32 + CXX: ccache g++ -m32 + run: *linux_update_config_cmd + + - *nix_sysinfo_step + - *checkout_step + - *ccache_restore_step + - *linux_prepare_workspace + + - name: Configure + shell: *su_postgres_shell + run: | + meson setup \ + ${MESON_COMMON_PG_CONFIG_ARGS} \ + -Duuid=e2fs \ + --buildtype=debug \ + --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ + -DPERL=perl5.40-i386-linux-gnu \ + -Dlibnuma=disabled \ + build + + - name: Build + shell: *su_postgres_shell + run: *ninja_build_command + + - *ccache_save_step + + - name: Test world + shell: *su_postgres_shell + env: + PYTHONCOERCECLOCALE: 0 + LANG: C + run: *meson_test_world_cmd + + - *linux_collect_cores + - *upload_logs_step + + # Linux Meson, 64 bit + # + # SPECIAL: + # - Uses undefined behaviour and alignment sanitizers, (sanitizer failures + # are typically printed in the server log) + # - Uses io_method=io_uring + # - Uses meson feature autodetection + linux-meson-64: + name: Linux - Meson (64-bit) + needs: [setup, sanity-check] + if: *linux_job_if + runs-on: ubuntu-latest + container: *linux_ci_container + timeout-minutes: 60 + env: *linux_env + + steps: + - name: Update Environment + env: + SANITIZER_FLAGS: -fsanitize=alignment,undefined + PG_TEST_INITDB_EXTRA_OPTS: -c io_method=io_uring + run: *linux_update_config_cmd + + - *nix_sysinfo_step + - *checkout_step + - *ccache_restore_step + - *linux_prepare_workspace + + - name: Configure + shell: *su_postgres_shell + run: | + meson setup \ + ${MESON_COMMON_PG_CONFIG_ARGS} \ + -Duuid=e2fs \ + --buildtype=debug \ + -Dllvm=enabled \ + build + + - name: Build + shell: *su_postgres_shell + run: *ninja_build_command + + - *ccache_save_step + + - name: Test world + shell: *su_postgres_shell + run: *meson_test_world_cmd + + - *linux_collect_cores + - *upload_logs_step + + # SPECIAL: # - Enables --clone for pg_upgrade and pg_combinebackup # - Specifies configuration options that test reading/writing/copying of node trees @@ -449,13 +529,6 @@ jobs: runs-on: macos-15 timeout-minutes: 60 env: - BUILD_JOBS: 4 - # Test performance regresses noticeably when using all cores. 8 works OK. - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - # Fix: Needs to be re-tested for GitHub Actions. - TEST_JOBS: 8 - - CCACHE_DIR: ${{ github.workspace }}/ccache_dir MACPORTS_CACHE: ${{ github.workspace }}/macports-cache MESON_FEATURES: >- @@ -497,44 +570,28 @@ jobs: -c debug_parallel_query=regress steps: - - *checkout_step + - *nix_sysinfo_step - - name: Sysinfo - run: | - id - uname -a - ulimit -a -H && ulimit -a -S - env + - *checkout_step - name: Setup core files run: | mkdir -p $HOME/cores sudo sysctl kern.corefile="$HOME/cores/core.%P" - - name: Restore ccache - uses: actions/cache@v5 - with: - path: ${{ env.CCACHE_DIR }} - key: ccache-macos-${{ github.ref_name }}-${{ github.run_id }} - restore-keys: | - ccache-macos-${{ github.ref_name }}- - ccache-macos- - - - name: Compute MacPorts cache key + - name: "Macports: Compute cache key" id: mpkey run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) - echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}-${GITHUB_RUN_ID}" >> "$GITHUB_OUTPUT" - echo "restore-key=macports-${macos_major}-${pkglist_hash}-${script_hash}-" >> "$GITHUB_OUTPUT" + echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" - - name: Restore MacPorts cache + - name: "MacPorts: Restore cache" uses: actions/cache@v5 with: path: ${{ env.MACPORTS_CACHE }} key: ${{ steps.mpkey.outputs.key }} - restore-keys: ${{ steps.mpkey.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -546,7 +603,7 @@ jobs: # the large MacPort tree around to figure out that p5-io-tty is # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - - name: Install dependencies (MacPorts) + - name: "MacPorts: Install dependencies" env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -560,11 +617,14 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + - *ccache_restore_step + - name: Configure + env: + PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ run: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ + ${{env.MESON_COMMON_PG_CONFIG_ARGS}} \ --buildtype=debug \ -Dextra_include_dirs=/opt/local/include \ -Dextra_lib_dirs=/opt/local/lib \ @@ -574,25 +634,21 @@ jobs: build - name: Build - run: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} + run: *ninja_build_command + + - *ccache_save_step - name: Test world - run: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test ${MTEST_ARGS} --num-processes ${TEST_JOBS} + env: + # default is 256, pretty low + ADDITIONAL_SETUP: ulimit -n 1024 + run: *meson_test_world_cmd - name: Core backtraces - if: failure() + if: failure() && !cancelled() run: src/tools/ci/cores_backtrace.sh macos "$HOME/cores" - - name: Upload logs - if: failure() - uses: actions/upload-artifact@v7 - with: - name: macos-logs-${{ github.run_id }} - path: *log_paths - if-no-files-found: ignore + - *upload_logs_step windows-vs: @@ -605,10 +661,10 @@ jobs: runs-on: windows-2022 timeout-minutes: 60 env: - TEST_JOBS: 8 # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 PG_REGRESS_SOCK_DIR: 'd:\pgsock' + TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- -Dcpp_args=/std:c++20 @@ -618,13 +674,13 @@ jobs: -Dssl=openssl -Dplperl=enabled -Dplpython=enabled - TAR: "c:/windows/system32/tar.exe" defaults: run: shell: cmd steps: - - name: Disable Windows Defender + - &windows_disable_defender + name: Disable Windows Defender shell: powershell run: | Set-MpPreference -DisableRealtimeMonitoring $true -SubmitSamplesConsent NeverSend -MAPSReporting Disable @@ -723,33 +779,36 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir %PG_REGRESS_SOCK_DIR% + run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} - name: Configure run: | call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% %MESON_FEATURES% --buildtype debug -Db_pch=true -Dextra_lib_dirs=d:\openssl\1.1\lib -Dextra_include_dirs=d:\openssl\1.1\include -DTAR=%TAR% build + meson setup ^ + --backend ninja ^ + ${{env.MESON_COMMON_PG_CONFIG_ARGS}} ^ + ${{env.MESON_FEATURES}} ^ + --buildtype debug ^ + -Db_pch=true ^ + -Dextra_lib_dirs=d:\openssl\1.1\lib -Dextra_include_dirs=d:\openssl\1.1\include ^ + -DTAR=${{env.TAR}} ^ + build - name: Build run: | call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64 - ninja -C build %MBUILD_TARGET% + ninja -C build ${{env.MBUILD_TARGET}} ninja -C build -t missingdeps - name: Test world - run: | - call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% + env: + ADDITIONAL_SETUP: | + call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64 + run: *meson_test_world_cmd # FIX: We need to collect crashlogs but they are not collected. cdb.exe # is installed on the runner so it needs to be configured. - - name: Upload logs - if: failure() - uses: actions/upload-artifact@v7 - with: - name: windows-vs-logs-${{ github.run_id }} - path: *log_paths - if-no-files-found: ignore + - *upload_logs_step windows-mingw: @@ -762,7 +821,7 @@ jobs: runs-on: windows-2022 timeout-minutes: 60 env: - TEST_JOBS: 4 # higher concurrency causes occasional failures + # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 PG_REGRESS_SOCK_DIR: 'd:\pgsock' TAR: "c:/windows/system32/tar.exe" @@ -776,7 +835,6 @@ jobs: MESON_FEATURES: >- -Dnls=disabled - CCACHE_DIR: D:/a/ccache CCACHE_MAXSIZE: "500M" CCACHE_SLOPPINESS: pch_defines,time_macros CCACHE_DEPEND: 1 @@ -786,17 +844,7 @@ jobs: shell: 'D:\msys64\usr\bin\bash.exe --login -eo pipefail "{0}"' steps: - - name: Disable Windows Defender - shell: powershell - run: | - Set-MpPreference -DisableRealtimeMonitoring $true -SubmitSamplesConsent NeverSend -MAPSReporting Disable - # Verify Defender status - $status = Get-MpComputerStatus -ErrorAction SilentlyContinue - if ($status) { - Write-Host "RealTimeProtectionEnabled: $($status.RealTimeProtectionEnabled)" - Write-Host "AntivirusEnabled: $($status.AntivirusEnabled)" - } - + - *windows_disable_defender - *checkout_step # Relocate the preinstalled MSYS2 tree from C:\ (slow system disk) to @@ -835,6 +883,8 @@ jobs: ${MINGW_PACKAGE_PREFIX}-readline \ ${MINGW_PACKAGE_PREFIX}-zlib + - *nix_sysinfo_step + - name: Install additional dependencies run: | # Pin IPC::Run to NJM/IPC-Run-20250809.0; TODDR/IPC-Run-20260322.0 @@ -845,42 +895,32 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir %PG_REGRESS_SOCK_DIR% + run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} - - name: Restore ccache - uses: actions/cache@v5 - with: - path: ${{ env.CCACHE_DIR }} - key: ccache-mingw-${{ github.ref_name }}-${{ github.run_id }} - restore-keys: | - ccache-mingw-${{ github.ref_name }}- - ccache-mingw- + - *ccache_restore_step - name: Configure run: | meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ + ${{env.MESON_COMMON_PG_CONFIG_ARGS}} \ -Ddebug=true -Doptimization=g -Db_pch=true \ - ${MESON_COMMON_FEATURES} \ - ${MESON_FEATURES} \ - -DTAR=${TAR} \ + ${{env.MESON_COMMON_FEATURES}} \ + ${{env.MESON_FEATURES}} \ + -DTAR=${{env.TAR}} \ build - name: Build - run: ninja -C build ${MBUILD_TARGET} + run: *ninja_build_command + + - *ccache_save_step - name: Test world - run: meson test ${MTEST_ARGS} --num-processes ${TEST_JOBS} + run: *meson_test_world_cmd # FIX: We need to collect crashlogs but they are not collected. cdb.exe # is installed on the runner so it needs to be configured. - - name: Upload logs - if: failure() - uses: actions/upload-artifact@v7 - with: - name: windows-mingw-logs-${{ github.run_id }} - path: *log_paths - if-no-files-found: ignore + - *upload_logs_step + # Test that code can be built with both gcc and clang without warnings, # with various combinations of cassert/dtrace flags. Trace probes have @@ -900,24 +940,12 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 60 container: - image: ${{ needs.setup.outputs.linux_ci_image }} + image: ${{ needs.setup.outputs.container_linux_ci_docs }} env: - BUILD_JOBS: 4 - CCACHE_DIR: /tmp/ccache_dir # Use larger ccache cache as this job compiles with multiple # compilers / flag combinations. CCACHE_MAXSIZE: "1G" steps: - - *checkout_step - - - name: Restore ccache - uses: actions/cache@v5 - with: - path: ${{ env.CCACHE_DIR }} - key: ccache-compiler-warnings-${{ github.ref_name }}-${{ github.run_id }} - restore-keys: | - ccache-compiler-warnings-${{ github.ref_name }}- - ccache-compiler-warnings- - name: Sysinfo run: | @@ -929,83 +957,108 @@ jobs: clang -v env + - *checkout_step + + - *ccache_restore_step + - name: Setup workspace run: | echo "COPT=-Werror" > src/Makefile.custom - mkdir -p "$CCACHE_DIR" # gcc, cassert off, dtrace on - name: gcc warnings + (dtrace) - if: always() + if: ${{ !cancelled() }} run: | + echo "::group::configure" ./configure \ --cache gcc.cache \ --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ + ${{env.LINUX_CONFIGURE_FEATURES}} \ CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - make -s -j${BUILD_JOBS} world-bin + echo "::endgroup::" + + make -s -j${{env.BUILD_JOBS}} clean + make -s -j${{env.BUILD_JOBS}} world-bin + # gcc, cassert on, dtrace off - name: gcc warnings + (cassert) - if: always() + if: ${{ !cancelled() }} run: | + echo "::group::configure" ./configure \ --cache gcc.cache \ --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ + ${{env.LINUX_CONFIGURE_FEATURES}} \ CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - make -s -j${BUILD_JOBS} world-bin + echo "::endgroup::" + + make -s -j${{env.BUILD_JOBS}} clean + make -s -j${{env.BUILD_JOBS}} world-bin # clang, cassert off, dtrace off - name: clang warnings - if: always() + if: ${{ !cancelled() }} run: | + echo "::group::configure" ./configure \ --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ + ${{env.LINUX_CONFIGURE_FEATURES}} \ CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - make -s -j${BUILD_JOBS} world-bin + echo "::endgroup::" + + make -s -j${{env.BUILD_JOBS}} clean + make -s -j${{env.BUILD_JOBS}} world-bin + # clang, cassert on, dtrace on - name: clang warnings + (cassert + dtrace) - if: always() + if: ${{ !cancelled() }} run: | + echo "::group::configure" ./configure \ --cache clang.cache \ --enable-cassert \ --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ + ${{env.LINUX_CONFIGURE_FEATURES}} \ CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - make -s -j${BUILD_JOBS} world-bin + echo "::endgroup::" + + make -s -j${{env.BUILD_JOBS}} clean + make -s -j${{env.BUILD_JOBS}} world-bin + - name: mingw warnings (cross compilation) - if: always() + if: ${{ !cancelled() }} run: | + echo "::group::configure" ./configure \ --host=x86_64-w64-mingw32ucrt \ --enable-cassert \ --without-icu \ CC="ccache x86_64-w64-mingw32ucrt-gcc" \ CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - make -s -j${BUILD_JOBS} world-bin + echo "::endgroup::" + + make -s -j${{env.BUILD_JOBS}} clean + make -s -j${{env.BUILD_JOBS}} world-bin + ### # Verify docs can be built ### # XXX: Only do this if there have been changes in doc/ since last build - name: Build documentation - if: always() + if: ${{ !cancelled() }} run: | + echo "::group::configure" ./configure \ --cache gcc.cache \ CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - make -s -j${BUILD_JOBS} -C doc + echo "::endgroup::" + + make -s -j${{env.BUILD_JOBS}} clean + make -s -j${{env.BUILD_JOBS}} -C doc ### # Verify headerscheck / cpluspluscheck succeed @@ -1015,12 +1068,19 @@ jobs: # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose ### - name: headerscheck + cpluspluscheck - if: always() + if: ${{ !cancelled() }} run: | + echo "::group::configure" ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ + ${{env.LINUX_CONFIGURE_FEATURES}} \ --cache gcc.cache \ --quiet \ CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' + echo "::endgroup::" + + make -s -j${{env.BUILD_JOBS}} clean + make -s -j${{env.BUILD_JOBS}} -k ${{env.CHECKFLAGS}} \ + headerscheck cpluspluscheck \ + EXTRAFLAGS='-fmax-errors=10' + + - *ccache_save_step -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0003-disable-cirrus.patch" From d97c1280b85bcfe0ae4f46be1d4e91a1bc44335a Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Thu, 28 May 2026 13:31:41 -0400 Subject: [PATCH v6a 3/5] disable cirrus Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- .cirrus.yml | 91 ---- .cirrus.star | 143 ------- .cirrus.tasks.yml | 1022 --------------------------------------------- 3 files changed, 1256 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/17358-89806e7420797025@postgresql.org - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net" - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0004-smaller-segsize-for-tests.patch" From c877607c5a0f2878fb9e28e93a40e5be22d7eb6c Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Thu, 28 May 2026 16:17:06 -0400 Subject: [PATCH v6a 4/5] smaller segsize for tests --- meson.build | 3 ++- contrib/test_decoding/expected/ddl.out | 2 +- contrib/test_decoding/sql/ddl.sql | 3 +-- src/test/perl/PostgreSQL/Test/Cluster.pm | 1 + src/test/recovery/t/039_end_of_wal.pl | 3 ++- src/test/recovery/t/051_effective_wal_level.pl | 2 +- src/test/regress/pg_regress.c | 2 +- src/Makefile.global.in | 2 +- 8 files changed, 10 insertions(+), 8 deletions(-) diff --git a/meson.build b/meson.build index 20b887f1a1b..b7a42e4373a 100644 --- a/meson.build +++ b/meson.build @@ -3927,7 +3927,8 @@ sys.exit(sp.returncode) test_initdb_template, temp_install_bindir / 'initdb', '--auth', 'trust', '--no-sync', '--no-instructions', '--lc-messages=C', - '--no-clean' + '--no-clean', + '--wal-segsize=1', ], priority: setup_tests_priority - 1, timeout: 300, diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out index 6819812e806..30a1877a76f 100644 --- a/contrib/test_decoding/expected/ddl.out +++ b/contrib/test_decoding/expected/ddl.out @@ -58,7 +58,7 @@ SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_d SELECT slot_name, plugin, slot_type, active, NOT catalog_xmin IS NULL AS catalog_xmin_set, xmin IS NULl AS data_xmin_not_set, - pg_wal_lsn_diff(restart_lsn, '0/01000000') > 0 AS some_wal + pg_wal_lsn_diff(restart_lsn, '0/00000001') > 0 AS some_wal FROM pg_replication_slots; slot_name | plugin | slot_type | active | catalog_xmin_set | data_xmin_not_set | some_wal -----------------+---------------+-----------+--------+------------------+-------------------+---------- diff --git a/contrib/test_decoding/sql/ddl.sql b/contrib/test_decoding/sql/ddl.sql index 6d0b7d77778..ba250e5d529 100644 --- a/contrib/test_decoding/sql/ddl.sql +++ b/contrib/test_decoding/sql/ddl.sql @@ -22,14 +22,13 @@ SELECT 'init' FROM pg_create_physical_replication_slot('repl'); SELECT data FROM pg_logical_slot_get_changes('repl', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1'); SELECT pg_drop_replication_slot('repl'); - SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding'); /* check whether status function reports us, only reproduceable columns */ SELECT slot_name, plugin, slot_type, active, NOT catalog_xmin IS NULL AS catalog_xmin_set, xmin IS NULl AS data_xmin_not_set, - pg_wal_lsn_diff(restart_lsn, '0/01000000') > 0 AS some_wal + pg_wal_lsn_diff(restart_lsn, '0/00000001') > 0 AS some_wal FROM pg_replication_slots; /* diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm index 4fcb1f6be56..5c74a8b690a 100644 --- a/src/test/perl/PostgreSQL/Test/Cluster.pm +++ b/src/test/perl/PostgreSQL/Test/Cluster.pm @@ -671,6 +671,7 @@ sub init 'initdb', '--no-sync', '--pgdata' => $pgdata, '--auth' => 'trust', + '--wal-segsize' => '1', @{ $params{extra} }); } else diff --git a/src/test/recovery/t/039_end_of_wal.pl b/src/test/recovery/t/039_end_of_wal.pl index f46d089a0fb..af6a6ced73d 100644 --- a/src/test/recovery/t/039_end_of_wal.pl +++ b/src/test/recovery/t/039_end_of_wal.pl @@ -112,7 +112,8 @@ sub build_page_header # set to "minimal" avoids random standby snapshot records. Autovacuum # could also trigger randomly, generating random WAL activity of its own. my $node = PostgreSQL::Test::Cluster->new("node"); -$node->init; +$node->init(extra=>['--wal-segsize=16']); + $node->append_conf( 'postgresql.conf', q[wal_level = minimal diff --git a/src/test/recovery/t/051_effective_wal_level.pl b/src/test/recovery/t/051_effective_wal_level.pl index c862073c34e..6329947a33f 100644 --- a/src/test/recovery/t/051_effective_wal_level.pl +++ b/src/test/recovery/t/051_effective_wal_level.pl @@ -34,7 +34,7 @@ sub wait_for_logical_decoding_disabled # Initialize the primary server with wal_level = 'replica'. my $primary = PostgreSQL::Test::Cluster->new('primary'); -$primary->init(allows_streaming => 1); +$primary->init(allows_streaming => 1, extra=>['--wal-segsize=16']); $primary->append_conf('postgresql.conf', "log_min_messages = debug1"); $primary->start(); diff --git a/src/test/regress/pg_regress.c b/src/test/regress/pg_regress.c index 1c052cc0fbf..23f9dfe781f 100644 --- a/src/test/regress/pg_regress.c +++ b/src/test/regress/pg_regress.c @@ -2410,7 +2410,7 @@ regression_main(int argc, char *argv[], note("initializing database system by running initdb"); appendStringInfo(&cmd, - "\"%s%sinitdb\" -D \"%s/data\" --no-clean --no-sync", + "\"%s%sinitdb\" -D \"%s/data\" --no-clean --no-sync --wal-segsize=1", bindir ? bindir : "", bindir ? "/" : "", temp_instance); diff --git a/src/Makefile.global.in b/src/Makefile.global.in index cef1ad7f87d..f529bacc4b2 100644 --- a/src/Makefile.global.in +++ b/src/Makefile.global.in @@ -442,7 +442,7 @@ ifeq ($(MAKELEVEL),0) $(MAKE) -C '$(top_builddir)' DESTDIR='$(abs_top_builddir)'/tmp_install install >'$(abs_top_builddir)'/tmp_install/log/install.log 2>&1 $(MAKE) -j1 $(if $(CHECKPREP_TOP),-C $(CHECKPREP_TOP),) checkprep >>'$(abs_top_builddir)'/tmp_install/log/install.log 2>&1 - $(with_temp_install) initdb --auth trust --no-sync --no-instructions --lc-messages=C --no-clean '$(abs_top_builddir)'/tmp_install/initdb-template >>'$(abs_top_builddir)'/tmp_install/log/initdb-template.log 2>&1 + $(with_temp_install) initdb --auth trust --no-sync --no-instructions --lc-messages=C --no-clean --wal-segsize=1 '$(abs_top_builddir)'/tmp_install/initdb-template >>'$(abs_top_builddir)'/tmp_install/log/initdb-template.log 2>&1 endif endif endif -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6a-0005-gha-Only-run-main-regression-tests.patch" From fb5c68339f49aba97ab8cfd9dd358a3b8e59228d Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Mon, 1 Jun 2026 15:31:32 -0400 Subject: [PATCH v6a 5/5] gha: Only run main regression tests --- .github/workflows/postgresql-ci.yml | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/.github/workflows/postgresql-ci.yml b/.github/workflows/postgresql-ci.yml index e2795ca0ffb..c6d4c960a55 100644 --- a/.github/workflows/postgresql-ci.yml +++ b/.github/workflows/postgresql-ci.yml @@ -35,7 +35,8 @@ env: # Check target for the autoconf builds. Can be set to e.g. check to only # only test the main regression tests. - CHECK: check-world PROVE_FLAGS=--timer + # FIXME: Reset + CHECK: check PROVE_FLAGS=--timer CHECKFLAGS: -Otarget # Build test dependencies as part of the build step, to see compiler @@ -45,7 +46,8 @@ env: # Can be set to a non-empty value to run a limited set of tests # (e.g. --suite regress to only run the main regression tests). - MTEST_TARGET: + # FIXME: Reset + MTEST_TARGET: --suite regress PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests TEMP_CONFIG: ${{ github.workspace }}/src/tools/ci/pg_ci_base.conf -- 2.54.0.380.gc69baaf57b --rv3g7aw7ud36z5b5--