agora inbox for pgsql-hackers@postgresql.org
help / color / mirror / Atom feedFrom: Andres Freund <andres@anarazel.de>
To: Nazir Bilal Yavuz <byavuz81@gmail.com>
Cc: Jacob Champion <jacob.champion@enterprisedb.com>
Cc: Peter Eisentraut <peter@eisentraut.org>
Cc: Jelte Fennema-Nio <postgres@jeltef.nl>
Cc: Thomas Munro <thomas.munro@gmail.com>
Cc: pgsql-hackers@postgresql.org, Zsolt Parragi <zsolt.parragi@percona.com>
Subject: Re: Heads Up: cirrus-ci is shutting down June 1st
Date: Fri, 12 Jun 2026 17:54:40 -0400
Message-ID: <iggjozfshwbqpv33x5jqwtju5k5zrkyu3257dlifxkhtpg7eoq@k2ccyrdi5dtu> (raw)
In-Reply-To: <CAN55FZ3sKWUutKjK-3VTUTJo5Tv=FczS=y0hNzMP1yLRUcW7HA@mail.gmail.com>
References: <fovrlyxqimnvl55iiopsaauwrjipws3dmbcxpyutc4v5irat3y@rn5gnt47hebj>
<CAN55FZ1nf0gqzobxaKBjN+xTbJgEueWo+T=YPrVSsNXTqdF7+A@mail.gmail.com>
<ttwd74i2czh327b4xr6ysnx5zyexfemfeg56kiwakvzgqhbypo@kettwx63kmg2>
<vquztdpdt2etzyxmlnsehv2ineyazlh4rgarn3p6ey6iy3nvps@3x2fyuvmf5ro>
<CAOYmi+nPBe7LBewiEVdiLrgkGVpoQRYqnTFiff0DtqLzok_exw@mail.gmail.com>
<dafwo5z5jxryfhq5txcue47e7dufi4rbebkchp6deo6alpdidd@tlukqy532v7z>
<CAOYmi+knRemECWD5VBcMHcuM2fTU55GEdTtGkcdVuRTbOeyhZw@mail.gmail.com>
<yhst2iacmavxsqlsrjkqsjodtf7finznzwrc7dm5dzkf3ygg3w@l6q2grflyjqx>
<vzdsj4tlvc3oph7hawo6jocjenc4dwoyhh4edvdfxfslkjj6nz@4z7dukcvjiph>
<CAN55FZ3sKWUutKjK-3VTUTJo5Tv=FczS=y0hNzMP1yLRUcW7HA@mail.gmail.com>
Hi,
Here's a series implementing a lot, but not all, that I brought up. See the
commit messages for details. One change that I hadn't mentioned on here is
that I increased the compiler optimizations for mingw, which improves the test
runtimes by a enough to even offset the cold-ccache compile times.
I for now used the setup-msys2 action, with a pin to the SHA to address
Jacob's concern (even though I still don't see what it buys us).
I did try creating an archive of an install, but that ends up with a good bit
of additional complexity, because the installations aren't relocatable (there
are header paths that are patched during install, apparently). And where the
install should be depends on the runner type, larger runners don't have D:,
but slower runners require using D:. I think Bilal hacked further on my
experiments around this, but I don't quite know where that stands. It's
certainly not a trivial change.
I'm pretty agnostic whether we want 0005, I just included that because it
looks a bit nicer.
On 2026-06-12 16:44:25 +0300, Nazir Bilal Yavuz wrote:
> On Wed, 10 Jun 2026 at 17:55, Andres Freund <andres@anarazel.de> wrote:
> >
> > Before we can backpatch the CI support I think we need to resolve a few more
> > things:
> >
> > - re-enabling crash reporting for windows, Bilal sent a patch [1]
> >
> > I think that's pretty much a must have, otherwise debuggin windows issues is
> > really hard.
>
> That seems working based on my testing, just needs a review.
I'll merge it, I think it's good to go.
> > - Cold or inapplicable (e.g. due to a core header change) compiler warnings
> > task is very slow (35min). I have a patch that I need to send out to
> > convert everything but the headercheck in compilerwarnings to meson, that
> > reduces the worst case build times considerably (primarily due to the cross
> > build being able to use precompiled headers)
> >
> > I'll try to send that out later today.
>
> Nice. I encountered this problem a couple of times and they were frustrating.
Yea, it's definitely not great as-is. It's a bit weird that with the patch
above we still use both meson and autoconf (for headerscheck), but I think
it's ok for now.
> > - I comparison to cirrus-ci it's considerably more painful (and it wasn't
> > exactly pain-free on cirrus either) to access the logs of failed tasks. One
> > can't just link to the failure or such.
> >
> > I have wondered about determining which test failed first, and uploading the
> > most crucial logs for that test separately, so one could at least look and
> > link to those without unpacking a .zip.
>
> I have questions about this. If we do this for all jobs then we can
> end up having just too many uploaded files to look at. In this case,
> unpacking .zip would be easier to access the logs of failed tasks.
I was thinking we'd do this only for the first failure...
> > - A decent chunk of test time is spent setting up the containers (I've
> > optimized them a bit to reduce that already). Somehow docker is pretty slow
> > around container extraction. I had already split the containers into one
> > for docs and one for the rest, if we did that further, we could make startup
> > of e.g. sanitycheck (which has an outsized impact) a decent bit faster, but
> > we can't use the same container for e.g. linux-meson-32.
> >
> > I think it may be smart to just add per-task tags for the containers. Then
> > we can have them initially be the same (by just pushing the same container
> > with different tags), which would allow us to adjust the containers contents
> > later, without needing to patch the workflow in the postgres repo.
> >
> > I suspect we should do the tag aliases before backpatching, but I'm very
> > willing to be convinced otherwise.
>
> It would make sense to do this before backpatching as some of the
> improvements can be done on the images themselves, which can speed up
> the process.
I did that on the container generation side and now attached a patch to use
them. It does reduce the time a decent bit.
Greetings,
Andres Freund
Attachments:
[text/x-diff] v13a-0001-ci-Generate-crashlogs-on-Windows.patch (6.3K, ../iggjozfshwbqpv33x5jqwtju5k5zrkyu3257dlifxkhtpg7eoq@k2ccyrdi5dtu/2-v13a-0001-ci-Generate-crashlogs-on-Windows.patch)
download | inline diff:
From cad64d49bd6bde4db06aa99b9c032844581b7f04 Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Thu, 11 Jun 2026 09:15:11 -0400
Subject: [PATCH v13a 1/9] ci: Generate crashlogs on Windows
cdb.exe is configured to log all crashes to
"<workspace>\crashlogs\crashlog-<pid-in-hex>.txt". Upload logs step
already configured to collect these logs, so no change is needed on
there.
Logic is copied from where Postgres CI Windows images are generated [1].
Since this would be too long to include inline in pg-ci.yml, it is implemented
as 'src/tools/ci/gha_setup_windows_debugger.ps1' script.
[1] https://github.com/anarazel/pg-vm-images/blob/main/scripts/windows_install_dbg.ps1
Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/CAN55FZ1BgsXSTzOpehnMa4NzWL8Aivsxx-di7-VT6bZ3j2Omow%40mail.gmail.com
---
.github/workflows/pg-ci.yml | 13 ++--
src/tools/ci/gha_setup_windows_debugger.ps1 | 75 +++++++++++++++++++++
2 files changed, 82 insertions(+), 6 deletions(-)
create mode 100644 src/tools/ci/gha_setup_windows_debugger.ps1
diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml
index 5bc5292d2a5..742a8431782 100644
--- a/.github/workflows/pg-ci.yml
+++ b/.github/workflows/pg-ci.yml
@@ -990,6 +990,11 @@ jobs:
shell: cmd
run: mkdir ${{env.PG_REGRESS_SOCK_DIR}}
+ - &windows_setup_debugger_step
+ name: Setup Windows debugger
+ shell: pwsh
+ run: src/tools/ci/gha_setup_windows_debugger.ps1
+
- name: Configure
run: |
call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64
@@ -1019,9 +1024,6 @@ jobs:
call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64
run: *meson_test_world_cmd
- # TODO: We need to collect crashlogs but for them to be generated, we'd
- # have to configure the JIT Debugger to do so. cdb.exe is installed on
- # the runner so that is possible.
- *upload_logs_step
@@ -1116,6 +1118,8 @@ jobs:
shell: cmd
run: mkdir ${{env.PG_REGRESS_SOCK_DIR}}
+ - *windows_setup_debugger_step
+
- *ccache_restore_default_step
- *ccache_restore_branch_step
@@ -1138,9 +1142,6 @@ jobs:
- name: Test world
run: *meson_test_world_cmd
- # TODO: We want to include crashlogs, but they are not yet
- # collected. cdb.exe is installed on the runner, so we can configure it
- # appropriately.
- *upload_logs_step
diff --git a/src/tools/ci/gha_setup_windows_debugger.ps1 b/src/tools/ci/gha_setup_windows_debugger.ps1
new file mode 100644
index 00000000000..babfbe76278
--- /dev/null
+++ b/src/tools/ci/gha_setup_windows_debugger.ps1
@@ -0,0 +1,75 @@
+# Setup Windows debugger to log all crashes to
+# <workspace>\crashlogs\crashlog-<pid-in-hex>.txt
+
+$ErrorActionPreference = 'Stop'
+
+$crashdir = "$env:GITHUB_WORKSPACE/crashlogs"
+New-Item -ItemType Directory -Force -Path $crashdir
+
+# Ensure restricted child processes can write the log file
+icacls $crashdir /grant "${env:USERNAME}:(OI)(CI)F" /Q
+
+# Prevent windows error handling dialog from causing hangs
+New-ItemProperty -Force -Path 'HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting' `
+ -Name 'DontShowUI' -Value 1 -PropertyType DWord
+New-ItemProperty -Force -Path 'HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting' `
+ -Name 'Disabled' -Value 1 -PropertyType DWord
+
+### Fallback minidumps if the JIT debugger below doesn't run
+New-Item -Force -Path 'HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting' `
+ -Name 'LocalDumps'
+New-ItemProperty -Force -Path 'HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps' `
+ -Name 'DumpFolder' -Value $crashdir -PropertyType ExpandString
+New-ItemProperty -Force -Path 'HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps' `
+ -Name 'DumpCount' -Value 5 -PropertyType DWord
+New-ItemProperty -Force -Path 'HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps' `
+ -Name 'DumpType' -Value 1 -PropertyType DWord
+###
+
+$cdb64 = @(
+ 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\cdb.exe',
+ 'C:\Program Files\Windows Kits\10\Debuggers\x64\cdb.exe'
+ ) | Where-Object { Test-Path $_ } | Select-Object -First 1
+$cdb86 = $cdb64.Replace('\x64\', '\x86\')
+
+###
+# -p PID:
+# Specifies the decimal process ID to be debugged. This is used to debug a
+# process that is already running.
+# -e Event:
+# Signals the debugger that the specified event has occurred. This option is
+# only used when starting the debugger programmatically.
+# -g:
+# Ignores the initial breakpoint in target application. This option will
+# cause the target application to continue running after it is started or
+# CDB attaches to it, unless another breakpoint has been set.
+# -kqm:
+# Starts CDB/NTSD in quiet mode.
+# -c "command":
+# Specifies the initial debugger command to run at start-up. This command
+# must be surrounded with quotation marks. Multiple commands can be
+# separated with semicolons.
+###
+$debuggerArgs = ' -p %ld -e %ld -g -kqm -c ".lines -e; .symfix+ ; aS /x proc $tpid ; .block {.logappend ' + "$crashdir/crashlog-" + '${proc}.txt} ; lsa $ip ; ~*kP ; !peb ; .logclose ; q "'
+
+Write-Host "Using cdb (x64): $cdb64"
+Set-ItemProperty `
+ -Path 'HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug' `
+ -Name 'Debugger' -Value ('"' + $cdb64 + '"' + $debuggerArgs)
+New-ItemProperty -Force -PropertyType DWord -Value 1 `
+ -Path 'HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug' `
+ -Name 'Auto'
+
+Write-Host "Using cdb (x86): $cdb86"
+Set-ItemProperty `
+ -Path 'HKLM:\SOFTWARE\WOW6432Node\Microsoft\Windows NT\CurrentVersion\AeDebug' `
+ -Name 'Debugger' -Value ('"' + $cdb86 + '"' + $debuggerArgs)
+New-ItemProperty -Force -PropertyType DWord -Value 1 `
+ -Path 'HKLM:\SOFTWARE\WOW6432Node\Microsoft\Windows NT\CurrentVersion\AeDebug' `
+ -Name 'Auto'
+
+# Show registered AeDebug values for diagnostics
+Get-ItemProperty 'HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug' |
+ Format-List Debugger,Auto
+Get-ItemProperty 'HKLM:\SOFTWARE\WOW6432Node\Microsoft\Windows NT\CurrentVersion\AeDebug' |
+ Format-List Debugger,Auto
--
2.54.0.450.g9ac3f193c0
[text/x-diff] v13a-0002-ci-Use-meson-for-most-of-CompilerWarnings-it-s-.patch (5.4K, ../iggjozfshwbqpv33x5jqwtju5k5zrkyu3257dlifxkhtpg7eoq@k2ccyrdi5dtu/3-v13a-0002-ci-Use-meson-for-most-of-CompilerWarnings-it-s-.patch)
download | inline diff:
From a31ad0492f02fb55693f2306ab5693985de560d7 Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Tue, 9 Jun 2026 10:18:50 -0400
Subject: [PATCH v13a 2/9] ci: Use meson for most of CompilerWarnings, it's a
lot faster
Previously, with a cold cache or a change in a central header, the
CompilerWarnings job would take ~35 minutes. The worst aspect of that was the
windows cross build, which would take about 8-10 minutes.
Migrate the compiler warnings tasks to meson, that generally makes the
cold-cache build a bit faster, and makes the windows cross build a *lot*
faster, due to being able to use precompiled headers.
With that the cold cache performance improves to about 12 minutes.
Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33
---
.github/workflows/pg-ci.yml | 38 +++++++++++++--------------
src/tools/ci/meson-cross-w64-ucrt.txt | 21 +++++++++++++++
2 files changed, 40 insertions(+), 19 deletions(-)
create mode 100644 src/tools/ci/meson-cross-w64-ucrt.txt
diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml
index 742a8431782..81801800dac 100644
--- a/.github/workflows/pg-ci.yml
+++ b/.github/workflows/pg-ci.yml
@@ -1171,6 +1171,8 @@ jobs:
# compilers / flag combinations.
CCACHE_MAXSIZE: "1G"
DEFAULT_BUILD: world-bin
+ CCACHE_SLOPPINESS: pch_defines,time_macros
+ CCACHE_DEPEND: 1
steps:
- *nix_sysinfo_step
@@ -1186,53 +1188,56 @@ jobs:
- name: gcc warnings + (dtrace)
if: ${{ !cancelled() }}
env:
- CONF: ${{env.LINUX_CONFIGURE_FEATURES}} --cache gcc.cache --enable-dtrace
+ CONF: ${{env.MESON_COMMON_FEATURES}} -Ddtrace=enabled
CC: ccache gcc
CXX: ccache g++
CLANG: ccache clang
run: &compiler_warnings_cmd |
echo "::group::configure"
- ./configure \
- ${{env.CONF}} \
- CLANG="ccache clang"
+ rm -rf build-meson
+ meson setup -Dwerror=true $CONF build-meson
echo "::endgroup::"
- make -s -j${{env.BUILD_JOBS}} clean
- make -s -j${{env.BUILD_JOBS}} ${{env.DEFAULT_BUILD}}
+ ninja -C build-meson --quiet -j${{env.BUILD_JOBS}} ${{env.MBUILD_TARGET}}
# gcc, cassert on, dtrace off
- name: gcc warnings + (cassert)
if: ${{ !cancelled() }}
env:
- CONF: ${{env.LINUX_CONFIGURE_FEATURES}} --cache gcc.cache --enable-cassert
+ CONF: ${{env.MESON_COMMON_FEATURES}} -Dcassert=true
CC: ccache gcc
CXX: ccache g++
+ CLANG: ccache clang
run: *compiler_warnings_cmd
# clang, cassert off, dtrace off
- name: clang warnings
if: ${{ !cancelled() }}
env:
- CONF: ${{env.LINUX_CONFIGURE_FEATURES}} --cache clang.cache
+ CONF: ${{env.MESON_COMMON_FEATURES}}
CC: ccache clang
CXX: ccache clang++
+ CLANG: ccache clang
run: *compiler_warnings_cmd
# clang, cassert on, dtrace on
- name: clang warnings + (cassert + dtrace)
if: ${{ !cancelled() }}
env:
- CONF: ${{env.LINUX_CONFIGURE_FEATURES}} --cache clang.cache --enable-cassert --enable-dtrace
+ CONF: ${{env.MESON_COMMON_FEATURES}} -Dcassert=true -Ddtrace=enabled
CC: ccache clang
CXX: ccache clang++
+ CLANG: ccache clang
run: *compiler_warnings_cmd
- name: mingw warnings (cross compilation)
if: ${{ !cancelled() }}
env:
- CONF: --host=x86_64-w64-mingw32ucrt --enable-cassert --without-icu
- CC: ccache x86_64-w64-mingw32ucrt-gcc
- CXX: ccache x86_64-w64-mingw32ucrt-g++
+ CONF: >-
+ --cross-file src/tools/ci/meson-cross-w64-ucrt.txt
+ --buildtype debug
+ -Db_pch=true
+ -Dcassert=true
run: *compiler_warnings_cmd
###
@@ -1241,12 +1246,8 @@ jobs:
# XXX: Only do this if there have been changes in doc/ since last build
- name: Build documentation
if: ${{ !cancelled() }}
- env:
- CONF: --cache gcc.cache
- CC: ccache gcc
- CXX: ccache g++
- DEFAULT_BUILD: -C doc
- run: *compiler_warnings_cmd
+ run:
+ ninja -C build-meson docs
###
# Verify headerscheck / cpluspluscheck succeed
@@ -1261,7 +1262,6 @@ jobs:
echo "::group::configure"
./configure \
${{env.LINUX_CONFIGURE_FEATURES}} \
- --cache gcc.cache \
--quiet \
CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang"
echo "::endgroup::"
diff --git a/src/tools/ci/meson-cross-w64-ucrt.txt b/src/tools/ci/meson-cross-w64-ucrt.txt
new file mode 100644
index 00000000000..b6144979416
--- /dev/null
+++ b/src/tools/ci/meson-cross-w64-ucrt.txt
@@ -0,0 +1,21 @@
+[constants]
+triple = 'x86_64-w64-mingw32ucrt'
+prefix = '/usr/bin' / triple + '-'
+
+[binaries]
+c = ['ccache', prefix + 'gcc']
+cpp = ['ccache', prefix + 'g++']
+ar = prefix + 'ar'
+strip = prefix + 'strip'
+pkg-config = prefix + 'pkg-config'
+windres = prefix + 'windres'
+
+[properties]
+# Directory that contains 'bin', 'lib', etc
+root = '/usr/x86_64-w64-mingw32ucrt'
+
+[host_machine]
+system = 'windows'
+cpu_family = 'x86_64'
+cpu = 'x86_64'
+endian = 'little'
--
2.54.0.450.g9ac3f193c0
[text/x-diff] v13a-0003-ci-Make-msys2-install-smaller.patch (1.8K, ../iggjozfshwbqpv33x5jqwtju5k5zrkyu3257dlifxkhtpg7eoq@k2ccyrdi5dtu/4-v13a-0003-ci-Make-msys2-install-smaller.patch)
download | inline diff:
From 81f52c2cc8fe15cb96522da08a43dfa9021f53ae Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Wed, 10 Jun 2026 23:29:47 -0400
Subject: [PATCH v13a 3/9] ci: Make msys2 install smaller
We only needed git and diffutils installed via pacman because the already
installed tools where hidden by the login script resetting PATH. "Fix" that
instead by telling msys to leave the old PATH around.
Also don't install libbacktrace it is not needed at the moment.
Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33
---
.github/workflows/pg-ci.yml | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml
index 81801800dac..442052b72e1 100644
--- a/.github/workflows/pg-ci.yml
+++ b/.github/workflows/pg-ci.yml
@@ -1056,6 +1056,11 @@ jobs:
CCACHE_SLOPPINESS: pch_defines,time_macros
CCACHE_DEPEND: 1
+ # We don't want using an msys bash to "hide" all the other already
+ # installed tools, that would require us to install tools into msys that
+ # are already available otherwise.
+ MSYS2_PATH_TYPE: inherit
+
defaults:
run:
shell: 'D:\msys64\usr\bin\bash.exe --login -eo pipefail "{0}"'
@@ -1086,11 +1091,10 @@ jobs:
# MSYS2. It dynamically expands to the correct prefix for the active
# shell environment.
pacman -S --noconfirm --needed --asdeps \
- git bison flex diffutils \
+ bison flex \
${MINGW_PACKAGE_PREFIX}-ccache \
${MINGW_PACKAGE_PREFIX}-gcc \
${MINGW_PACKAGE_PREFIX}-icu \
- ${MINGW_PACKAGE_PREFIX}-libbacktrace \
${MINGW_PACKAGE_PREFIX}-libxml2 \
${MINGW_PACKAGE_PREFIX}-libxslt \
${MINGW_PACKAGE_PREFIX}-lz4 \
--
2.54.0.450.g9ac3f193c0
[text/x-diff] v13a-0004-ci-Make-our-own-msys2-install-from-scratch-inst.patch (4.6K, ../iggjozfshwbqpv33x5jqwtju5k5zrkyu3257dlifxkhtpg7eoq@k2ccyrdi5dtu/5-v13a-0004-ci-Make-our-own-msys2-install-from-scratch-inst.patch)
download | inline diff:
From ec313988704520705aaae85b86767187d2f9f27d Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Fri, 12 Jun 2026 08:46:00 -0400
Subject: [PATCH v13a 4/9] ci: Make our own msys2 install from scratch, instead
of moving
Previously we relocated the existing install, for performance reasons. However
that has two disadvantages:
1) moving the install and then installing the packages we need is slower than
just creating a new install
2) It hardcodes that D: is the fast drive, but that turns out to depend on the
type of runner used
3) We were not using any caching and therefore downloaded the same files over
and over. We could have added caching in the previous approach too, but
it'd have been extra work.
I previously prototyped handrolling the install, instead of using
msys2/setup-msys2@v2, and that does turn out to be a bit faster, but it
doesn't include caching. By the time we add that, the overall complexity seems
like it'd be too big.
The other alternative would be to generate a downloadable release in the
pg-vm-images repo. That'd likely be even faster.
Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33
---
.github/workflows/pg-ci.yml | 64 +++++++++++++++++--------------------
1 file changed, 29 insertions(+), 35 deletions(-)
diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml
index 442052b72e1..794a2600a23 100644
--- a/.github/workflows/pg-ci.yml
+++ b/.github/workflows/pg-ci.yml
@@ -1063,48 +1063,42 @@ jobs:
defaults:
run:
- shell: 'D:\msys64\usr\bin\bash.exe --login -eo pipefail "{0}"'
+ shell: msys2 {0}
steps:
- *windows_disable_defender_step
- *window_setup_hosts_step
- *checkout_step
- # Relocate the preinstalled MSYS2 tree from C:\ (slow system disk) to
- # D:\ (faster ephemeral data disk). Every subsequent MSYS2 step uses
- # D:\msys64\usr\bin\bash.exe via the job's `defaults.run.shell`.
+ # The pre-existing msys install is on C:\, often a rather slow system
+ # disk, whereas the workspace is often on a faster, ephemeral disk.
+ # Using the pre-existing msys would often increase the total runtime of
+ # this task by ~15 minutes.
#
- # This reduces the total runtime of this task by ~15 minutes.
- #
- # robocopy returns 0-7 on success (with various "files copied" bits
- # set) and 8+ on real failure, so we have to translate its exit code.
- - name: Relocate MSYS2 to D
- shell: pwsh
- run: |
- robocopy C:\msys64 D:\msys64 /E /NJS /NJH /NFL /NDL /NP
- if ($LASTEXITCODE -ge 8) { exit $LASTEXITCODE }
- exit 0
-
- - name: Setup MSYS2
- run: |
- # ${MINGW_PACKAGE_PREFIX} is an environment variable used in the
- # MSYS2. It dynamically expands to the correct prefix for the active
- # shell environment.
- pacman -S --noconfirm --needed --asdeps \
- bison flex \
- ${MINGW_PACKAGE_PREFIX}-ccache \
- ${MINGW_PACKAGE_PREFIX}-gcc \
- ${MINGW_PACKAGE_PREFIX}-icu \
- ${MINGW_PACKAGE_PREFIX}-libxml2 \
- ${MINGW_PACKAGE_PREFIX}-libxslt \
- ${MINGW_PACKAGE_PREFIX}-lz4 \
- ${MINGW_PACKAGE_PREFIX}-make \
- ${MINGW_PACKAGE_PREFIX}-meson \
- ${MINGW_PACKAGE_PREFIX}-perl \
- ${MINGW_PACKAGE_PREFIX}-pkgconf \
- ${MINGW_PACKAGE_PREFIX}-readline \
- ${MINGW_PACKAGE_PREFIX}-zlib \
- ${MINGW_PACKAGE_PREFIX}-zstd
+ # Instead we use msys2/setup-msys2 to set up our own install. That's
+ # faster than moving the install from C:\, and also works when the
+ # workspace is not on a separate disk.
+ - name: Install MSYS2
+ uses: msys2/setup-msys2@66cd2cce69caa17b53920067426061ca1de3a884 # v2.31.1
+ with:
+ msystem: ${{env.MSYSTEM}}
+ update: true
+ location: ${{github.workspace}}
+ install: >-
+ bison flex
+ mingw-w64-ucrt-x86_64-ccache
+ mingw-w64-ucrt-x86_64-gcc
+ mingw-w64-ucrt-x86_64-icu
+ mingw-w64-ucrt-x86_64-libxml2
+ mingw-w64-ucrt-x86_64-libxslt
+ mingw-w64-ucrt-x86_64-lz4
+ mingw-w64-ucrt-x86_64-make
+ mingw-w64-ucrt-x86_64-meson
+ mingw-w64-ucrt-x86_64-perl
+ mingw-w64-ucrt-x86_64-pkgconf
+ mingw-w64-ucrt-x86_64-readline
+ mingw-w64-ucrt-x86_64-zlib
+ mingw-w64-ucrt-x86_64-zstd
- *nix_sysinfo_step
--
2.54.0.450.g9ac3f193c0
[text/x-diff] v13a-0005-squash-or-drop-Use-pacboy-for-shorter-package-n.patch (1.5K, ../iggjozfshwbqpv33x5jqwtju5k5zrkyu3257dlifxkhtpg7eoq@k2ccyrdi5dtu/6-v13a-0005-squash-or-drop-Use-pacboy-for-shorter-package-n.patch)
download | inline diff:
From 4ca78ad49183ba5899ac489970d51dd61e70e8f0 Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Fri, 12 Jun 2026 17:06:37 -0400
Subject: [PATCH v13a 5/9] squash-or-drop: Use pacboy for shorter package names
---
.github/workflows/pg-ci.yml | 28 +++++++++++++++-------------
1 file changed, 15 insertions(+), 13 deletions(-)
diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml
index 794a2600a23..41a807af801 100644
--- a/.github/workflows/pg-ci.yml
+++ b/.github/workflows/pg-ci.yml
@@ -1086,19 +1086,21 @@ jobs:
location: ${{github.workspace}}
install: >-
bison flex
- mingw-w64-ucrt-x86_64-ccache
- mingw-w64-ucrt-x86_64-gcc
- mingw-w64-ucrt-x86_64-icu
- mingw-w64-ucrt-x86_64-libxml2
- mingw-w64-ucrt-x86_64-libxslt
- mingw-w64-ucrt-x86_64-lz4
- mingw-w64-ucrt-x86_64-make
- mingw-w64-ucrt-x86_64-meson
- mingw-w64-ucrt-x86_64-perl
- mingw-w64-ucrt-x86_64-pkgconf
- mingw-w64-ucrt-x86_64-readline
- mingw-w64-ucrt-x86_64-zlib
- mingw-w64-ucrt-x86_64-zstd
+ # name:p means MINGW_PACKAGE_PREFIX-only
+ pacboy: >-
+ ccache:p
+ gcc:p
+ icu:p
+ libxml2:p
+ libxslt:p
+ lz4:p
+ make:p
+ meson:p
+ perl:p
+ pkgconf:p
+ readline:p
+ zlib:p
+ zstd:p
- *nix_sysinfo_step
--
2.54.0.450.g9ac3f193c0
[text/x-diff] v13a-0006-ci-Use-optimized-build-for-mingw.patch (1.0K, ../iggjozfshwbqpv33x5jqwtju5k5zrkyu3257dlifxkhtpg7eoq@k2ccyrdi5dtu/7-v13a-0006-ci-Use-optimized-build-for-mingw.patch)
download | inline diff:
From 707a3f799121b514c99746f04cdda87eaf087acd Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Thu, 11 Jun 2026 00:01:53 -0400
Subject: [PATCH v13a 6/9] ci: Use optimized build for mingw
The test runtime dominates over the compile time on GHA. Note that we just
need to remove options, as postgres's default is debugoptimized.
Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33
---
.github/workflows/pg-ci.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml
index 41a807af801..ab7950f6f14 100644
--- a/.github/workflows/pg-ci.yml
+++ b/.github/workflows/pg-ci.yml
@@ -1127,7 +1127,7 @@ jobs:
run: |
meson setup \
${{env.MESON_COMMON_PG_CONFIG_ARGS}} \
- -Ddebug=true -Doptimization=g -Db_pch=true \
+ -Db_pch=true \
${{env.MESON_COMMON_FEATURES}} \
${{env.MESON_FEATURES}} \
-DTAR=${{env.TAR}} \
--
2.54.0.450.g9ac3f193c0
[text/x-diff] v13a-0007-ci-get-build-test-concurrency-from-environment.patch (5.3K, ../iggjozfshwbqpv33x5jqwtju5k5zrkyu3257dlifxkhtpg7eoq@k2ccyrdi5dtu/8-v13a-0007-ci-get-build-test-concurrency-from-environment.patch)
download | inline diff:
From e9e2172ebe07c94848874f9a793518136af34eb1 Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Wed, 10 Jun 2026 14:49:28 -0400
Subject: [PATCH v13a 7/9] ci: get build/test concurrency from environment
Previously we hardcoded the amount of concurrency in the CI definition. This
practice IIRC originated from a) using make, where the concurrency is not
sourced from the current system b) early cirrus-ci macos runners, which slowed
down a lot with full concurrency.
Hardcoding the concurrency is problematic as e.g. private repositories have
lower concurrency and larger github runners have more cores.
For most of the jobs we can rely on just using meson's and ninja's logic for
getting the current core count. For the autoconf we can use the nproc helper.
To make this at least somewhat understandable, I added the existing sysinfo
step to show the number of cores (and made it a bit easier to understand the
information already printed).
Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33
---
.github/workflows/pg-ci.yml | 47 +++++++++++++++++++++++--------------
1 file changed, 29 insertions(+), 18 deletions(-)
diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml
index ab7950f6f14..9384ec0d902 100644
--- a/.github/workflows/pg-ci.yml
+++ b/.github/workflows/pg-ci.yml
@@ -40,14 +40,6 @@ env:
# concurrent jobs and retrying older runs have a chance of working.
CLONE_DEPTH: 500
- # At the moment all jobs use 4vcore runners, and none seems to benefit from
- # increasing concurrency further.
- BUILD_JOBS: 4
-
- # It's possible that some jobs benefit from an increased test concurrency,
- # but a default of 4 is a safe bet. Individual jobs can override.
- TEST_JOBS: 4
-
CCACHE_MAXSIZE: "250M"
CCACHE_DIR: ${{ github.workspace }}/ccache_dir
@@ -215,10 +207,19 @@ jobs:
- &nix_sysinfo_step
name: sysinfo
run: |
- id
- uname -a
+ echo -n "Number of cores: "
+ nproc 2> /dev/null || sysctl hw.physicalcpu hw.logicalcpu
+
+ echo "id: $(id)"
+ echo "uname: $(uname -a)"
+
+ echo ::group::ulimit
ulimit -a -H && ulimit -a -S
+ echo ::endgroup::
+
+ echo ::group::env
env
+ echo ::endgroup::
- name: Parse ci-os-only
id: os
@@ -352,7 +353,7 @@ jobs:
- name: Build
shell: *su_postgres_shell
run: &ninja_build_cmd |
- ninja -C build -j${{env.BUILD_JOBS}} ${{env.MBUILD_TARGET}}
+ ninja -C build ${{env.MBUILD_TARGET}}
ninja -C build -t missingdeps
# Decide if it's worth uploading a new version of the ccache cache. If
@@ -405,7 +406,7 @@ jobs:
meson test ${{env.MTEST_ARGS}} --suite setup --logbase setup || exit 1
echo ::endgroup::
- meson test ${{env.MTEST_ARGS}} --num-processes ${{env.TEST_JOBS}} --no-suite setup ${{env.MTEST_TARGET}}
+ meson test ${{env.MTEST_ARGS}} --no-suite setup ${{env.MTEST_TARGET}}
- &linux_collect_cores_step
name: Core backtraces
@@ -511,7 +512,7 @@ jobs:
- name: Build
shell: *su_postgres_shell
run: |
- make -s -j${BUILD_JOBS} world-bin
+ make -s -j$(nproc) world-bin
- *ccache_decide_save_step
- *ccache_save_step
@@ -519,7 +520,7 @@ jobs:
- name: Test world
shell: *su_postgres_shell
run: |
- make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS}
+ make -s ${CHECK} ${CHECKFLAGS} -j$(nproc)
- *linux_collect_cores_step
- *upload_logs_step
@@ -611,7 +612,7 @@ jobs:
build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start
# Run the tests supporting running against an already running
- meson test ${{env.MTEST_ARGS}} --num-processes ${{env.TEST_JOBS}} --setup running
+ meson test ${{env.MTEST_ARGS}} --setup running
build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop
@@ -893,9 +894,19 @@ jobs:
- name: Sysinfo
run: |
+ echo Number of cores:
+ bash -c nproc
+
+ echo codepage:
chcp
+
+ echo ::group::systeminfo
systeminfo
+ echo ::endgroup::
+
+ echo ::group::env
set
+ echo ::endgroup::
# The TAP tests build an initdb template under build/tmp_install and
# then `robocopy` it into per-test data directories. Robocopy with the
@@ -1198,7 +1209,7 @@ jobs:
meson setup -Dwerror=true $CONF build-meson
echo "::endgroup::"
- ninja -C build-meson --quiet -j${{env.BUILD_JOBS}} ${{env.MBUILD_TARGET}}
+ ninja -C build-meson --quiet ${{env.MBUILD_TARGET}}
# gcc, cassert on, dtrace off
- name: gcc warnings + (cassert)
@@ -1266,8 +1277,8 @@ jobs:
CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang"
echo "::endgroup::"
- make -s -j${{env.BUILD_JOBS}} clean
- make -s -j${{env.BUILD_JOBS}} -k ${{env.CHECKFLAGS}} \
+ make -s -j$(nproc) clean
+ make -s -j$(nproc) -k ${{env.CHECKFLAGS}} \
headerscheck cpluspluscheck \
EXTRAFLAGS='-fmax-errors=10'
--
2.54.0.450.g9ac3f193c0
[text/x-diff] v13a-0008-ci-Make-runs-on-overridable.patch (9.3K, ../iggjozfshwbqpv33x5jqwtju5k5zrkyu3257dlifxkhtpg7eoq@k2ccyrdi5dtu/9-v13a-0008-ci-Make-runs-on-overridable.patch)
download | inline diff:
From 9ebb45746bbb04b1f44692f0f00aed9cfd67d3dd Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Wed, 10 Jun 2026 13:09:10 -0400
Subject: [PATCH v13a 8/9] ci: Make runs-on overridable
Previously what runners CI ran on was hardcoded. Unfortunately that makes it
harder to use faster or self hosted runners (e.g. in a private repository
where the runners are very slow, or self hosted runners for cfbot).
Fix that by allowing to override the runner types via repository/organization
variables. How to do that is documented in src/tools/ci/README.
Testing faster github hosted runners showed that we have some dependencies on
specific locations, fix most of those. The remainder will be in a separate
commit, changing how we deal with the msys2 installation.
Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33
---
.github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++-------------
src/tools/ci/README | 21 +++++++++++++
2 files changed, 60 insertions(+), 21 deletions(-)
diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml
index 9384ec0d902..c737173abe0 100644
--- a/.github/workflows/pg-ci.yml
+++ b/.github/workflows/pg-ci.yml
@@ -106,12 +106,6 @@ env:
--with-uuid=ossp
--with-zstd
- # Centrally define the version of linux runners, to make it easier to
- # update. We don't just want to use ubuntu-latest, as it's not implausible
- # there will be breakage when that switches to the next ubuntu version.
- _LINUX_RUNS_ON: &linux_runs_on |
- ubuntu-24.04
-
# Debian Trixie containers used by all Linux jobs. Built by
# 'https://github.com/anarazel/pg-vm-images/'.
CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main
@@ -154,7 +148,8 @@ jobs:
warn-if-not-opted-in:
name: Report if not opted into CI
if: ${{vars.PG_CI_ENABLED != '1'}}
- runs-on: ubuntu-slim
+ # See the setup task for an explanation
+ runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }}
steps:
- name: Warn
env:
@@ -184,21 +179,43 @@ jobs:
# Only run CI if repo owner opted in. If this task is skipped due to the
# if, none of it's depending tasks (i.e. the actual CI tasks) run either.
if: ${{vars.PG_CI_ENABLED == '1'}}
- runs-on: *linux_runs_on
timeout-minutes: 1
outputs:
+ # Are certain tasks enabled?
linux: ${{ steps.os.outputs.linux }}
macos: ${{ steps.os.outputs.macos }}
windows: ${{ steps.os.outputs.windows }}
mingw: ${{ steps.os.outputs.mingw }}
compilerwarnings: ${{ steps.os.outputs.compilerwarnings }}
sanitycheck: ${{ steps.os.outputs.sanitycheck }}
+
+ # What runners to run on:
+ #
+ # To allow over-riding what runners jobs use, we allow setting
+ # repository / org level variables influencing that choice on a
+ # per-host-operating-system basis.
+ #
+ # This also makes it easier to change the version of linux / windows
+ # used centrally. We don't just want to use ubuntu-latest, as it's not
+ # implausible there will be breakage when that switches to the next
+ # ubuntu version.
+ runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }}
+ runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }}
+ runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }}
+ # runs_on_setup: an output can't be used as the input of the setup job
+ # itself, so it's done inline below.
+
+ # Exports:
+ #
# Re-export workflow-level env vars that other jobs need to reference
# from contexts (e.g. `jobs.<id>.container.image`) where the `env`
# context is not available.
container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }}
container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }}
+ # Can't use the output from above, so repeated verbatim here
+ runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }}
+
steps:
# Anchor reused by other jobs further down. GitHub Actions supports YAML
# anchors/aliases but not merge keys, so the alias copies the whole step
@@ -258,7 +275,7 @@ jobs:
if: |
!cancelled() &&
needs.setup.outputs.sanitycheck == 'true'
- runs-on: *linux_runs_on
+ runs-on: ${{ needs.setup.outputs.runs_on_linux }}
timeout-minutes: 15
container: &linux_ci_container
image: ${{ needs.setup.outputs.container_linux_ci }}
@@ -446,7 +463,7 @@ jobs:
!cancelled() &&
needs.setup.outputs.linux == 'true' &&
needs.sanity-check.result != 'failure'
- runs-on: *linux_runs_on
+ runs-on: ${{ needs.setup.outputs.runs_on_linux }}
container: *linux_ci_container
timeout-minutes: 60
@@ -540,7 +557,7 @@ jobs:
name: Linux - Meson (32-bit)
needs: [setup, sanity-check]
if: *linux_job_if
- runs-on: *linux_runs_on
+ runs-on: ${{ needs.setup.outputs.runs_on_linux }}
container: *linux_ci_container
timeout-minutes: 60
env: *linux_env
@@ -632,7 +649,7 @@ jobs:
name: Linux - Meson (64-bit)
needs: [setup, sanity-check]
if: *linux_job_if
- runs-on: *linux_runs_on
+ runs-on: ${{ needs.setup.outputs.runs_on_linux }}
container: *linux_ci_container
timeout-minutes: 60
env: *linux_env
@@ -688,7 +705,7 @@ jobs:
!cancelled() &&
needs.setup.outputs.macos == 'true' &&
needs.sanity-check.result != 'failure'
- runs-on: macos-15
+ runs-on: ${{ needs.setup.outputs.runs_on_macos }}
timeout-minutes: 60
env:
MACPORTS_CACHE: ${{ github.workspace }}/macports-cache
@@ -845,7 +862,7 @@ jobs:
!cancelled() &&
needs.setup.outputs.windows == 'true' &&
needs.sanity-check.result != 'failure'
- runs-on: windows-2022
+ runs-on: ${{ needs.setup.outputs.runs_on_windows }}
timeout-minutes: 60
# As described at the top of the task, split the tests across two runners
@@ -861,7 +878,7 @@ jobs:
env:
# Avoid port conflicts between concurrent tap tests
PG_TEST_USE_UNIX_SOCKETS: 1
- PG_REGRESS_SOCK_DIR: 'd:\pgsock'
+ PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock
TAR: "c:/windows/system32/tar.exe"
MESON_FEATURES: >-
@@ -967,7 +984,8 @@ jobs:
- name: Install dependencies
shell: pwsh
run: |
- # meson is not preinstalled on windows-2022. Install via pip
+ # meson is not preinstalled, at least on windows-2022. Install via
+ # pip
echo ::group::pip
python -m pip install --upgrade meson
if (!$?) { throw 'cmdfail' }
@@ -999,7 +1017,7 @@ jobs:
- name: Setup socket directory
shell: cmd
- run: mkdir ${{env.PG_REGRESS_SOCK_DIR}}
+ run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}"
- &windows_setup_debugger_step
name: Setup Windows debugger
@@ -1046,12 +1064,12 @@ jobs:
!cancelled() &&
needs.setup.outputs.mingw == 'true' &&
needs.sanity-check.result != 'failure'
- runs-on: windows-2022
+ runs-on: ${{ needs.setup.outputs.runs_on_windows }}
timeout-minutes: 60
env:
# Avoid port conflicts between concurrent tap tests
PG_TEST_USE_UNIX_SOCKETS: 1
- PG_REGRESS_SOCK_DIR: 'd:\pgsock'
+ PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock
TAR: "c:/windows/system32/tar.exe"
MSYS: winjitdebug
@@ -1127,7 +1145,7 @@ jobs:
- name: Setup socket directory
shell: cmd
- run: mkdir ${{env.PG_REGRESS_SOCK_DIR}}
+ run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}"
- *windows_setup_debugger_step
@@ -1173,7 +1191,7 @@ jobs:
!cancelled() &&
needs.setup.outputs.compilerwarnings == 'true' &&
needs.sanity-check.result != 'failure'
- runs-on: *linux_runs_on
+ runs-on: ${{ needs.setup.outputs.runs_on_linux }}
timeout-minutes: 60
container:
image: ${{ needs.setup.outputs.container_linux_ci_docs }}
diff --git a/src/tools/ci/README b/src/tools/ci/README
index 642e518c296..9276d4a654c 100644
--- a/src/tools/ci/README
+++ b/src/tools/ci/README
@@ -75,3 +75,24 @@ messages. Currently the following controls are available:
Only runs CI on operating systems specified. This can be useful when
addressing portability issues affecting only a subset of platforms.
+
+
+Controlling which runners CI uses
+=================================
+
+By default each job runs on a GitHub-hosted runner appropriate for its
+operating system. This can be overridden on a per-operating-system basis by
+creating repository or organization variables, at
+https://github.com/<username>/<reponame>/settings/variables/actions
+
+This is useful to run on faster GitHub-hosted runners, or on self-hosted
+runners.
+
+The following variables are recognized. If a variable is unset or empty, a
+default runner suitable for that operating system is used:
+
+- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings
+- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs
+- PG_CI_RUNS_ON_MACOS: the macOS job
+- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and
+ job setup)
--
2.54.0.450.g9ac3f193c0
[text/x-diff] v13a-0009-ci-Move-to-per-job-naming-for-containers.patch (5.2K, ../iggjozfshwbqpv33x5jqwtju5k5zrkyu3257dlifxkhtpg7eoq@k2ccyrdi5dtu/10-v13a-0009-ci-Move-to-per-job-naming-for-containers.patch)
download | inline diff:
From b5412dc3b24334f8eb6396b1e37c05a7096eb03e Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Fri, 12 Jun 2026 12:57:36 -0400
Subject: [PATCH v13a 9/9] ci: Move to per-job naming for containers
Previously we only had two types of containers, "ci" for everything but the
CompilerWarnings job and ci_docs that had the tools necessary to build the
docs as part of CompilerWarnings. But that was unnecessarily heavy for several
of the jobs. As the pull time is a noticeable performance factor, that's not
great.
Instead use containers that are named by their job. Today some of those are
the same, but after this change we can optimize the containers for their jobs
without a problem.
It probably would make sense to allow over-riding CONTAINER_REPO on a
per-repo/org basis, but that seems better done as a separate change.
Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33
---
.github/workflows/pg-ci.yml | 32 +++++++++++++++++++++-----------
1 file changed, 21 insertions(+), 11 deletions(-)
diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml
index c737173abe0..4034f76e0a1 100644
--- a/.github/workflows/pg-ci.yml
+++ b/.github/workflows/pg-ci.yml
@@ -108,9 +108,12 @@ env:
# Debian Trixie containers used by all Linux jobs. Built by
# 'https://github.com/anarazel/pg-vm-images/'.
+ #
+ # The containers are named
+ # ghcr.io/anarazel/pg-vm-images/<branch>/linux_debian_ci/<debian version>/<ci-job>:latest
CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main
- CONTAINER_LINUX_CI: linux_debian_trixie_ci:latest
- CONTAINER_LINUX_CI_DOCS: linux_debian_trixie_ci_docs:latest
+ CONTAINER_LINUX_CI_NAME: linux_debian_ci/trixie
+ CONTAINER_TAG: latest
# The full set of OS / job selectors recognized by the `ci-os-only:`
# commit-message directive parsed in the `setup` job below.
@@ -193,7 +196,7 @@ jobs:
#
# To allow over-riding what runners jobs use, we allow setting
# repository / org level variables influencing that choice on a
- # per-host-operating-system basis.
+ # per-host-operating-system basis. See also src/tools/ci/README
#
# This also makes it easier to change the version of linux / windows
# used centrally. We don't just want to use ubuntu-latest, as it's not
@@ -210,8 +213,8 @@ jobs:
# Re-export workflow-level env vars that other jobs need to reference
# from contexts (e.g. `jobs.<id>.container.image`) where the `env`
# context is not available.
- container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }}
- container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }}
+ container_linux_ci_base: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_NAME }}
+ container_tag: ${{ env.CONTAINER_TAG }}
# Can't use the output from above, so repeated verbatim here
runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }}
@@ -277,8 +280,8 @@ jobs:
needs.setup.outputs.sanitycheck == 'true'
runs-on: ${{ needs.setup.outputs.runs_on_linux }}
timeout-minutes: 15
- container: &linux_ci_container
- image: ${{ needs.setup.outputs.container_linux_ci }}
+ container:
+ image: ${{ needs.setup.outputs.container_linux_ci_base }}/sanity-check:${{ needs.setup.outputs.container_tag }}
# Options passed to all linux containers. Not all of the jobs need
# all of them, but it's easier to just define them centrally.
@@ -464,7 +467,9 @@ jobs:
needs.setup.outputs.linux == 'true' &&
needs.sanity-check.result != 'failure'
runs-on: ${{ needs.setup.outputs.runs_on_linux }}
- container: *linux_ci_container
+ container:
+ image: ${{ needs.setup.outputs.container_linux_ci_base }}/linux-autoconf:${{ needs.setup.outputs.container_tag }}
+ options: *linux_container_options
timeout-minutes: 60
env: &linux_env
@@ -558,7 +563,9 @@ jobs:
needs: [setup, sanity-check]
if: *linux_job_if
runs-on: ${{ needs.setup.outputs.runs_on_linux }}
- container: *linux_ci_container
+ container:
+ image: ${{ needs.setup.outputs.container_linux_ci_base }}/linux-meson-32:${{ needs.setup.outputs.container_tag }}
+ options: *linux_container_options
timeout-minutes: 60
env: *linux_env
@@ -650,7 +657,9 @@ jobs:
needs: [setup, sanity-check]
if: *linux_job_if
runs-on: ${{ needs.setup.outputs.runs_on_linux }}
- container: *linux_ci_container
+ container:
+ image: ${{ needs.setup.outputs.container_linux_ci_base }}/linux-meson-64:${{ needs.setup.outputs.container_tag }}
+ options: *linux_container_options
timeout-minutes: 60
env: *linux_env
@@ -1194,7 +1203,8 @@ jobs:
runs-on: ${{ needs.setup.outputs.runs_on_linux }}
timeout-minutes: 60
container:
- image: ${{ needs.setup.outputs.container_linux_ci_docs }}
+ image: ${{ needs.setup.outputs.container_linux_ci_base }}/compiler-warnings:${{ needs.setup.outputs.container_tag }}
+ options: *linux_container_options
env:
# Use larger ccache cache as this job compiles with multiple
# compilers / flag combinations.
--
2.54.0.450.g9ac3f193c0
view thread (118+ messages) latest in thread
Message-ID: <iggjozfshwbqpv33x5jqwtju5k5zrkyu3257dlifxkhtpg7eoq@k2ccyrdi5dtu>
Permalink: ../iggjozfshwbqpv33x5jqwtju5k5zrkyu3257dlifxkhtpg7eoq@k2ccyrdi5dtu/
Also on: postgresql.org/message-id/iggjozfshwbqpv33x5jqwtju5k5zrkyu3257dlifxkhtpg7eoq@k2ccyrdi5dtu
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: andres@anarazel.de, byavuz81@gmail.com, jacob.champion@enterprisedb.com, peter@eisentraut.org, postgres@jeltef.nl, thomas.munro@gmail.com, zsolt.parragi@percona.com
Subject: Re: Heads Up: cirrus-ci is shutting down June 1st
In-Reply-To: <iggjozfshwbqpv33x5jqwtju5k5zrkyu3257dlifxkhtpg7eoq@k2ccyrdi5dtu>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox