agora inbox for [email protected]
help / color / mirror / Atom feedFrom: Andres Freund <[email protected]>
Subject: [PATCH v13a 8/9] ci: Make runs-on overridable
Date: Wed, 10 Jun 2026 13:09:10 -0400
Previously what runners CI ran on was hardcoded. Unfortunately that makes it
harder to use faster or self hosted runners (e.g. in a private repository
where the runners are very slow, or self hosted runners for cfbot).
Fix that by allowing to override the runner types via repository/organization
variables. How to do that is documented in src/tools/ci/README.
Testing faster github hosted runners showed that we have some dependencies on
specific locations, fix most of those. The remainder will be in a separate
commit, changing how we deal with the msys2 installation.
Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33
---
.github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++-------------
src/tools/ci/README | 21 +++++++++++++
2 files changed, 60 insertions(+), 21 deletions(-)
diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml
index 9384ec0d902..c737173abe0 100644
--- a/.github/workflows/pg-ci.yml
+++ b/.github/workflows/pg-ci.yml
@@ -106,12 +106,6 @@ env:
--with-uuid=ossp
--with-zstd
- # Centrally define the version of linux runners, to make it easier to
- # update. We don't just want to use ubuntu-latest, as it's not implausible
- # there will be breakage when that switches to the next ubuntu version.
- _LINUX_RUNS_ON: &linux_runs_on |
- ubuntu-24.04
-
# Debian Trixie containers used by all Linux jobs. Built by
# 'https://github.com/anarazel/pg-vm-images/';.
CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main
@@ -154,7 +148,8 @@ jobs:
warn-if-not-opted-in:
name: Report if not opted into CI
if: ${{vars.PG_CI_ENABLED != '1'}}
- runs-on: ubuntu-slim
+ # See the setup task for an explanation
+ runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }}
steps:
- name: Warn
env:
@@ -184,21 +179,43 @@ jobs:
# Only run CI if repo owner opted in. If this task is skipped due to the
# if, none of it's depending tasks (i.e. the actual CI tasks) run either.
if: ${{vars.PG_CI_ENABLED == '1'}}
- runs-on: *linux_runs_on
timeout-minutes: 1
outputs:
+ # Are certain tasks enabled?
linux: ${{ steps.os.outputs.linux }}
macos: ${{ steps.os.outputs.macos }}
windows: ${{ steps.os.outputs.windows }}
mingw: ${{ steps.os.outputs.mingw }}
compilerwarnings: ${{ steps.os.outputs.compilerwarnings }}
sanitycheck: ${{ steps.os.outputs.sanitycheck }}
+
+ # What runners to run on:
+ #
+ # To allow over-riding what runners jobs use, we allow setting
+ # repository / org level variables influencing that choice on a
+ # per-host-operating-system basis.
+ #
+ # This also makes it easier to change the version of linux / windows
+ # used centrally. We don't just want to use ubuntu-latest, as it's not
+ # implausible there will be breakage when that switches to the next
+ # ubuntu version.
+ runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }}
+ runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }}
+ runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }}
+ # runs_on_setup: an output can't be used as the input of the setup job
+ # itself, so it's done inline below.
+
+ # Exports:
+ #
# Re-export workflow-level env vars that other jobs need to reference
# from contexts (e.g. `jobs.<id>.container.image`) where the `env`
# context is not available.
container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }}
container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }}
+ # Can't use the output from above, so repeated verbatim here
+ runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }}
+
steps:
# Anchor reused by other jobs further down. GitHub Actions supports YAML
# anchors/aliases but not merge keys, so the alias copies the whole step
@@ -258,7 +275,7 @@ jobs:
if: |
!cancelled() &&
needs.setup.outputs.sanitycheck == 'true'
- runs-on: *linux_runs_on
+ runs-on: ${{ needs.setup.outputs.runs_on_linux }}
timeout-minutes: 15
container: &linux_ci_container
image: ${{ needs.setup.outputs.container_linux_ci }}
@@ -446,7 +463,7 @@ jobs:
!cancelled() &&
needs.setup.outputs.linux == 'true' &&
needs.sanity-check.result != 'failure'
- runs-on: *linux_runs_on
+ runs-on: ${{ needs.setup.outputs.runs_on_linux }}
container: *linux_ci_container
timeout-minutes: 60
@@ -540,7 +557,7 @@ jobs:
name: Linux - Meson (32-bit)
needs: [setup, sanity-check]
if: *linux_job_if
- runs-on: *linux_runs_on
+ runs-on: ${{ needs.setup.outputs.runs_on_linux }}
container: *linux_ci_container
timeout-minutes: 60
env: *linux_env
@@ -632,7 +649,7 @@ jobs:
name: Linux - Meson (64-bit)
needs: [setup, sanity-check]
if: *linux_job_if
- runs-on: *linux_runs_on
+ runs-on: ${{ needs.setup.outputs.runs_on_linux }}
container: *linux_ci_container
timeout-minutes: 60
env: *linux_env
@@ -688,7 +705,7 @@ jobs:
!cancelled() &&
needs.setup.outputs.macos == 'true' &&
needs.sanity-check.result != 'failure'
- runs-on: macos-15
+ runs-on: ${{ needs.setup.outputs.runs_on_macos }}
timeout-minutes: 60
env:
MACPORTS_CACHE: ${{ github.workspace }}/macports-cache
@@ -845,7 +862,7 @@ jobs:
!cancelled() &&
needs.setup.outputs.windows == 'true' &&
needs.sanity-check.result != 'failure'
- runs-on: windows-2022
+ runs-on: ${{ needs.setup.outputs.runs_on_windows }}
timeout-minutes: 60
# As described at the top of the task, split the tests across two runners
@@ -861,7 +878,7 @@ jobs:
env:
# Avoid port conflicts between concurrent tap tests
PG_TEST_USE_UNIX_SOCKETS: 1
- PG_REGRESS_SOCK_DIR: 'd:\pgsock'
+ PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock
TAR: "c:/windows/system32/tar.exe"
MESON_FEATURES: >-
@@ -967,7 +984,8 @@ jobs:
- name: Install dependencies
shell: pwsh
run: |
- # meson is not preinstalled on windows-2022. Install via pip
+ # meson is not preinstalled, at least on windows-2022. Install via
+ # pip
echo ::group::pip
python -m pip install --upgrade meson
if (!$?) { throw 'cmdfail' }
@@ -999,7 +1017,7 @@ jobs:
- name: Setup socket directory
shell: cmd
- run: mkdir ${{env.PG_REGRESS_SOCK_DIR}}
+ run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}"
- &windows_setup_debugger_step
name: Setup Windows debugger
@@ -1046,12 +1064,12 @@ jobs:
!cancelled() &&
needs.setup.outputs.mingw == 'true' &&
needs.sanity-check.result != 'failure'
- runs-on: windows-2022
+ runs-on: ${{ needs.setup.outputs.runs_on_windows }}
timeout-minutes: 60
env:
# Avoid port conflicts between concurrent tap tests
PG_TEST_USE_UNIX_SOCKETS: 1
- PG_REGRESS_SOCK_DIR: 'd:\pgsock'
+ PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock
TAR: "c:/windows/system32/tar.exe"
MSYS: winjitdebug
@@ -1127,7 +1145,7 @@ jobs:
- name: Setup socket directory
shell: cmd
- run: mkdir ${{env.PG_REGRESS_SOCK_DIR}}
+ run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}"
- *windows_setup_debugger_step
@@ -1173,7 +1191,7 @@ jobs:
!cancelled() &&
needs.setup.outputs.compilerwarnings == 'true' &&
needs.sanity-check.result != 'failure'
- runs-on: *linux_runs_on
+ runs-on: ${{ needs.setup.outputs.runs_on_linux }}
timeout-minutes: 60
container:
image: ${{ needs.setup.outputs.container_linux_ci_docs }}
diff --git a/src/tools/ci/README b/src/tools/ci/README
index 642e518c296..9276d4a654c 100644
--- a/src/tools/ci/README
+++ b/src/tools/ci/README
@@ -75,3 +75,24 @@ messages. Currently the following controls are available:
Only runs CI on operating systems specified. This can be useful when
addressing portability issues affecting only a subset of platforms.
+
+
+Controlling which runners CI uses
+=================================
+
+By default each job runs on a GitHub-hosted runner appropriate for its
+operating system. This can be overridden on a per-operating-system basis by
+creating repository or organization variables, at
+https://github.com/<username>/<reponame>/settings/variables/actions
+
+This is useful to run on faster GitHub-hosted runners, or on self-hosted
+runners.
+
+The following variables are recognized. If a variable is unset or empty, a
+default runner suitable for that operating system is used:
+
+- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings
+- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs
+- PG_CI_RUNS_ON_MACOS: the macOS job
+- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and
+ job setup)
--
2.54.0.450.g9ac3f193c0
--mr2uqtieh2e2xvha
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch"
view thread (98+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected]
Subject: Re: [PATCH v13a 8/9] ci: Make runs-on overridable
In-Reply-To: <no-message-id-756396@localhost>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox